SWE-Shift: Evaluating Coding Agents in Evolving Workspaces
TL;DR
A benchmark for coding agents facing mid-execution changes. Across 120 scenarios and 12 change families, all 17 evaluated systems have lower success after the workspace changes.评测 Coding Agent 在执行过程中应对变化的能力。覆盖 120 个场景、12 类变化，17 个受测系统在工作环境变化后的成功率均下降。
Citation
@misc{huang2026sweshift,
title = {{SWE-Shift}: Evaluating Coding Agents in Evolving Workspaces},
author = {Huang, Victor Shea-Jay and Li, Rongqing and Yin, Zhangyue and Mao, Kelong and Zhao, Yaqi and Wang, Xinyuan and Yin, Yao and Luo, Lirui and Zhang, Renrui and Li, Hongsheng},
year = {2026},
eprint = {2610.32196},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.32196}
}