CASIA's SWE-Touch: coding agents drop 7.7 points when users edit code

Every SWE-Bench headline you have read this year has assumed the same setup: the agent gets a clean repo, a task, and no interference from a human while it works. A new paper from the Chinese Academy of Sciences, posted on Hugging Face as SWE-Touch, argues that assumption is the problem. Real developers do not sit on their hands while an agent thinks. They open files, tweak lines, save things. The moment they do, the benchmark stops describing the product.

The authors, from CASIA's Institute of Automation, built a framework that mines task-critical code regions from repair trajectories, then uses a separate generator to construct plausible user edits that conflict with what the agent is trying to do. The Counter-Edits are validated to be realistic and task-conflicting, then injected mid-run with a short user message when the agent reaches the relevant code. Across nine frontier coding models on SWE-Bench Verified, average resolve rate falls by 7.7 percentage points. Open-source models degrade by as much as 16.5 points. Only Claude Opus 4.8 and GPT 5.5 stay resilient, and the drop persists on longer-horizon runs against SWE-Bench Pro and DeepSWE.

The most telling number is not the headline drop, it is the 63.3%. In nearly two thirds of failed runs, the agent simply left the user's conflicting code in place. It did not detect the workspace had changed, did not reconcile the conflict, did not re-validate affected behavior. The paper calls out four failure modes behind that pattern: limited workspace awareness, conflicting retention, inadequate re-inspection, and poor validation. Their reading is that optimizing for static leaderboards has not taught agents any of the collaborative behaviors an IDE actually needs.

The honest caveat is that Counter-Edits are synthetically generated, and the paper does not quantify how closely they match the 59% of real SWE-chat sessions where users touch code. It also does not tell you whether a workspace-awareness scaffold or targeted post-training can close the gap for open-source models. Those are the interesting follow-ups. For anyone building or buying coding agents right now, the useful takeaway is smaller and more concrete: if the pitch rests on a SWE-Bench Verified number, ask what happens when a developer types a line while the agent is thinking.