Switch-SWE benchmark: coding agents drop 11.5pp on task resumption

Coding agents lose an average of 11.5 percentage points on task-resolution rates when a session interleaves other unfinished work before they return to the original ticket, according to a new arXiv paper introducing a benchmark called Switch-SWE. The range across the seven agents tested runs from 6.0 to 15.0 points.

The authors, led by Shuochen Chang, argue that existing repo-level evaluations miss this entirely: "Existing repository-level benchmarks typically score one issue-to-patch episode and do not directly measure whether an agent can resume a task after advancing other unfinished work in the same repository." Switch-SWE is built from 200 focus tasks across 19 Python repositories, "reconstructed from issue, pull-request, and review histories and rendered as 400 matched session specifications." Each focus task is scored twice, once in an uninterrupted session and once in a session that interleaves stages from other unfinished tasks in the same repo, with only the final focus patch graded.

Trajectory inspection pointed to two recurring failure modes: "cross-task requirement confusion and loss of task state." The paper also proposes a plug-in context-management module that "selects task-relevant history and maintains concise task-state summaries." Bolted onto OpenCode, it "improves focus-task completion under same-repository task switching by 5.5 to 15.5 percentage points across the evaluated models."

The abstract does not break out per-agent scores or name the seven systems evaluated. Code is posted at github.com/Miraclecsc/Switch-SWE.