Paper: leading LLMs skip code deletions, add guards instead

The most interesting bug in coding models this year is not a hallucination, it is a habit. A new arxiv paper, To Add Is Machine, To Delete Is Human, argues that leading LLMs systematically avoid removing code when a fix requires removal, and prefer to wrap the offending block in a conditional and call it done.

The numbers are the interesting part. Across the top SWE-bench models the authors tested, deletion recall against the developer patch tops out at 71.7%, and the split between file-level and line-level accuracy is the tell: models reach the right file for over 92% of required deletions but cut the exact line in under 52%. About 29.0% of passing patches use what the authors call Guard-and-Go, wrapping the targeted code in a guard or fallback rather than deleting it. And when the researchers retrofitted SWE-bench Verified tasks with tests that fail unless the code actually goes away, four frontier models spanning closed and open weights fell from 63.2% to 41.9% on the same tasks.

Why this matters if you are shipping AI-written patches: a green test suite is the checkpoint that lets teams merge model-generated fixes with less human review, and this result says the checkpoint is not measuring what people think it is. A vulnerability that was meant to be removed can survive as a code path guarded by a flag, still reachable, still there. On a purpose-built benchmark the authors call CanItDelete, the best model still fails one task in five, and smaller open models fall to 18.0%.

The honest caveat is that a lot is not settled by this paper. It does not name which frontier models sit in the four-model retrofit, does not quantify how often Guard-and-Go patches later break in production, and does not test popular agentic coding assistants end to end. Even supplying the exact lines to remove in the prompt only lifts success to 80.5%.

The forward-looking piece is more optimistic. The authors' read is that the behavior is undertrained rather than beyond reach, which points to a training-data and post-training fix rather than a model-scale one. That is a real opening for anyone building coding tools or eval suites: add a check that the target line actually disappeared to the review pipeline, and treat deletion recall as a first-class metric alongside pass rate.