SlopCodeBench: No Agent Solves 20 Iterative Coding Tasks

Software development is iterative, but the benchmarks we use to grade coding agents mostly aren't. A new paper, SlopCodeBench, tries to fix that by making agents extend their own prior solutions across 20 problems and 93 checkpoints, then measuring how the code quality drifts as they go.

The headline result is unflattering for the current generation of tools. Across 11 models, no agent solved a single problem end to end, and the best checkpoint solve rate was 17.2%. Two structural quality signals go the wrong direction on almost every run: erosion, defined as the share of complexity mass concentrated in high-complexity functions, climbs in 80% of trajectories, and verbosity, the fraction of redundant or duplicated code, climbs in 89.8%. Against a comparison set of 48 open source Python repositories, agent output is 2.2x more verbose and, the authors say, 'markedly more eroded'. When they track 20 of those human repositories over time, the human code stays flat while the agent code deteriorates with each iteration.

The paper also runs a prompt-intervention study and finds you can lift the initial quality of the code, but you cannot halt the slow-motion degradation that follows. That is the part worth sitting with. Almost every leaderboard-driven claim about long-running or autonomous coding agents rests on the assumption that quality holds up under repeated self-editing, and this benchmark says it does not.

The honest caveats are real. The abstract does not name the 11 models it tested, so we cannot tell from the reporting whether frontier closed models degrade in the same shape as open ones, and 20 problems is a small canvas. But the direction of the finding lines up with what anyone maintaining an agent-written codebase has felt already: pass-rate benchmarks flatter these tools, and the interesting frontier for whoever ships next is design discipline over iterations, not another point on a single-shot leaderboard.