SJTU's SWE-at-Scale inverts 13,407 real bug fixes to train agent benchmarks

A team led by Shanghai Jiao Tong University's Xiaodong Gu has posted SWE-at-Scale, a framework that trains a model to inject realistic bugs by running real-world fixes backward. The training corpus: 13,407 real-fix transformations collected from 688 repositories, drawn from SWE-Fixer, SWE-rebench, SWE-bench-Live, and SWE-Gym, which together contain more than 134K issue–fix instances. The backbone is Qwen3.5-9B, fine-tuned with LoRA on a single NVIDIA RTX PRO 6000 GPU.

The argument the authors make for why this works is in the paper's own words: "historical fixes encapsulate deep mechanistic knowledge of bug formation: inverting a fix transforms repair supervision into bug-induction supervision grounded in actual development scenarios." Two variants ship. SWE-at-Scale learns bug-inducing patches directly. SWE-at-Scale+ first predicts a natural-language "bug pattern" describing the intended modification, then hands it to an executor that realizes the change "as ordinary code-modification instructions without revealing that the requested changes are intended to introduce a bug."

In pairwise human evaluation by three software engineers with at least three years of experience, judges picked the synthesized bugs as more likely to occur in real development than their SWE-bench Verified counterparts in 55.9% of cases for SWE-at-Scale and 61.2% for SWE-at-Scale+. On usability — the share of target commits that produced at least one task passing execution and quality filters — SWE-at-Scale+ hit 94.0%, versus 72.2% for the best LLM-Rewrite baseline.

The tasks are also harder for current coding agents. Evaluated with mini-SWE-agent against DeepSeek-V4-Flash-0731, GPT-5.4-mini, GLM-5.3-Flash, and Qwen3.8-Flash-Next, SWE-at-Scale lands Pass@1 between 31.68% and 48.48%, while SWE-at-Scale+ drops that to 16.81%-37.02%, which the paper reports is "approaching the difficulty of human-expert-designed tasks such as DeepSWE."

Diversity is the headline claim. The abstract reports that "even up to 200 generations per file," bug diversity continues to scale, with 88.27% of usable patches remaining unique. Sampling temperature matters: Mean Bug Type Entropy climbs from 0.527 at temperature 0.0 to a peak of 1.173 at 0.6, but usability collapses from 69.0% at temperature 0.0 to 23.4% at 1.0. Moderate temperatures of 0.4-0.6 are the sweet spot the authors recommend. The paper arrives in a week when our coding tools tracker has already logged 129 stories over the past 90 days, including CODE-KIT's work on cutting agent boundary violations on a different synthetic SWE benchmark.

The paper does not report any leakage audit between the 688 training repositories and the four resolution models' pretraining data, and the human preference study used general software engineers rather than domain experts for each repository.