SWE-Trail history-guided collection lifts SWE-bench 2.33 pts
TL;DR
- SWE-Trail lifts a 124B student from Ling-3.0-flash-base by 2.33 points on SWE-bench Multilingual and 1.60 points on SWE-bench Pro versus independent sampling.
- Without the history-guided targeting, the same pipeline improved only 0.11 points on Multilingual.
- The authors argue exposing prior trajectories to the agent in context would create a train-test mismatch, since that advice is unavailable at inference.
A new preprint from Ziming Li and colleagues argues that for coding-agent post-training, trajectory history is best used to steer where data gets collected, not fed to the model as advice. In experiments posted to arXiv, a 124B student initialized from Ling-3.0-flash-base and trained on SWE-Trail trajectories improved mean resolution over independent sampling by 2.33 points on SWE-bench Multilingual and 1.60 points on SWE-bench Pro.
The authors motivate the design by what breaks when you do the obvious thing. "Training on such trajectories can make the model rely on advice that is unavailable at test time, creating a train-test mismatch," they write. SWE-Trail instead uses history to pick a state where more collection effort might help; the coding agent then "proposes candidate next actions from its ordinary context," and the collector briefly executes those candidates from the same checkpoint to decide which route continues.
The clean comparison lands the point: without the history-guided targeting, the same pipeline improved only 0.11 points on Multilingual. The abstract does not publish absolute resolution rates or the compute cost of the rollback step.
Originally reported by paper
Read the original article →Original headline: SWE-Trail: Using Git History to Guide Where to Collect Coding-Agent Training Data Lifts SWE-bench +3 Points