Self-Harness: LLM agents improve their own scaffolding

A paper posted to arxiv last month proposes something quietly interesting for anyone running LLM agents at scale: let the model rewrite its own harness. Self-Harness, from Hangfan Zhang and colleagues, sets up a three-stage loop where the agent mines its own failed execution traces, proposes concrete harness edits, and validates each edit against held-in and held-out task splits before accepting it. No human engineer in the loop, no stronger external model doing the tutoring.

The reported gains on Terminal-Bench-2.0 are the part that will make people look. MiniMax M2.5 went from a 40.5% to a 61.9% pass rate. Qwen3.5-35B-A3B went from 23.8% to 38.1%. GLM-5 went from 42.9% to 57.1%. The authors' framing is that the edits are model-specific and minimal, addressing a distinct failure mechanism each time rather than piling on generic instructions.

Why this matters if you are not writing agent papers: harnesses, the prompts, tool definitions, retry logic and scratchpad conventions that wrap a base model, are still mostly hand-engineered per model. That scales poorly as the model list keeps growing and every release behaves a little differently. A loop that produces interpretable, minimal edits gated by a regression test is at least a plausible route to automating the maintenance work that currently sits with an expert.

The honest caveat is that this is one benchmark and the safeguard is a regression test the agent itself is running. Terminal-Bench-2.0 is a narrow surface, and there is a real question about whether an autonomous edit loop can quietly drift toward gaming its own eval when you point it at messier production tasks. What the paper does not give you is compute cost of the loop, evidence that accepted edits compose safely over many iterations, or a read on whether the same trick lifts closed frontier models the way it lifts these open ones.

The interesting bet embedded in the work is that harness engineering is a legitimate axis of self-improvement, separate from training and separate from tool use. If that holds, the teams who benefit most are the ones running many models across many tasks who cannot afford to keep a human harness expert per model.