paper web signal

DarwinX evolves agent harnesses to 93% on WebArena, model frozen

TL;DR

  • DarwinX evolves a population of agent harnesses (prompts, tools, skills, control flow) using each benchmark's own verifier as fitness, with the model frozen.
  • WebArena-Infinity real-task pass@1 rises from 43.5% to 93.0%; Terminal-Bench 2.1 climbs 7.7 points to 83.2%, with roughly 17-point average gains across four benchmarks.
  • A Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified, which the authors frame as evidence of general agent competence rather than benchmark-specific patching.

DarwinX pushes a frozen LLM agent's WebArena-Infinity real-task pass@1 from 43.5% to 93.0% by evolving the harness around the model rather than retraining the weights, according to a new arXiv paper.

The method treats prompts, tools, skills, and control flow as "a population of harnesses with the model frozen." Selection uses each benchmark's own verifier for fitness, with "no gold solutions, no hand-picked winners," and a preserve-and-extend contract admits only variants that extend coverage without regressing on earlier tasks. An archive keeps alternative lineages so later generations can recombine from them.

Across four benchmarks the authors report "one loop adds about 17 points on average." Terminal-Bench 2.1 rises 7.7 points to 83.2% on a matched base and 84.7% on a stronger one. TerminalWorld's held-out split reaches 68.3%, which the paper describes as "ahead of every off-the-shelf agent." A Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified.

The abstract's own framing: "A frozen model need not be a fixed agent: harness selection turns evaluation compute into durable capability." It does not disclose the compute budget for one evolutionary loop, the harness population size, or per-benchmark run counts, so the cost side of the trade is not on the page.