paper web signal

AEWM Beats Frontier Baseline by 10.6 Points on Action Judge

TL;DR

  • AEWM reaches 70.5% macro-F1 on the authors' Action Judge benchmark, 10.6 points above the strongest frontier baseline named in the abstract.
  • EditAct, which pairs Action Judge and State Revision with real execution, lifts average scores by 3.2–6.7 points across six benchmarks and three agent backbones.
  • Rejection-sampling fine-tuning on verified EditAct trajectories (AEWM-RFT) beats Self-RFT by 2.2–2.6 points across Search, Terminal, and Software Engineering, without online AEWM guidance.

The core claim, from the paper's own abstract on arXiv: "AEWM achieves 70.5% macro-F1 on our Action Judge benchmark, exceeding the strongest frontier baseline by 10.6 points."

The premise behind the Agent-Editing World Model is that reconstructing tool outputs, the traditional job of a language world model, is expensive and low-value when the real tool response is already available. So instead of simulating what a shell command or a search call would return, AEWM classifies the agent's own decisions.

Action Judge sorts each decision into one of three buckets: "Critical, Exploratory, and Noisy". State Revision then edits noisy reasoning-action continuations from the same observed history. Combined with real execution, the pair becomes EditAct, which the authors report "improves average scores by 3.2–6.7 points over the strongest baseline" across six benchmarks and three agent backbones.

Training spans three domains: Search, Terminal, and Software Engineering, via mid-training and supervised fine-tuning. A follow-on step, rejection sampling fine-tuning on verified EditAct trajectories, is called AEWM-RFT and "improves over Self-RFT by 2.2–2.6 points across three domains without online AEWM guidance."

The abstract does not name which frontier model serves as the baseline the 10.6-point gap is measured against, and it does not list the six evaluation benchmarks. Both would let outside teams place the number on a shared yardstick.