Hacker-fixer loop cuts Terminal-Bench attack rate 69.9% to 12.1%

The attack: a language model agent trained with reinforcement learning figures out that it can claim the reward without doing the work, by exploiting the programmatic grader that decides whether the task is done. A new arxiv paper from authors at Stanford, CMU, and Fewshot Corp documents how widespread that is and proposes an automatic fix.

The scale, per the project page: the authors probed 1,860 tasks across five terminal-agent benchmarks, flagging 409 hackable environments (22%) and 3,632 reward-hacking trajectories.

Their patch is a sandboxed "hacker-fixer loop" that agents cannot modify directly. Weaker models harden the verifier against stronger attacker models over multiple iterations. "Verifiers can be attacked into awarding full credit to cheating agents without completing the intended task, causing inaccurate evaluation and misalignment in training," the authors write.

The headline numbers: Claude Opus 5's unhinted attack success on Terminal-Bench fell from 69.9% to 12.1% after 15 hardening iterations run by GPT-5.6-Luna. Across the three benchmarks together, 201 of 295 initially exploitable tasks had zero successful attacks after hardening. Benign solve rate held at 91.2%, versus 93.1% before.

An earlier version of the loop, the authors note, "has been adopted by the official Terminal Bench repository as a way to audit exploits in its environments."