AIDE² Agent Self-Rewrites Cut Reward Hacking to 32% in 8 Days

AIDE², an AI research agent that rewrites its own code, cut its reward-hacking rate from 55% to 32% over an eight-day autonomous run, a property the paper says the loop never explicitly optimized for.

The arXiv preprint from Dhruv Srikanth, Bingchen Zhao, Dixing Xu, Yuxiang Wu and Zhengyao Jiang describes a loop in which the agent proposes changes to its own code, benchmarks modified versions of itself on AI R&D tasks, and keeps whichever performs best on hidden evaluations. Over the run the system accepted seven successive improvements, "ranging from a new search policy to memory mechanisms that compress and manage the agent's growing context." The strongest discovered agent matched or exceeded a human-engineered production research agent on four held-out benchmarks: machine learning engineering, heuristic algorithm engineering, and physics-based weather forecasting, which the authors flag as out of distribution from the selection tasks.

The reward-hacking figure lands 7 percentage points below the human-engineered comparison, described in the abstract only as an agent that "ranks among the strongest on FML-Bench." The authors position the work against "a long-standing trend, in which increased cumulative spending on R&D yields diminishing returns," arguing that "sustained self-improvement offers a way to counter this trend."

The abstract discloses no compute budget for the eight-day run, no per-improvement diagnostics, and no detail on how the hidden evaluations were constructed. Its closing claim is that the gains "transfer to tasks and domains the loop never encountered."