Google open-sources EnvHarness, +9pt lift on agent benchmarks

Google Cloud AI Research has open-sourced EnvHarness, a wrapper that reshapes static agent-training benchmarks without altering their verification code, and reports up to a 9.0-point gain on held-out instances across five benchmarks in four domains.

The paper describes the system as "a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic," designed so that "every reshaped environment retains its original verifier."

On the concrete benchmarks, Crypto Briefing reports, ALFWorld climbs from 62.4% to 68.3%, with a 9.0-point jump to 70.4% on out-of-distribution tasks. On SWE-bench Verified, EnvHarness scores 54.79 against 52.13 for the original environment and 50.37 for a generative-environment baseline. The paper reports 9.8% fewer execution steps.

The companion tool, EnvRigger, "treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts." Code is at google-research/envharness on GitHub.

The team, from Google Cloud AI Research, Washington University in St. Louis and UNC Chapel Hill, frames the release around the RL training loop rather than any single leaderboard win, arguing EnvHarness "provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment."