OpenForgeRL Trains Agents Directly in Their Real Harnesses

For anyone trying to do open-source research on the kinds of agents that actually get deployed, there has been a stubborn mismatch. The interesting agents live inside elaborate inference harnesses like Claude Code, Codex, and OpenClaw, but the standard SFT and RL stacks the research world uses were not built to express what those harnesses do. A new paper on arXiv called OpenForgeRL is a bid to close that gap.

The trick, per the abstract, is to stop trying to reimplement the harness inside the training loop and instead sit next to it. OpenForgeRL puts a lightweight proxy between the harness and the model, serving the harness's model calls in the normal way while quietly recording them as training data for a standard RL codebase like veRL. A Kubernetes orchestrator then runs each rollout in its own remote container, so many rollouts can happen in parallel across whatever environments the harness needs. Training and inference stay decoupled, which is what makes the authors' "any harness in any environment" claim tractable.

They validate on two settings. OpenForgeClaw, the tool and claw-based variant, reaches 31.7 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI, the browser and computer-use variant, hits 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. The authors say both outperform open baselines of similar size on nearly all benchmarks, and in the GUI case match or surpass models several times larger, using only hundreds to a few thousand tasks.

The honest caveat is that Claude Code and Codex are named as motivating examples of this harness class, but the paper's actual runs are on tool/claw agents and GUI agents, not on the closed commercial products themselves. What the reporting does not give you is any head-to-head against those closed agents on their own turf, any cost figure for standing up the Kubernetes rollout fleet, or a sense of how the proxy behaves when a harness's call pattern changes between versions.

Still, if the framework holds up outside the authors' setups, the interesting shift is who gets to do serious agent RL. Until now the people best positioned to train an agent end-to-end inside a real harness were the labs that owned the harness. A proxy-plus-container recipe that plugs into veRL is exactly the kind of infrastructure that lets a university group or a small startup enter that conversation.