OpenForgeRL Trains RL Agents Through Real Inference Harnesses

There is a persistent gap between how open-source teams train AI agents and how those agents actually run in production. The harnesses that drive Claude Code, Codex, and things like OpenClaw manage stateful, multi-turn reasoning, tool use, and external system access, and open SFT/RL stacks have not really known how to speak that language. A new paper on Hugging Face from researchers at Columbia University, Dartmouth College, and Microsoft Research proposes closing that gap directly.

The framework, OpenForgeRL, is pragmatic in its design. A lightweight proxy sits in front of the harness, intercepts its model calls, and records them as training data for a standard RL codebase (they use veRL). A Kubernetes orchestrator, run on Azure, then executes each rollout in its own container, which is how the authors get around the fact that these harnesses cannot be co-located on training nodes at scale.

The reported numbers are the interesting part. The paper claims 31.7 pass@1 on ClawEval (with 55.9 at pass@3), 33.7 on QwenClawBench, 37.7 on OSWorld-Verified, and 72.3 on WebVoyager, saying the trained agents outperform open baselines of similar size on nearly all benchmarks and, in the GUI setting, match or surpass models several times larger. They also emphasize that only hundreds to a few thousand automatically curated tasks were needed, which, if it holds up, is a much lower data bar than the field has been assuming.

The honest caveat is baked into their own analysis. The authors say RL improves tool coverage, self-verification and multi-step task completion, but they also report that error recovery remains weak, which is exactly the failure mode that decides whether an agent survives contact with a real user. Benchmark pass@1 numbers do not settle that, and the specific Azure-Kubernetes setup is not something every team will find trivial to reproduce. There is also no head-to-head against closed frontier agents in what has been released so far.

What is worth watching is the direction. If the code, data and weights land as promised, small teams could start post-training their own agents inside the harness they actually deploy, rather than the simplified reimplementation they wish they were deploying. That closes a mismatch that has quietly hurt every open agent effort for the last two years.