ActObs supervises observation tokens to boost agent pass@k

The claim is small and specific: apply the supervised fine-tuning loss to the observation tokens that agent trajectories already contain, and the policy explores better after reinforcement learning. No new data, no new parameters, no new forward passes.

"Although deployed agents never generate observations, learning to predict them encourages the policy to model action consequences without adding data, parameters, sequence tokens, or forward passes," the arXiv paper writes. The method is called ActObs.

The gap only opens after GRPO. Both variants look similar after SFT. On Terminal-Bench 2.0 with Qwen3-4B, ActObs beats the action-only baseline at every evaluated sampling budget. On Qwen3-8B it "trades some pass@1 reliability for higher pass@k (+3.4 pp at pass@16)" and solves more distinct tasks. On aider-polyglot code editing, whose tasks the models never see during SFT or RL, the 4B version gains +4.2 pp at pass@1.

The paper traces the effect back to what happens during SFT itself. With joint supervision, "action and observation gradients rapidly become orthogonal." Action-only training, by contrast, "leaves a large residual observation gradient and degrades environment prediction below the base model." The one-sided objective quietly damages the policy's model of consequences before RL begins.

After GRPO, ActObs "retains more entropy during RL while requiring less policy movement," leaving the final policy closer to its SFT initialization.

The abstract publishes no per-domain wall-clock figures, no results on model families outside Qwen3, and no comparison against explicit exploration bonuses.