TIPS trains 4B process judge that outscores Claude-4.7-Opus
TL;DR
- TIPS-Qwen3-4B-Thinking-2507 reaches 85.2 F1 on ProcessBench using only 3.2K outcome-labeled trajectories, with no human step annotation.
- The 4B judge surpasses evaluated trained PRMs and prompt-only judges GPT-5.4-Instruct and Claude-4.7-Opus, while still trailing o1-mini.
- The reward signal is only whether the predicted final answer matches ground truth; step-level verification emerges from that alone.
A 4-billion-parameter model trained only on whether its final answer was right or wrong scores 85.2 F1 on ProcessBench, beating prompt-only judgments from GPT-5.4-Instruct and Claude-4.7-Opus but still trailing o1-mini. That is the headline result from "Inducing Process Supervision from Outcome-Only Reinforcement Learning", a paper by Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen, and Yankai Lin.
Process reward models usually need step-by-step human labels, which are the expensive part. The authors' method, TIPS (Thinking-Induced Process Supervision), skips that entirely. The model generates a chain-of-thought, then step-level labels, then an outcome label. The reward depends solely on whether that outcome matches the ground truth. "Outcome-only RL can therefore reinforce step-level verification without explicit process supervision," the abstract states.
The training set is 3.2K outcome-labeled trajectories; the backbone is Qwen3-4B-Thinking-2507. The paper reports the resulting judge surpasses "all evaluated trained PRMs" and the two named prompt-only judges on ProcessBench, though the abstract names only math and agent benchmarks and gives no per-category breakdown. Code and data are posted at RUCBM/TIPS.
Originally reported by paper
Read the original article →Original headline: Outcome-Only RL Trains a 4B Process Judge That Beats GPT-5.4-Instruct and Claude-4.7-Opus on ProcessBench