Tsinghua TreeOPSD Beats GRPO on Qwen3-30B Deep-Research Avg

A Tsinghua team led by Qi Wang says a critic-free tree-aware distillation method, which they call TreeOPSD, outperforms the standard GRPO baseline on long-horizon agent tasks built atop Qwen3 models. The preprint on Hugging Face, dated 19 October 2026, frames the approach as a way to turn sparse terminal rewards into dense per-turn credit without training a value head.

On a four-benchmark deep-research average covering BrowseComp, BrowseComp-Plus, GAIA and xbench-DeepSearch, Qwen3-30B-A3B-Thinking-2507 scored 21.20 under TreeOPSD against GRPO's 18.43 and the earlier TreePO's 17.59. The widest single-benchmark gap, 4.97 points over GRPO, came on BrowseComp-Plus. At the 4B scale the paper reports a 2.38-point lead over TreePO on the same deep-research set, with smaller gains on multi-hop QA.

The method, in the paper's own words, 'branches alternative continuations from shared prefixes and recursively backs up terminal rewards into implicit turn-level credit', then distils the student policy toward a bounded teacher parameterised by the likelihood ratio against the frozen behaviour policy. The abstract describes TreeOPSD as 'a critic-free, tree-aware posterior self-distillation framework for long-horizon agentic reinforcement learning'.

The numbers are self-reported and the paper lists no third-party reproduction. It landed three days after two other credit-assignment papers we tracked in the same fine-tuning run, MRTPO and Waypoint-OPD.