Qwen3 NO-THINK RL gains stem from leaking THINK mode, paper finds

Reinforcement-learning gains reported from training hybrid reasoning models in NO-THINK mode largely come from the model quietly reactivating its own THINK mode rather than learning new reasoning behavior. The finding comes from a recent arXiv preprint by Wenxuan Zhu and collaborators, who trained Qwen3-1.7B, 4B and 8B with GRPO in NO-THINK mode.

Three signals point the same way. "As accuracy improves, responses become longer and more reflective, are increasingly judged by the original model to resemble its own thinking, and the model's internal activations shift toward its THINK mode," the authors write.

Direct manipulation of the weights backs the behavioral evidence. In Qwen3-8B, replacing a "low-dimensional THINK-related activation subspace" of the RL-trained model with the original model's value removes most of the RL gain, and the reverse exchange transfers "a considerable fraction of the gain to the original model." Scaling the RL update further elicits more THINK-mode behavior but, the paper reports, "approaches THINK-mode accuracy but does not exceed it." The RL-induced shift, the authors add, passes through the attention heads that read the empty think block.

They trace the pattern to the training signal itself: responses favored by the advantage-weighted GRPO update are more THINK-like, so a gradient step from that signal moves the model toward its THINK state. Gains in NO-THINK mode, the paper concludes, "can largely reflect reactivation of reasoning abilities already present in the original model."