HKUST Traces 'Wait' in Reasoning Models to Data, Not RL
TL;DR
- 17 of 18 zero-RL runs kept 'wait' below 0.05 per 1k tokens while MATH-500 accuracy rose a median of 42.9 points.
- 13 of 14 models fine-tuned on reasoning traces picked up the marker, pointing to training data rather than RL as its origin.
- Under OLMo-3's RL recipe, the canonical 'Wait,' form share falls from 88.9% to 37.5% as the comma drops away.
'Wait' did not come from reinforcement learning; it came from the training data, according to a paper from HKUST, Huawei Noah's Ark Lab and Peking University that traced the token across 120 model endpoints.
The authors ran 18 zero-RL recipes on bases that did not already produce the word. In 17 of them, 'wait' stayed below 0.05 per 1k tokens even as RL raised MATH-500 accuracy by a median of 42.9 points. Fine-tuning on reasoning traces, by contrast, installed the marker in 13 of 14 cases.
'zero-RL almost never coins the marker,' the paper states; it 'arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it.'
Where a base already carries the word, the released RL-Zero recipes behave differently from their KL-free GRPO counterparts. OLMo-3 and MiMo multiply the marker's frequency while eroding its form: the canonical 'Wait,' share drops from 88.9% to 37.5% as the comma falls away, which the authors describe as 'a more sensitive but more liberal alarm.' The result lands amid a run of recent fine-tuning work also probing where reasoning behaviors come from.
The paper closes with a warning against reading too much into LLM-based revision metrics. Automated judges, it finds, can 'reverse human-labelled trends.'
Originally reported by huggingface.co
Read the original article →Original headline: HKUST Traces 'Wait' in Reasoning Models to Training Data, Not RL — 13 of 14 SFT Runs Inherit It