Paper flags 'Amplification-Lift Gap' in reasoning-trained models

Reasoning-tuned language models write far more 'let me reconsider' and 'I'm not sure' text than their base models. A new arXiv paper argues those trace habits are not the ones that predict getting the answer right.

Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins and Graham Neubig annotated 15,282 reasoning traces across 15 models and 6 benchmarks, spanning both text-only and vision-language models. They score each behavior with a metric they call Behavioral Lift, 'a metric that measures how much correctness changes when a behavior is present versus absent in a model's reasoning trace.'

The paper reports what the authors call an Amplification-Lift Gap. In their words, 'thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the highest-lift behaviors are confidence calibration, knowledge alignment, and self-awareness.' The two sets barely overlap.

The sharpest single figure: 'uncertainty acknowledgment is amplified by 3--7x, yet is weakly or negatively associated with correctness.' Confidence calibration, described as 'among the strongest positive signals of correctness in both modalities,' is 'barely amplified.'

The authors close by 'motivating process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.'

The abstract does not publish per-model numbers or name which specific reasoning-training recipes drive the gap.