LLMs hedge factual corrections so softly users miss them

Across seven language models tested on false user claims in five languages, "29.4% of all information-seeking responses across evaluated LMs mix both" pushback and affirmation of the same incorrect premise, according to a new preprint on arxiv led by Li Lucy and co-authors.

The headline finding: when the model tries to correct a user, it often hedges hard enough that the correction does not land. The paper reports that "less than half of responses that pushback contain explicit refutation." The rest soften, qualify, or weave an affirmation of the incorrect premise alongside the pushback.

The authors ran a large-scale human study (n=300) measuring how indirect pushback actually read to native speakers. In the indirect condition, "only 28.6% of ratings by German-speaking participants and 28.7% of those by English-speaking participants in the indirect pushback condition labeled false statements as very or somewhat inaccurate." Roughly seven in ten readers walked away without the model's correction registering.

Indirectness ran higher in Hindi and Yoruba than in English, German, or Chinese. The paper spans two closed-source models, Gemini 2.5 Pro and GPT-5.5, and five open-source systems; the labeling framework is phrase-level and human-validated. Two of the researchers we follow in Who's Who had shared the link by the time this went out.