Singh, Linzen, Ravfogel: LLM introspection claims unproven

Three researchers argue the growing literature claiming large language models can introspect has not cleared the bar for the claim. In a paper accepted at COLM 2026, Shashwat Singh, Tal Linzen and Shauli Ravfogel write that recent studies' conclusion 'may be premature,' and lay out two conditions they say an introspection test has to meet: privileged access, meaning the task 'should not be solvable using cues available in the input,' and second-order computation, requiring 'meta-representations of first-order, task-related representations.'

The authors then re-examine two paradigms others have used to claim introspection. In the first, models must predict labels derived from their own hidden states. 'Classifiers that can only access the input match the models' in-context predictions,' the paper reports. On that result, the models are not drawing on any hidden-state information the input does not already carry.

In the second, models are asked to notice when their internal states have been tampered with. The paper finds they cannot reliably separate that from manipulations of the input, suggesting 'their success reflects generic anomaly detection rather than sensitivity to internal interventions in particular.'

'Current evidence is insufficient to establish metacognitive monitoring in LLMs,' the authors conclude. The abstract names no specific model families or sizes. Two researchers we track shared the link within our directory.