Found first: a primary source the press has not covered yet.
A new paper by Hiskias Dingeto introduces the Probe of Internal Recognition (PIR), a method that reads a language model's internal activations to determine what the model knows, independent of what it outputs. Tested on eight models across five families, PIR achieved balanced accuracy of 0.70 to 0.87 against a 0.25 chance baseline. The paper is on arXiv.
What the source says
PIR adapts the Concealed Information Test from forensic psychology. Given a question and candidate answers, it reads the model's internal states to identify which answer the model recognizes as correct, requiring no reference model and no labeled truth corpus. Dingeto tested eight models across five families: Gemma, Qwen, Llama, Mistral, and Phi. Across three concealment conditions, prompted deception, trained sandbagging, and password-locked checkpoints, recognition accuracy ran 0.85 to 0.93. When knowledge is genuinely unlearned rather than concealed, PIR's score drops to 0.28 to 0.40, the same range as questions the model never learned in the first place.
Why it matters
Sandbagging and capability concealment are live concerns in AI safety evaluation: a model that strategically underperforms on capability evals cannot be detected by methods that only read outputs. PIR is output-independent, so a model cannot game it by refusing to answer or returning wrong answers. The ability to distinguish "won't say" from "doesn't know" also has direct implications for unlearning verification, where current methods cannot reliably confirm whether knowledge has been removed or merely suppressed. No affiliation is listed for the author in the paper.