LLM agents voice agreement before internal states catch up

When two language models hash out a subjective disagreement across many turns, a probe pointed at any single conversation does not tell you much. The internal 'agreement' signal lurches around turn to turn and, by the authors' own measurement, 'does not reliably track external agreement at the individual conversation level.' Pool the same probe across many conversations, and a tidy structure appears.

In a preprint posted to arXiv, Adi Simhi, Yaniv Nikankin, Jonathan Herzig, Idan Szpektor, Yonatan Belinkov and Shay B. Cohen train a linear classifier on activations from multi-turn dialogues in which two LLMs discuss subjective matters and may change their stances. At the single-conversation level, the agreement representation 'varies substantially across turns and conversations.' Aggregated across conversations, external agreement is well-approximated by a Markov process, with agreement at each turn 'a direct function of agreement at the previous turn.' The internal signal, aggregated the same way, maps onto that external process through a power-law.

The sharper observation sits at the end of the abstract: 'after a change of stance, models express agreement externally before their internal representations reflect the same shift.' The output flips first; the representation catches up. Two researchers we track posted the paper within days of its submission.

The authors frame the result as 'a joint structure between internal and external agreement that emerges only when internal features are aggregated across conversations.'