Transformers encode partner expertise early, act on it late

A transformer can figure out how expert its dialogue partner is in its early layers, then wait until past the middle of the network to do anything with that inference. That is the finding of a new preprint by Mika Okamoto and Gabriele Sarti, extending a known interpretability gap (attributes encoded early, used late) from things stated in the input to something the model has to infer gradually over a multi-turn conversation.

The setup is a corpus called ExpertCollab, containing "multi-turn research-planning dialogues between model-played personas at four expertise levels." Linear probes recover partner expertise best in early layers, and by the middle of the network the signal has decayed to near chance. Causally, the picture inverts. When the authors patch the expertise difference into activations at the layer where it is most readable, a fixed late-layer readout barely moves. Patch the same difference past the midpoint and it "propagates almost completely," what the paper describes as "a separation of more than an order of magnitude" between where the signal is legible and where flipping it flips behavior.

To rule out artifacts, the abstract reports a "content-matched random control and a probe-free diagnostic" that both locate the transition at the same early layer, while a statically specified control attribute stays decodable across the full network. The decay-then-act pattern is specific to the inferred relational attribute, not a probe or corpus quirk.

The caveats sit in the last line of the abstract itself. "We use one model on a synthetic corpus as an initial demonstration." The paper does not name that model in the abstract, does not publish the layer indices, and does not report per-condition accuracy numbers there. It also does not test whether the same pattern holds for other inferred relational attributes beyond expertise.