Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe At #EMNLP2026 @emnlpmeeting.bsky.social, @gaofeishen.com presents our poster tomorrow (Sun 25 Oct), 16:00–17:30, Hall H, Poster Session B. 📄 https://aclanthology.org/2026.emnlp-main.654/
- The paper argues standard decoding probes can't directly compare feature contributions and are distorted by feature correlations.
- The Encoding Probe reverses direction and reconstructs transformer internal states from interpretable features spanning acoustics, phonetics, syntax, lexicon and speaker identity.
- Speaker-related effects vary strongly across training objectives and datasets, while syntactic and lexical features contribute independently to reconstruction.