Multilingual LLMs share grammar circuits across 29 languages

Multilingual language models appear to reuse the same internal machinery for subject-verb agreement in languages that mark it overtly, according to a new arxiv preprint accepted to COLM 2026. The authors (Isabella Gidi, Antonio Almudévar, Core Francisco Park, Naomi Saphra, Ricard Marxer) ran activation patching and attention analysis across 29 languages and five open-source model families, identifying the attention heads causally implicated in agreement.

The headline finding: "languages with overt person/number inflection exhibit more similar agreement circuitry than non-conjugating languages, with the strongest sharing appearing when the analysis isolates recovery of the inflectional contrast itself."

English sits as a bridge case, the paper says, becoming more similar to conjugating languages "precisely in contexts where overt agreement is required." The implicated heads also show similar attention patterns across languages, which the authors read as evidence of "shared functional roles as well as shared localization," not just overlapping position. Two of the researchers we follow posted the arxiv link the same week it went up.