Builds on stodgier academic work here - www.science.org/stoken/autho... - this, by Roberts et al, is also very interesting on language and LLMs albeit much more political www.nature.com/articles/s41...
- Chinese state-media content appears in typical LLM training sets at roughly 41 times the rate of Chinese-language Wikipedia.
- Across 37 countries, models prompted in the local language produce more regime-favorable responses in countries with lower press freedom.
- A pretraining experiment with just 6,400 state-scripted documents pushed an open-weight model to pro-government responses nearly 80 percent of the time.