Pre-2019 LLMs reproduce 84% of COVID partisan splits

A sociology paper posted to arXiv makes a claim worth sitting with: the way Americans split on COVID-19 was largely predictable from what people were already saying before the pandemic. Austin C. Kozlowski, Hyunku Kwon, and James A. Evans train a large language model only on text published through 2019, then ask it to answer pandemic-related questions as either a simulated liberal or a simulated conservative American. Their paper reports that the model reproduces the observed partisan differences in COVID-19 attitudes in 84% of cases, which the authors describe as significantly greater than chance.

The framing is what makes this interesting beyond the number. The authors argue that LLMs can "serve as a valuable tool for sociological inquiry" by simulating respondents from specific social and cultural contexts, treating the model less as a chatbot and more as a compressed record of a prior discourse. When they prompt the simulated respondents to justify their answers, the partisan gaps map onto differing appeals to freedom, safety, and institutional trust. The paper concludes that this event "served to advance history along its track rather than change the rails."

If the method holds up, the practical use is straightforward. Pollsters, campaigns, and public-health communicators could pilot messages against simulated 2019 respondents before spending on real fieldwork, or run counterfactuals on events for which no survey data exists.

There are things this abstract does not settle. It reports an aggregate 84% match on group-level attitudes, not evidence that any individual simulated respondent tracks a real one, and it does not name the base model, the exact question battery, or the specific baseline the "significantly greater than chance" claim is measured against. A method that works when the ideological grooves are already carved may say little about a genuinely novel issue with no 2019 analog. Still, if simulated respondents keep matching real ones at this rate on other topics, sociology gets a new instrument, and anyone who buys survey data gets a cheaper first draft.