Nature review maps four LLM roles across social-science work

Writing in Nature Human Behaviour, Fengli Xu, James Evans and colleagues from Tsinghua University, HKUST and Microsoft Research Asia argue that large language models now play four distinct roles in computational social science: subjects that simulate human behaviour, assistants that automate analysis, collaborators that generate hypotheses, and evaluators that audit and improve research quality.

Evans said on X that the review covers "where the field is fooling itself."

The survey runs through LLM-driven work in economics, psychology, political science, sociology and anthropology. It describes agent-based modelling with populations reaching millions of simulated actors, political-opinion and polarization simulations, and automated text-annotation and literature-screening pipelines. Two researchers we track shared the link on publication day.

Alongside the applications, the paper flags that LLMs struggle with causal inference and that their capacity to reproduce social complexity remains uneven and unproven. The authors warn that models can misportray identity groups, may exaggerate gender differences, and show a measurable preference for text generated by other LLMs, which becomes a concern as journals experiment with LLM-assisted peer review.

The recommendations include validation frameworks comparing LLM outputs with human behaviour, culturally diverse benchmarks, and transparency standards for LLM-assisted review.