Nature Human Behaviour maps four LLM roles in social science

A review in Nature Human Behaviour places large language models in four separate research roles: subjects that simulate human behaviour, assistants that automate analysis, collaborators that generate hypotheses, and evaluators that audit and improve research quality.

The piece is led by Fengli Xu, James Evans and colleagues at Tsinghua University, HKUST and Microsoft Research Asia, and it surveys sociology, political science, psychology, anthropology and economics. The authors argue LLMs 'accelerate discovery while creating new methodological paradigms,' and set the frame plainly: 'society itself becomes saturated with artificial intelligence agents—making LLMs both our research tools and authentic research subjects.'

Prompted with demographic or ideological profiles, the authors report, models 'reproduce survey distributions, economic games and polarization dynamics,' and AI interviewers deliver protocols 'with quality comparable to human interviews.' In qualitative coding, LLMs act as 'alien readers,' surfacing patterns a human coder would miss.

The review is blunt about the ceiling. 'LLMs struggle with causal inference and raise urgent ethical concerns about privacy and misinformation,' the authors write, 'and their ability to reproduce social complexity remains uneven and unproven.' They call the overall situation 'a recursive challenge that will define the future of social science.'

Two researchers in our tracker shared it the day it went up.