Princeton and Chicago: LLMs stereotype hires more than humans

A study out of Princeton and the University of Chicago, covered by MIT Technology Review and presented at ICML in Seoul in July, ran ChatGPT, Claude, and Gemini through a simulated hiring game and found the models stereotyped candidates more aggressively than humans did. Each model was told it had been hired as a consultant by the mayor of a fictional city and asked to help fill 20 jobs, including doctors, lawyers, child-care aides, and janitors, from four made-up ethnic groups called Tufa, Aima, Reku, and Weki. On a segregation scale where human participants scored 0.84, the models averaged roughly 65% higher, and OpenAI's o3 reasoning model hit 1.83.

The mechanism, according to Princeton PhD student and study coauthor Ryan Liu, is that LLMs "really are eager to create generalizations from limited data." A few early outcomes in the game got projected onto the whole group, so once one Tufa candidate was slotted into a role, the rest of the Tufa pool tended to follow. Newer reasoning-tuned models, including o3 and DeepSeek's R1, showed the effect more strongly than earlier versions, which cuts against the intuition that better reasoning ought to help.

If you are a company piloting LLMs for résumé triage or interview scheduling, the practical read is that the popular guardrail, a system prompt telling the model to be fair, barely moved behavior in this setup. What did move it: promising the model a bonus for diverse hiring, and giving it relevant personal information about candidates such as age and education. Irrelevant details like hair color or tattoos pushed the models back toward ethnic sorting.

The honest caveat is that this is a simulated game with invented group names, not a real applicant pool, so the segregation numbers should be read as a signal about how these models generalize under uncertainty, not as a measured lift in real-world discrimination. The reporting also does not say which production HR vendors currently ship on the reasoning models that scored worst, or how the incentive fix behaves outside a single-session game.

The forward-looking piece is that there is now a fairly clean audit method, a segregation score on a controlled hiring scenario, that a procurement team, a regulator, or an internal responsible-AI group could plausibly demand of any LLM-driven hiring tool before it is turned loose on real candidates.