EMNLP paper: LLMs lean on stereotypes to explain misogyny

A paper out of EMNLP 2024 keeps rattling around in my head because it names a failure mode that safety and moderation teams keep papering over. When you ask a large language model to explain why a piece of text is misogynistic, it does not actually reason its way there. It reaches for the stereotype. The paper by Muti, Ruggeri, Al Khatib, Barrón-Cedeño and Caselli frames misogyny detection as an argumentative reasoning task, then tests whether zero-shot and few-shot prompts built on argumentation theory can coax LLMs into filling in the implicit link between a comment and its misogynistic meaning.

The finding, in the authors' own words, is that LLMs 'fall short on reasoning capabilities about misogynistic comments' and 'mostly rely on their implicit knowledge derived from internalized common stereotypes about women to generate implied assumptions, rather than on inductive reasoning.' They ran the evaluation in Italian and English, and paired chain-of-thought prompting with augmented knowledge to give models the best shot at doing real inference. Models still leaned on the stereotype rather than on the argument.

Why this matters if you are shipping moderation or content classifiers: the useful part of a model's explanation is meant to be the reasoning trace, the thing a reviewer can audit. If that trace is actually stereotype retrieval dressed up as inference, an 'explainable' classifier is misleading the people relying on it. It looks like careful analysis and reads like a prior.

The honest caveat is that this is one paper, on two languages, and the line between 'reasoning' and 'stereotype retrieval' is a judgment call the authors make from within their own framework. What the reporting does not give you is which specific models were tested at which settings, how large the evaluation was, or how the same prompts fare on other implicit-bias domains. The direction worth watching is whether argumentation-theory prompting, refined further, can actually pull explicit inductive steps out of a model, because if it can, the same scaffolding is useful well beyond misogyny.