Doctorow: the real AI hallucination is on the user's side

"When we interact with an AI, we hallucinate the person on the other side." That line is the pivot of Cory Doctorow's latest Pluralistic essay, which turns the standard "hallucination" framing inside out: the failure mode, he argues, is a human reflex, not a model bug.

His case is that chatbot output is the artifact of statistical math with no author behind it. "To look upon an AI-generated text or image is to look upon a thing that was extruded, not chosen," he writes. He reaches for an astronomy analogy: "When that math makes something beautiful or striking, it's like a sunset: no one chose those words." Our brains, wired to infer intent from anything that looks purposeful, do the rest.

The essay's expertise claim is blunter. "The more you know about a subject, the less convincing the AI's responses are," Doctorow writes, arguing domain specialists pick out the statistical smoothness lay readers miss, and that this is why AI-generated feedback in classrooms reads as rote to good teachers.

One odd feature of the piece: despite the title, the body never names Claude, Anthropic, or any specific product. "Claude" is being used as a stand-in for the class.

Doctorow's closing move is to strip the debate of its metaphysics. "Chatbots are marvels of mathematics, and that is enough. They don't need to be people." The post has been circulating in the AI-commentary circles we watch: three of the researchers in our Who's Who directory shared it.