Seth, Block, Tononi: AI consciousness 'extraordinarily unlikely'

Three of the most cited scientists working on consciousness have used a Nature comment to tell the field that today's AI systems almost certainly do not have inner experience, and that the technology's current trajectory is unlikely to produce it. Writing in Nature on 19 October, Anil Seth, Ned Block and Giulio Tononi say that "consciousness in AI is extraordinarily unlikely, at least in the digital, silicon-based systems that underpin today's technologies."

Their case rests on three moves. The first is a hard split between what a system does and what it feels: "Consciousness is about being and feeling; intelligence — very broadly — is all about doing, or performing functions." As evidence, they point at the cerebellum, which holds nearly four-fifths of the neurons in the human brain and performs precise, adaptive motor computations entirely outside conscious awareness. Lots of intelligence happens without anyone being home.

The second move takes aim at computational functionalism, the assumption that running the right program on any hardware is enough for experience. The authors write that "brains are not — or at least not just — computers that are made of meat," and that "LLMs are mirrors that reflect an image of our collective digitized past." The sense that something is on the other end of the conversation, in their reading, is largely a projection the medium invites.

The third move narrows from philosophy to the two scientific frameworks each author has helped build: integrated information theory and biological naturalism. Both, Seth, Block and Tononi argue, treat specific wiring and specific material properties as constitutive of experience, and both conclude that "today's deep-learning architectures lack the specific wiring, and the specific material properties, that each framework treats as constitutive of experience." Layered, feed-forward networks are not the kind of thing either theory predicts can feel anything.

The closing stake is practical rather than metaphysical. Today's systems, they argue, are "smart tools, not smart people," and treating appearances of consciousness as the real thing risks pulling welfare attention from beings whose capacity to suffer is already certain. The piece landed with weight: seven of the AI researchers we track in our Who's Who had shared the link within a day of publication.