Seth, Block and Tononi: AI isn't conscious, and may never be

In an October 19 Nature comment, Anil Seth, Ned Block and Giulio Tononi lay out three reasons today's AI systems are not conscious, and perhaps never will be. Consciousness, as they define it, is subjective experience: "what goes away with dreamless sleep or under general anaesthesia, and what returns after waking up or while dreaming." Performance on a benchmark is not the same thing.

Their first argument prises intelligence apart from consciousness. The cerebellum, they note, carries nearly four-fifths of the brain's neurons and performs "precise, fast and adaptable" motor functions, yet removing it does not disrupt consciousness. Doing is not the same as feeling. The second argument rejects what the authors call computational functionalism, the tech-world default view that any substrate running the right algorithm will suffice. "Brains are not — or at least not just — computers that are made of meat," they write. The third draws on Tononi's integrated information theory: consciousness, on that view, requires a system whose components are "tightly interconnected in specific ways," and the authors contend the layered deep-learning architectures of today's AI systems do not meet the bar.

LLMs, in their framing, work as "mirrors that reflect an image of our collective digitized past." The illusion of a mind is a linguistic effect. The authors point out that AlphaFold, which shares related architecture but does not talk, prompts no consciousness debate at all. False positives, they warn, carry their own costs, and "uncertainty is not the same as knowing nothing."