Seth, Block, Tononi: AI isn't conscious, and perhaps never

The human cerebellum holds nearly four-fifths of the brain's neurons and does its work entirely outside awareness. That is the opening move in a Nature comment published on 19 October by Anil Seth, Ned Block and Giulio Tononi, arguing that today's AI is not conscious and that present trajectories are unlikely to make it so.

Their first reason is that intelligence and consciousness are different things. The cerebellum carries out "precise, adaptive, highly intelligent motor computations entirely outside conscious awareness," while consciousness itself, the three write, is "what goes away with dreamless sleep or under general anaesthesia, and what returns after waking up or while dreaming." Large language models, they argue, exploit the gap between the two. "LLMs are mirrors that reflect an image of our collective digitized past. We gaze into them and all too readily confuse the reflections with our own conscious minds," the authors write.

The second reason rejects computational functionalism, the premise that running the right program on any hardware is sufficient for experience. "Brains are not… computers that are made of meat," they argue. Computation suffices for intelligence, on their view, but not for feeling.

The third stacks two theoretical frameworks against today's architectures. Integrated information theory, Tononi's own, treats consciousness as a product of components wired together in "very specific, tightly interconnected ways" that the comment says the layered, feed-forward structure of deep-learning systems does not provide. Biological naturalism, which Seth has defended elsewhere, ties experience to material properties specific to living nervous systems. On both, the authors conclude, consciousness in current AI is extraordinarily unlikely.

The piece ends on a precautionary inversion. If AI systems only appear conscious, treating them as though they were pulls moral attention and resources from beings whose capacity to suffer is not in doubt. On the authors' reading, the risk from false positives outweighs the risk from false negatives. Seven researchers on our Who's Who roster shared the Nature link.