Seth, Block and Tononi: AI isn't conscious, probably won't be

Three of the field's most prominent consciousness researchers have published a Nature comment arguing that today's AI systems are not conscious, and that current approaches to building them are unlikely to make them so.

The piece appeared in Nature on 19 October, signed by Anil Seth of the University of Sussex, Ned Block of NYU's philosophy department, and Giulio Tononi, director of the Wisconsin Institute for Sleep and Consciousness. They offer three arguments. First, intelligence is not consciousness: the human cerebellum holds nearly four-fifths of the brain's neurons and performs 'precise, adaptive, highly intelligent motor computations entirely outside conscious awareness.' Second, computational functionalism is wrong, the assumption that running the right algorithm on any substrate produces experience. 'Brains are not… computers that are made of meat,' they write. Third, the leading scientific theories of consciousness all exclude today's models. Tononi's own integrated information theory requires components wired together in tightly interconnected ways that the 'layered, feed-forward' architecture of deep-learning systems lacks, while biological-naturalism accounts tie consciousness to properties such as electrochemical synapses and metabolism that silicon does not have.

Language, the authors argue, is the trap. LLMs are 'mirrors that reflect an image of our collective digitized past,' and the sense of a mind on the other end is largely projection. AlphaFold, they note, 'almost nobody worries is conscious, precisely because it doesn't talk back.'

Their conclusion is blunt: today's models are 'smart tools, not smart people,' and precautionary resource allocation to AI welfare risks diverting attention from beings whose capacity to suffer is already established.