Seth, Block and Tononi: AI isn't conscious, maybe never

Anil Seth, Ned Block and Giulio Tononi, usually associated with competing theories of consciousness, have co-signed a Nature Comment arguing that today's AI isn't conscious and that the silicon path probably never will deliver it. The piece ran on 19 October.

They start by prising intelligence apart from consciousness, which they define as 'what goes away with dreamless sleep or under general anaesthesia, and what returns after waking up.' The example they lean on is the human cerebellum, which holds nearly four-fifths of the brain's neurons and performs sophisticated computations without producing any conscious experience. Large language models, by contrast, are 'mirrors that reflect an image of our collective digitized past,' which, the authors argue, is why people anthropomorphize them in a way nobody does with AlphaFold.

The second argument goes at computational functionalism directly. 'Brains are not… computers that are made of meat,' they write, pushing back on the assumption that any program running on silicon could in principle be conscious.

The third reason comes from integrated information theory, the framework Tononi has long defended. Consciousness, on that account, depends on a system's components being wired together in 'very specific, tightly interconnected ways' that today's deep-learning architectures do not satisfy.

The practical worry here is not an under-protected AI. It is the opposite. 'False positives are much more likely to occur than false negatives,' they write, and chasing 'AI welfare' and 'AI rights' risks pulling 'attention and resources away from the welfare of beings whose capacity to suffer is not in question.' Today's systems, they conclude, are 'smart tools, not smart people.'

Seven of the AI researchers on our radar posted the Nature Comment in the days after it ran.