Seth, Block, Tononi argue AI isn't conscious, may never be

Three of the most cited names in consciousness science have co-authored a comment in Nature arguing that today's AI systems are not conscious, and that current trajectories are unlikely to make them so. The authors are neuroscientist Anil Seth, philosopher Ned Block, and neuroscientist Giulio Tononi. They define consciousness plainly, as "what goes away with dreamless sleep or under general anaesthesia," then give three reasons why large language models do not have it.

The first is that intelligence and consciousness are not the same thing. The human cerebellum contains nearly four-fifths of the neurons in the brain and performs sophisticated computation entirely outside conscious awareness. Chatbots feel inhabited, the authors argue, because they were trained on human writing about thoughts and feelings. They are "mirrors that reflect an image of our collective digitized past."

The second reason is substrate. The authors reject computational functionalism, the view that running the right algorithm on any hardware is enough. "Computations are clearly sufficient to produce forms of intelligence," they grant, but experience is where they draw the line. Brains, they write, are not just "computers that are made of meat."

The third reason leans on the live scientific theories. Integrated information theory requires components wired together in tightly interconnected ways that are absent from the layered, largely feed-forward architecture of today's deep learning. Biological naturalism ties experience to features of living systems, electrochemical synapses and metabolism among them, that silicon does not share.

The conclusion is blunt. Today's silicon-based AI systems, the authors write, should be treated as "smart tools, not smart people."