Seth, Block and Tononi: AI isn't conscious, may never be

"The default position of the research community should therefore be that AI is not conscious." That is the thesis of a comment published 19 October in Nature by neuroscientist Anil Seth of the University of Sussex, philosopher Ned Block of New York University and neuroscientist Giulio Tononi of the University of Wisconsin–Madison.

The authors proceed in three moves. The first prises apart ideas people routinely conflate: "Consciousness is about being and feeling. Intelligence, very broadly, is about doing, or performing functions." Consciousness, they add, is simply "what goes away with dreamless sleep or under general anaesthesia." The human cerebellum, which holds "nearly four-fifths of the neurons in the human brain," performs sophisticated computation without any felt experience attached. Large language models, the authors argue, exploit a different trick: trained on text saturated with reports of thoughts and feelings, they are "mirrors that reflect an image of our collective digitized past."

The second move takes aim at computational functionalism — the assumption that running the right program on any substrate suffices for experience. Brains, the authors contend, cannot be cleanly separated into software and hardware the way digital machines can.

The third leans on two theories of consciousness. Integrated information theory requires components "to be tightly interconnected in specific ways," a pattern the authors say is present in parts of the cortex "but not, for instance, by the cerebellum," and absent from "layered, feed-forward" deep-learning systems. Biological naturalism ties experience to "electrochemical synapses," metabolism and autopoiesis — "the ability of living systems to continually regenerate themselves." Silicon has none of these.

Their closing policy line is unambiguous: "Silicon, digital AI systems should be considered objects, not subjects."