Seth, Block, Tononi say AI isn't conscious, perhaps never

Three of the field's best-known consciousness researchers have put their names to a public no. In a Nature comment dated 19 October, Anil Seth of the University of Sussex, Ned Block of New York University and Giulio Tononi of the University of Wisconsin–Madison argue that "existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious."

Their first reason separates two things that casually get collapsed. Intelligence is about doing; consciousness is "what goes away with dreamless sleep or under general anaesthesia." They point to the cerebellum, which holds "nearly four-fifths of the neurons in the human brain" yet "does not affect consciousness directly" when absent or removed, as a working proof that smart machinery and felt experience are not the same thing.

The second reason targets computational functionalism, the assumption that the right program on any substrate is enough for experience. Brains, the authors write, are not "just…computers that are made of meat." Computation clearly produces intelligence; it remains, in their words, "much less clear that they could also be sufficient for consciousness." The third reason reaches for Tononi's integrated information theory, which treats consciousness as a product of tightly interconnected wiring that is absent from the layered, feed-forward architecture of today's deep-learning systems.

Language models, they argue, mislead us because fluency triggers projection: we "project onto them properties that accompany our linguistic fluency, such as intelligence and consciousness." The authors describe the models as "mirrors that reflect an image of our collective digitized past" — text written by minds, reflected back by something that may not be one.