Seth, Block, Tononi: AI is 'smart tools, not smart people'

In a 19 October comment for Nature, neuroscientist Anil Seth, philosopher Ned Block and neuroscientist Giulio Tononi argue that today's large language models are not conscious, and that current development paths are unlikely to make them so. Their verdict: AI systems are "smart tools, not smart people."

Their first move is to prise intelligence apart from consciousness. The human cerebellum, the authors note, performs "precise, adaptive, highly intelligent motor computations entirely outside conscious awareness," evidence that sophisticated information processing does not by itself imply any inner experience. Fluent language from an LLM, on this view, produces an illusion of a mind through anthropomorphism rather than through a mind.

Second, they reject computational functionalism, the assumption that the right algorithm on any hardware is sufficient for subjective experience. On Tononi's integrated information account, consciousness requires "components being wired together in very specific, tightly interconnected ways" that the layered, feed-forward structure of today's deep-learning systems does not have.

Third, under biological naturalism, "what a brain does and what a brain is made of are inseparable." Silicon running inference is not the same kind of thing as a living nervous system, and the authors argue that gap matters. LLMs, they write, "are mirrors that reflect an image of our collective digitized past," and the sense of someone on the other end is largely a projection by the user.

The piece, which two researchers we track in consciousness and alignment circles posted the same day, closes on the practical stakes. Mistakenly attributing experience to systems that have none, the authors warn, carries greater costs than the reverse: resources diverted to protect non-conscious entities, and policy written around a projection.