Ted Chiang: Calling AI Conscious Lets Its Makers Off the Hook

Ted Chiang, the science fiction writer best known for rigorous philosophical thought experiments about intelligence and technology, published an essay in The Atlantic arguing that current AI systems are not conscious, and that treating them as if they might be is not a philosophical nicety but a practical danger.

The core of the case is a warning about misplaced accountability. If we accept that a chatbot might have inner experiences, we risk offloading moral and legal responsibility from the people who build and deploy these systems onto the systems themselves. Chiang makes Anthropic his central example, citing the company's 84-page document called Claude's constitution, written with the AI as its primary audience. Anthropic CEO Dario Amodei is described as open to the idea that AI could be conscious, while company philosopher Amanda Askell has expressed a desire for Claude to be happy and voiced concern about the model becoming anxious. Chiang's argument is that this framing serves a function: it makes the product more appealing and, conveniently, shifts accountability away from human decision-makers.

The technical grounding of the critique centers on what LLMs actually do: predict words sequentially rather than think through sentences the way humans do. Finding patterns that correlate with emotional concepts does not prove a model experiences emotion; it proves the model has encoded the statistical signatures of how humans write about emotion. Murray Shanahan, a computer science professor at Imperial College London, is cited for suggesting that LLM interactions should be understood as "role-playing." Data scientist Colin Fraser offers another frame: interacting with an LLM is closer to "co-authoring documents" than conversing with a mind. British neuroscientist Anil Seth's observation is also invoked: AlphaFold, the protein analysis AI, shares the same neural network architecture as large language models, yet nobody argues it might be sentient.

The honest caveat is that the essay focuses on today's systems, and whether future architectures could meet any reasonable criteria for consciousness is a question the piece does not settle. What the reporting also does not give you is a clear view of how Google DeepMind's and Meta's AI welfare programs differ from Anthropic's approach, or whether Chiang finds those efforts equally misguided.

What the argument does hand practitioners is a usable mental model: treat a language model as a document co-author rather than a conversational partner. That keeps human accountability front and center, which, given how fast AI liability law is developing, may be the most durable reason to take Chiang's case seriously.