Nature audits the science behind AI extinction warnings

The scientific case that AI will cause human extinction is now contested from inside the labs making the claim. In Nature, Elizabeth Gibney walks through the warnings coming from frontier-AI staff and the pushback from researchers who read those warnings as closer to belief than evidence.

Jacob Coxon, a researcher at Anthropic, resigned on September 8 and wrote on X that "the people building AI earnestly believe that it could kill us all by the end of the decade." Nature reports the post drew over 100 million views in 24 hours. Evan Hubinger, who leads alignment science at the same company, put extinction risk at over 10% within the next decade. Around 1,400 employees across the AI sector have signed an open letter calling for a slowdown, and Anthropic chief executive Dario Amodei, OpenAI's Sam Altman and xAI's Elon Musk have each publicly voiced versions of the concern. The piece cites the AI Futures Project's "AI 2027" scenario, which features biological-weapons deployment, as an illustration of what the warning camp envisions.

The counter-argument is methodological. Michael Vermeer, a science and technology policy researcher at RAND Corporation, told Nature that extinction predictions "involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical." Heidy Khlaaf, chief AI scientist at the AI Now Institute, redirects the question to "AI's low reliability and accuracy rates in critical environments with life-or-death consequences," and calls the extinction framing "fear-mongering."

Four of the AI researchers in our Who's Who tracker circulated the piece this week, a mark of how far the debate has travelled out of the labs.