Nature breaks down AI extinction fears after Anthropic exit

Jacob Coxon resigned from Anthropic on September 8 and told the Wall Street Journal the AI systems his colleagues were building 'could spiral out of control and destroy humanity.' On X he added, 'The people building AI earnestly believe that it could kill us all by the end of the decade.'

That is the trigger Nature set out to unpack in a 22 September feature by Elizabeth Gibney. Coxon's exit arrived in the same news window as Anthropic's alignment science lead Evan Hubinger pegging the risk of human extinction at '>10% within the next decade,' and CEO Dario Amodei posting an essay that called for a slowdown, though not a halt, in AI development. OpenAI's Sam Altman and xAI's Elon Musk subsequently backed the slowdown.

Nature then steps outside the labs. A 2025 RAND study by Michael Vermeer, Lathrop and Moon, 'On the Extinction Risk from Artificial Intelligence,' concluded that complete nuclear extinction is infeasible, while biotechnology and atmospheric-modification pathways 'could not be ruled out.' Any such scenario would require 'considerable ability to physically interact with the world' and would 'almost certainly take time and be detectable by humans.'

Vermeer himself is blunter on the discourse: extinction prediction, he tells Nature, 'involves so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical.'

Heidy Khlaaf of the AI Now Institute goes further. She tells Nature that 'AI's low reliability and accuracy rates in critical environments' worry her more than existential risk, a framing she calls 'fear-mongering.'