Nature probes the thin science of AI extinction warnings

Jacob Coxon resigned from Anthropic on September 8, telling the Wall Street Journal that the systems the company is developing could 'spiral out of control and destroy humanity.' His post drew more than 90 million views. Evan Hubinger, who leads Anthropic's alignment science, amplified the message with his own estimate that the risk of human extinction is '>10% within the next decade.'

In Nature, Elizabeth Gibney puts those warnings to independent AI-safety researchers, and the empirical floor is thinner than the headlines suggest.

Michael Vermeer, a RAND researcher who co-authored a 2025 study titled 'On the Extinction Risk from Artificial Intelligence,' is blunt. 'These predictions involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific,' he tells Gibney. His team examined practical pathways through nuclear weapons, biotechnology and atmospheric modification. Complete extinction by nuclear weapons is not feasible, the study found; the biotech and atmospheric scenarios could not be ruled out, but both would need 'considerable ability to physically interact with the world and would take time,' leaving room for human intervention.

Heidy Khlaaf, chief AI scientist at the AI Now Institute, pushes harder. She calls the existential framing 'fear-mongering' and tells Nature that today's systems are 'largely probabilistic systems' lacking human-like understanding. 'Scientific claims require falsifiability precisely to avoid the nature of religious arguments,' she adds. Her focus is on measurable harms: disinformation, AI-generated military intelligence, bioweapon-enabling misuse, and the unreliability of these models in life-or-death systems.

Four of the researchers we track surfaced the piece on their feeds this week.