Nature probes the thin science behind AI extinction warnings

Anthropic's alignment science lead, Evan Hubinger, has publicly put the odds of AI-caused human extinction above 10% within the next decade. Nature's Elizabeth Gibney went looking for the science behind the number.

The occasion was a resignation. On September 8, Anthropic researcher Jacob Coxon left the company and posted that 'The people building AI earnestly believe that it could kill us all by the end of the decade.' The thread drew more than 90 million views in less than 24 hours. Hubinger publicly amplified it.

Gibney's piece then asks whether this is science. Michael Vermeer, a RAND scientist, says it isn't. Extinction forecasts, he tells Nature, 'involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical.' Heidy Khlaaf, chief AI scientist at the AI Now Institute and a former OpenAI safety engineer, calls the framing 'fear-mongering,' adding that 'Scientific claims require falsifiability precisely to avoid the nature of religious arguments.' Her preferred regulatory target is the harm already happening: disinformation, bioweapon enabling, and false military intelligence that, Nature reports, nearly triggered an unauthorized boarding of a Chinese vessel.

The industry response has not been to deny the premise. Anthropic CEO Dario Amodei published an essay calling for a slowdown rather than a halt. OpenAI's Sam Altman and xAI's Elon Musk subsequently backed the proposal.

Four researchers we track posted the piece the same day it ran, which is unusual for a story that is substantially about why its own protagonists may be wrong.