Nature scrutinizes science behind AI-extinction warnings

Independent AI-safety researchers tell Nature there is thin scientific grounding for the extinction warnings now coming from inside the frontier labs. The latest wave started when Anthropic alignment researcher Jacob Coxon resigned saying the people building AI 'earnestly believe that it could kill us all by the end of the decade,' and Evan Hubinger, who leads the company's alignment science team, put the risk of human extinction at '>10% within the next decade.'

Michael Vermeer at RAND, co-author of a 2025 study, told Elizabeth Gibney that such forecasts 'involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical.' That study examined practical pathways through nuclear weapons, biotechnology and deliberate atmospheric modification. It found complete extinction by nuclear weapons is not feasible, and could not rule out biotech or atmospheric scenarios, but concluded both would need considerable ability to physically interact with the world and would take time, leaving room for human intervention.

Heidy Khlaaf, chief AI scientist at the AI Now Institute and a former OpenAI safety engineer, is blunter. She calls the extinction framing 'fear-mongering' and tells Nature that 'Scientific claims require falsifiability precisely to avoid the nature of religious arguments.' The harms she wants regulated instead are present-day: disinformation, bioweapon creation and false military intelligence, including a recent incident in which AI-generated intelligence nearly triggered an unauthorized boarding of a Chinese vessel.

Four of the researchers we follow passed Gibney's piece around on publication day. Khlaaf's policy demand lands harder than any body-count scenario: 'AI labs cannot continue to define the course of AI governance.'