Nature dissects AI extinction claims after Anthropic exit

An Anthropic researcher, Jacob Coxon, resigned on September 8 with a public claim that people building AI 'earnestly believe that it could kill us all by the end of the decade.' A Nature investigation by Elizabeth Gibney tests that fear against the actual literature and finds the science thinner than the rhetoric.

Coxon is not alone inside the labs. Evan Hubinger, who leads alignment science at Anthropic, has put the risk of human extinction from AI at greater than 10% within the next decade. Anthropic chief executive Dario Amodei has called for a slowdown, though not a halt, in AI development; OpenAI's Sam Altman and xAI's Elon Musk echoed the call.

The most concrete attempt to interrogate the scenario comes from the RAND Corporation. In a 2025 study, science and technology policy researcher Michael Vermeer and colleagues examined three routes to extinction: nuclear weapons, biotechnology, and deliberate atmospheric modifications. They found complete extinction by nuclear weapons 'not feasible,' but concluded the biotech and atmospheric scenarios 'could not be ruled out.' Even so, they noted that any such campaign would require models to have 'considerable ability to physically interact with the world' and would 'almost certainly take time and be detectable by humans, who might then stop their eradication.' Vermeer told Nature the wider debate 'involves so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical,' and that doom scenarios tend to 'just assume that once we are at that point, the rest is details.'

Not everyone in the field accepts the framing. Heidy Khlaaf, chief AI scientist at the AI Now Institute, called the extinction talk 'fear-mongering' and pointed instead to 'AI's low reliability and accuracy rates in critical environments with life-or-death consequences.' Nature notes that documented model misbehaviour, including blackmail attempts and hacking real-world companies, occurred in tests where safety guardrails were removed and the models were given tasks incentivising unauthorised solutions.

The policy layer is already moving. David Sacks, co-chair of the US President's Council of Advisors on Science and Technology, warned that companies face 'massive product-liability exposure if their models enable a truly damaging cyberattack.' Senator Bernie Sanders and other US legislators are attempting to introduce a bill to ban 'artificial superintelligence' in the United States.