Anthropic exit refuels AI extinction debate, Nature reports

Jacob Coxon resigned from Anthropic on September 8 and told the Wall Street Journal he feared the company's AI systems 'could spiral out of control and destroy humanity.' His farewell post drew over 100 million views in 24 hours. Evan Hubinger, who leads Anthropic's alignment science efforts, then amplified it, putting the risk of human extinction at '>10% within the next decade.'

In Nature, Elizabeth Gibney asks what evidence sits behind numbers like that, and hears from researchers who think the answer is thin.

'These predictions involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific,' Michael Vermeer of RAND tells her. A 2025 RAND study of extinction pathways found complete nuclear extinction infeasible but said biotechnology and atmospheric modification 'could not be ruled out.'

Heidy Khlaaf of the AI Now Institute is blunter. She calls the extinction framing 'fear-mongering' and points to what she studies day to day: 'AI's low reliability and accuracy rates in critical environments with life-or-death consequences.' Nuclear plants, she notes, are 'air-gapped' from the publicly accessible internet. 'These systems are built to a completely different, rigorous and regulated engineering standard that often requires physical hardening.'

The politics moved quickly around Coxon. Dario Amodei posted an essay calling for a slowdown, not a halt; Sam Altman of OpenAI and Elon Musk of xAI backed the frame; nearly 1,400 AI sector employees signed an open letter demanding one; and Senator Bernie Sanders proposed banning artificial superintelligence. David Sacks, co-chair of the US President's Council of Advisors on Science and Technology, warned that companies face 'massive product-liability exposure' from damaging cyberattacks enabled by their models.