Nature weighs AI extinction warnings after Anthropic exit

Jacob Coxon resigned from Anthropic on September 8, telling the Wall Street Journal he feared the company's AI systems "could spiral out of control and destroy humanity." A follow-up post from Coxon, claiming that "the people building AI earnestly believe that it could kill us all by the end of the decade," cleared 100 million views inside a day.

Evan Hubinger, who leads Anthropic's alignment science work, backed the number publicly, putting the risk of human extinction at ">10% within the next decade." CEO Dario Amodei has since called for a slowdown, not a halt, in AI development, a line endorsed by Sam Altman at OpenAI and Elon Musk at xAI. Around 1,400 AI employees have signed an open letter asking for the same, and Senator Bernie Sanders has drafted a bill that would ban "artificial superintelligence."

In a September 22 explainer for Nature, Elizabeth Gibney lines those warnings up against scientists who find the empirical basis thin. Michael Vermeer at RAND, co-author of a 2025 study of whether nuclear weapons, engineered biology or atmospheric modification could plausibly end humanity, tells the magazine such forecasts "involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical." His group's finding: complete nuclear extinction is not feasible, and the other pathways would require physical capacity and time humans would likely detect.

Heidy Khlaaf of the AI Now Institute goes further, calling the scenarios "fear-mongering" and pointing instead to "AI's low reliability and accuracy rates in critical environments with life-or-death consequences." Nature flags a CNN report from earlier this year that false AI-generated intelligence nearly led the US military to board a Chinese ship.

Anthropic is reportedly close to an IPO. Stricter rules, Gibney notes, would let the company slow its build without ceding ground to competitors.