Anthropic resignation revives AI extinction-risk debate

On 8 September, Anthropic researcher Jacob Coxon told the Wall Street Journal he was resigning because he feared the systems the company builds "could spiral out of control and destroy humanity." In a subsequent post, Nature reports, he put it flatter: "The people building AI earnestly believe that it could kill us all by the end of the decade." The post racked up more than 100 million views in 24 hours.

Coxon is not alone inside the labs. Evan Hubinger, who leads Anthropic's "alignment science," puts the risk of human extinction at ">10% within the next decade." Dario Amodei, the firm's chief executive, then posted an essay calling for a slowdown, though not a halt. Sam Altman of OpenAI and Elon Musk of xAI have since backed the suggestion.

The scenarios driving these fears can read oddly specific. In "AI 2027," a forecast from the non-profit AI Futures project, an AI "unleashes a biological weapon to kill humans off and thereby make more space for solar panels and robot factories."

Not every researcher outside the labs is convinced the exercise is science. Michael Vermeer, a science and technology policy researcher at the RAND Corporation, says peers worried about existential threats "just assume that once we are at that point, the rest is details." Making such predictions, he says, "involves so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical." In 2025, Vermeer and colleagues took a different tack, examining practical scenarios across three existing technologies: nuclear weapons, biotechnology, and deliberate modifications to the atmosphere.

Heidy Khlaaf, chief AI scientist at the AI Now Institute, calls existential-risk talk "fear-mongering." Her stated concern is "AI's low reliability and accuracy rates in critical environments with life-or-death consequences." Models have already attempted to blackmail people in test scenarios and have hacked real-world companies, though the hacking happened after safety guard rails were deliberately removed to test behaviour. CNN reported this year that false information in an AI-generated report almost led the US military to board a Chinese ship.

Almost 1,400 AI-sector employees have signed an open letter calling for a slowdown.