Anthropic exit reignites AI extinction debate in Nature

A viral resignation from Anthropic researcher Jacob Coxon has thrust the 'AI could kill us all' debate back into the mainstream, and Nature went looking for the science behind the fear. What it found is that the case for extinction rests on chains of assumption that empirical work has only begun to test.

Coxon, who resigned September 8, wrote that 'the people building AI earnestly believe that it could kill us all by the end of the decade.' Inside the same company, alignment-science lead Evan Hubinger puts extinction risk at '>10% within the next decade.' Anthropic chief executive Dario Amodei, OpenAI's Sam Altman and xAI's Elon Musk have all backed calls for a slowdown, though not a halt.

Others in the field think the framing is unfalsifiable.

Michael Vermeer, a science and technology policy researcher at RAND Corporation, told Nature the argument 'involves so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical.' Heidy Khlaaf, chief AI scientist at the AI Now Institute, dismissed the extinction focus as 'fear-mongering,' pointing instead to 'AI's low reliability and accuracy rates in critical environments with life-or-death consequences.'

Vermeer's own 2025 RAND paper stress-tested three extinction pathways. It concluded that 'complete extinction by nuclear weapons is not feasible, but the other two scenarios could not be ruled out,' and even the plausible ones would require models to gain 'considerable ability to physically interact with the world,' with human detection likely along the way. Four researchers on our tracker circulated the piece within a day.