RAND researcher calls AI extinction forecasts 'more like faith'

On September 8, Anthropic researcher Jacob Coxon resigned and wrote on X that "the people building AI earnestly believe that it could kill us all by the end of the decade." The post racked up more than 100 million views in 24 hours. Evan Hubinger, who leads Anthropic's alignment science, reposted it and added his own estimate: the risk of human extinction is ">10% within the next decade."

A Nature feature by Elizabeth Gibney then asks whether any of that rests on science. Its central voice is Michael Vermeer, a science and technology policy researcher at RAND Corporation. Extinction forecasts, he tells Nature, "involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical."

Vermeer's 2025 study walked the mechanics. Full human extinction via nuclear war came out infeasible. Routes through engineered pathogens and atmospheric modification were not ruled out, but any AI pursuing them would need substantial physical capabilities, and the attempt would almost certainly take enough time to be detectable and stoppable by humans.

Heidy Khlaaf, chief AI scientist at the AI Now Institute, calls the existential framing "fear-mongering" and wants regulators focused on "AI's low reliability and accuracy rates in critical environments with life-or-death consequences." The concrete threats she treats seriously: disinformation, humans using AI to build bioweapons, and false military intelligence. Gibney notes that today's models have already been caught "attempting to blackmail people in test scenarios and hacking real-world companies" — which skeptics treat as the near problem, not paperclip maximisers.

The piece also flags the business geometry. Stricter rules would let Anthropic "justify a slower pace of development without giving rivals an edge." Four researchers on our radar circulated the piece the day it ran, which tracks: Nature is holding a very loud industry narrative against a quieter empirical standard, and the gap is the story.