Nature probes thin science behind AI extinction warnings

"The people building AI earnestly believe that it could kill us all by the end of the decade." That line, posted on X by former Anthropic researcher Jacob Coxon after he resigned on September 8, drew more than 100 million views in 24 hours. Anthropic's alignment science lead Evan Hubinger reposted it and put a number on the belief: a greater than 10% chance within the next decade.

The Nature feature by Elizabeth Gibney is about what sits behind those numbers. The short answer, from the independent safety researchers Gibney spoke to: not much.

Michael Vermeer, who studies AI risk at the RAND Corporation in Santa Monica, California, told Nature that extinction forecasts "involve so many untestable claims that you just end up with a conversation that is really more like faith than something scientific or empirical." A 2025 RAND study he worked on looked at extinction feasibility across three concrete threat vectors: nuclear weapons, biotechnology and atmospheric modification. Hubinger's figure is not derived from anything comparable.

Heidy Khlaaf, at the AI Now Institute in New York City, is sharper. She calls the extinction framing "fear-mongering" and wants regulators focused on what she describes as "AI's low reliability and accuracy rates in critical environments with life-or-death consequences." The piece lists the concrete threats it treats seriously: disinformation, humans using AI to build bioweapons, and false military intelligence that reportedly nearly triggered a US-China incident.

Gibney notes that 1,400 AI-sector employees have signed an open letter calling for a slowdown, while Dario Amodei at Anthropic, Sam Altman at OpenAI and Elon Musk at xAI have all publicly flagged existential risk even as their labs continue to ship. Four of the AI experts we track in Who's Who had shared the piece by the week it published.