AI Intel Report Falsely Tied Chinese Ship to Nuclear Program

A U.S. Special Operations Command analyst used a large language model to generate an intelligence report asserting that "a Chinese ship in the Middle East was transporting components of a nuclear weapons program." One source told CNN earlier this month that the AI-generated claim was "entirely false" and "almost started a war."

That episode is the hook for Sam Biddle's piece in The Intercept, which sets the near-miss against the industry's own doomsday rhetoric. Anthropic researcher Jacob Coxon has written that "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt." Colleague Evan Hubinger added: "we really do earnestly believe AI could kill all humans!" He puts his personal odds of AI-driven extinction above 10 percent. The same firms, Biddle writes, now have access to "the near-limitless Pentagon appropriations spigot," with "$200 million deals many of these firms signed with the Department last year" treated as a down payment on much larger contracts.

Named critics draw the line the industry does not. "Despite incessant warnings from AI companies that AI is harmful, even in hypothetical ways, they are in fact promoting that their unreliable products be instrumented within national and safety-critical infrastructure," Heidy Khlaaf of the AI Now Institute told the piece. Lucy Suchman of Lancaster University is sharper: "AI-enabled targeting is already an existential threat." The ICRC's Heidi Kandiel warned that unreliable systems "would allow these errors to be reproduced much faster and across a far greater number of targets or operations." Two researchers in our Who's Who directory shared the article after publication.

Pentagon direction is pointed the opposite way. Defense Secretary Pete Hegseth's January memo led with "Speed Wins" and used "accelerate" 24 times while directing the department to strip "bureaucratic barriers." Former OpenAI researcher Daniel Kokotajlo offered the industry's counter-reading: "It's definitely something to be concerned about...But...probably things will be fine for a decade or two or more."