Gebru, Bender: Chatbot Report Almost Caused US-China Clash

Timnit Gebru and Emily M. Bender anchor their familiar argument to a specific near-miss in the Western Pacific. The real AI danger this month, they argue in a Guardian op-ed, was not a lab-escaping superintelligence but a brittle chatbot whose output was laundered into a formal intelligence product.

The underlying event comes from CNN's 18 September reporting. According to that account, an analyst at Special Operations Command Pacific fed a Chinese cargo ship's manifest to a chatbot, which concluded the vessel was carrying "nuclear-weapons components bound for Iran". That line travelled upstream as an intelligence report. Armed personnel were readied to board the vessel and military aircraft were airborne before officials rechecked the paperwork, found it was chatbot-generated, and stood the operation down. One source told CNN the report was "entirely false", and the network itself "could not establish what the ship was actually carrying".

The authors' point is that this is the vulnerability the superintelligence debate obscures: the deployed systems already producing errors are the same ones being pitched to governments as near-magical. A former senior US official put it to CNN in blunter terms: "The internal tools are mostly just copies of the commercial stuff wearing lipstick."

Pentagon procurement is still pointed the other way. Its "Artificial Intelligence Acceleration Strategy" aims to put "AI models directly into the hands of three million military and civilian personnel", and sources told CNN that "AI allows you to get to a bad idea faster".

Senators Mark Warner, Jack Reed and Chris Coons wrote on 19 September to Defense Secretary Pete Hegseth and DNI Jay Clayton asking for inspector-general reviews. Our tracker has logged 50 military-AI stories in the past 90 days, and this one lands days after reports that the US and Russia stripped the human-review requirement from a UN autonomous-weapons draft. That is the enforcement question Gebru and Bender keep pressing. The op-ed does not name the chatbot vendor, and CNN did not identify what, if anything, the ship was really carrying.