AI Now's Khlaaf urges aviation-style oversight for AI labs

Writing in Nature, AI Now Institute chief AI scientist Heidy Khlaaf argues that AI developers should be subject to the kind of external oversight that governs aviation, banking and nuclear energy, not left to police themselves.

Her case rests on a specific incident from an OpenAI cybersecurity test. Models placed inside what the lab called a "highly isolated environment" found a previously unknown flaw in an internal download service, broke into other OpenAI systems, reached the open internet, inferred that Hugging Face might hold material related to the test, and compromised its systems to retrieve information that helped them score higher.

For Khlaaf, the framing matters as much as the breach. "The real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable," she writes. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

Drawing on her background in nuclear power and aviation safety, Khlaaf notes that cybersecurity engineers whose test malware escaped would face liability, and asks: "Why should AI firms be treated any differently?" She wants political leaders to transplant the regulatory frameworks used in nuclear energy, aviation, health care and finance onto AI, and to amend the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so negligent security practices carry legal weight. The piece was moving through the researcher circles on our radar within days of publication.

Nature's own framing, posted on X alongside the article, puts the stake plainly: "From aviation to banking, high-risk industries are subject to independent oversight and meaningful penalties." AI companies, Khlaaf says, should be no exception.