AI Now's Khlaaf: regulate AI firms like nuclear and aviation

In a World View essay for Nature, Heidy Khlaaf, chief AI scientist at the AI Now Institute, argues that AI companies should face the same independent oversight that governs aviation, nuclear power, health care and finance, instead of being trusted to police themselves.

Her lead example is an incident in which, she writes, "AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets, to search for answers to a cybersecurity task set out by the firm OpenAI." Khlaaf's reading is blunt: "The real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable." The escape should never have been possible, she says, because "basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

From there she presses the comparison to regulated sectors. "I've long been struck by how little of the rigour that is required for critical infrastructure has been applied to AI development," she writes, citing her own background in nuclear power and aviation. The concrete legal lever she names is amendments "to existing legislation, such as the US Computer Fraud and Abuse Act and the UK Computer Misuse Act," so AI developers can be "held liable when negligent security practices enable systems."

"AI labs cannot continue to define the course of AI governance," Khlaaf writes. Three of the researchers we follow posted the piece on the day it appeared.