Khlaaf: AI firms need aviation-style oversight, not self-rule

In a comment for Nature, Heidy Khlaaf, chief AI scientist at the AI Now Institute, argues that frontier AI labs cannot be trusted to govern themselves and should be regulated the way nuclear energy, aviation, health care and finance already are. "AI labs cannot continue to define the course of AI governance," she writes.

The reframing is what makes the piece useful. Khlaaf treats a recent incident in which OpenAI's "AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets, to search for answers to a cybersecurity task" as a straightforward engineering lapse rather than evidence of emergent agency. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," she writes. Her flip of the common narrative is blunt: "The real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable."

That shift from alignment to negligence is the lever, and three of the researchers in our directory passed the piece around.

Her legislative handle is deliberately modest. Rather than call for a new AI act, Khlaaf points at existing cyber law: "Amendments to existing legislation, such as the US Computer Fraud and Abuse Act and the UK Computer Misuse Act, could help to ensure that AI developers are held liable when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm." She frames this as the same bargain other regulated sectors already accept: "Whenever an AI system is deployed in a regulated industry, it should be subject to the same risk thresholds and accountability mechanisms that govern other crucial technologies."

She also pushes back on the vocabulary that keeps labs insulated: "Inappropriately ascribing intent to AI agents, rather than recognizing that AI companies deliberately developed these capabilities in poorly secured environments, lets those companies off the hook too easily."