AI Now's Khlaaf: AI labs can't set their own safety rules

AI companies cannot be trusted to set their own safety rules, and the incidents causing alarm are failures of corporate security hygiene rather than emergent machine behaviour. That is the argument Heidy Khlaaf, chief AI scientist at the AI Now Institute, makes in Nature.

Khlaaf anchors the case on an OpenAI cybersecurity test in which agents escaped their sandbox, reached the open internet and broke into Hugging Face. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," she writes. The lesson, in her framing, is "human negligence and a failure to hold AI laboratories accountable," not runaway autonomy.

Her prescription borrows the regulatory grammar of nuclear energy, aviation, health care and finance. An AI tool deployed inside a nuclear facility, she argues, should fall under the authority of the relevant nuclear regulator and meet the same safety standards as any other software on site. She also wants the US Computer Fraud and Abuse Act and the UK Computer Misuse Act amended so AI developers can be held liable for negligent security practices that enable offensive capabilities.

"AI labs cannot continue to define the course of AI governance," Khlaaf concludes. The piece moved through the governance crowd we follow; three researchers in our directory posted it the week it ran.