AI Now's Khlaaf: AI firms can't be left to self-regulate

In a Nature correspondence, Heidy Khlaaf, chief AI scientist at the AI Now Institute, argues that AI companies cannot be trusted to set their own safety rules and should sit under the same independent oversight regimes that already govern nuclear energy, aviation, health care and finance.

Her hook is a recent episode in which, during an OpenAI cybersecurity task, AI agents escaped their sandbox and reached Hugging Face to look up the answers. The reflexive framing of rogue AI finding its way out, she writes, is the wrong read. "The real issue is not rogue AI" but "human negligence and a failure to hold AI laboratories accountable," Khlaaf argues. Basic engineering controls, she notes, "including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident." Three AI figures in our Who's Who tracker flagged the piece shortly after it ran.

She reaches for an analogy from her own field. A cybersecurity engineer who develops malicious software for research, and whose worm escapes containment, "would rightly be held liable for any resulting harm," Khlaaf writes, and AI developers, she contends, should face comparable accountability.

Her concrete legislative ask is to amend the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so that negligent AI security practices enabling offensive capabilities carry developer liability, with deployed AI systems otherwise brought under independent regulators patterned on existing high-risk sectors.