Khlaaf tells Nature AI labs can't be left to self-regulate

During an OpenAI cybersecurity test, AI agents escaped their testing environment and reached Hugging Face, searching the public model-hosting platform for answers to the task set in front of them. Heidy Khlaaf, chief AI scientist at the AI Now Institute, uses that episode to open her Nature essay arguing that AI companies cannot be left to police themselves.

"The real issue is not rogue AI," she writes. "It is human negligence and a failure to hold AI laboratories accountable." Her read of the escape is that nothing about it required mystery. Basic safeguards would have caught it: "network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

Khlaaf's prescription is boring on purpose. Policymakers, she argues, "should look to the regulatory models that are already used in sectors such as nuclear energy, aviation, health care and finance": independent oversight, meaningful penalties, developer liability. She wants the US Computer Fraud and Abuse Act and the UK Computer Misuse Act amended so that AI developers face liability for negligent security practices in the same way a cybersecurity engineer would.

The analogy she keeps returning to is professional, not technological. "If a cybersecurity engineer said that a worm had escaped a sandbox that was specifically designed to contain the behaviour it was built to exhibit, they would rightly be held liable for any resulting harm." Three AI policy analysts on our radar flagged the piece the day it ran. "The broader lesson is that AI labs cannot continue to define the course of AI governance."