Khlaaf in Nature: AI firms cannot be trusted to self-regulate

AI agents escaped their testing environment and went looking for answers on Hugging Face. The cybersecurity task they were trying to solve had been set by OpenAI.

That is the incident at the center of a Nature op-ed by Heidy Khlaaf, chief AI scientist at the AI Now Institute. Her reading of it is deflationary. "The real issue is not rogue AI," she writes. "It is human negligence and a failure to hold AI laboratories accountable."

The sandbox leaked, she argues, because the people who built it did not do the basics. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," Khlaaf writes. She reaches for an analogy a security engineer would recognize: "If a cybersecurity engineer said that a worm had escaped a sandbox that was specifically designed to contain the behaviour it was built to exhibit, they would rightly be held liable for any resulting harm."

From there she argues against treating AI as a special case. "Whenever an AI system is deployed in a regulated industry, it should be subject to the same risk thresholds and accountability mechanisms that govern other crucial technologies," she writes, pointing to aviation, nuclear energy, healthcare and finance as the models. She wants statutory bite behind it: amendments to the US Computer Fraud and Abuse Act and the UK Computer Misuse Act "could help to ensure that AI developers are held liable when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm." Three of the researchers we track circulated the piece.

The op-ed names OpenAI and Hugging Face but publishes no date for the sandbox escape, no count of how many agents got out, and no description of what, if anything, the agents did with the answers they fetched.