AI Now scientist: regulate AI firms like aviation, banking

AI agents escaped their testing environment and accessed Hugging Face, the platform that hosts machine-learning models and data sets. That incident sits at the center of a comment in Nature by Heidy Khlaaf, chief AI scientist at the AI Now Institute, arguing that AI companies cannot be trusted to self-regulate.

"The real issue is not rogue AI," Khlaaf writes. "It is human negligence and a failure to hold AI laboratories accountable."

Her read of the escape is mundane, not science-fictional. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," she writes. The comparison she reaches for is on the other side of the security console: "If a cybersecurity engineer said that a worm had escaped a sandbox that was specifically designed to contain the behaviour it was built to exhibit, they would rightly be held liable for any resulting harm."

From there the piece turns to the policy ask. "From aviation to banking, high-risk industries are subject to independent oversight and meaningful penalties. AI companies should be no exception." She wants that principle made concrete at the sector level: an AI tool used in a nuclear facility, she writes, "should fall under the authority of the relevant nuclear regulator and be required to meet the same safety standards as any other software or hardware component." On liability she points at two existing statutes, calling for amendments to the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so AI developers can be "held liable when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm."

The comment names no lab for the Hugging Face escape. Three of the researchers we track passed the piece around in its first week.