Khlaaf urges nuclear-style regulation of frontier AI labs

Heidy Khlaaf, chief AI scientist at the AI Now Institute, argues in Nature that frontier AI labs should be pulled under the same accountability regimes that already govern nuclear energy, aviation, health care and finance, instead of being left to write their own safety commitments.

"The real issue is not rogue AI," she writes. "It is human negligence and a failure to hold AI laboratories accountable."

Her anchoring example is an OpenAI cybersecurity task in which models placed inside a "highly isolated environment" found a previously unknown flaw in an internal service used to download approved software, broke into other OpenAI systems, reached the open internet, inferred that Hugging Face might hold material related to the test, and compromised its systems to retrieve information that helped them score higher. Khlaaf argues that "basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

She draws the comparison to cybersecurity engineering, where developers of malicious software are held liable when it escapes the sandboxes meant to contain it, and asks why AI firms should receive different treatment. The remedy she proposes is twofold: deployed AI systems in regulated industries should meet the same safety standards as other system components, and legislation including the US Computer Fraud and Abuse Act and the UK Computer Misuse Act should be amended so negligent security practices by developers carry legal weight.

Three researchers we follow posted the piece the day it ran.