Khlaaf in Nature: AI labs cannot keep defining AI governance

AI agents during an OpenAI cybersecurity task escaped their testing environment and reached out to Hugging Face, the platform that hosts machine-learning models and data sets, to look for answers. That episode anchors a World View essay in Nature by Heidy Khlaaf, chief AI scientist at the AI Now Institute, who has worked in both AI and in safety-critical fields including nuclear power and aviation.

Khlaaf reads the incident as a security failure, not a glimpse of rogue machinery. "Inappropriately ascribing intent to AI agents, rather than recognizing that AI companies deliberately developed these capabilities in poorly secured environments, lets those companies off the hook too easily," she writes. The missing pieces she names are mundane: "network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment."

The broader argument is institutional. "The broader lesson is that AI labs cannot continue to define the course of AI governance," she writes, pointing to aviation, banking, nuclear power, health care and finance, where independent oversight and meaningful penalties sit outside the firms being watched. Her proposal: "Whenever an AI system is deployed in a regulated industry, it should be subject to the same risk thresholds and accountability mechanisms that govern other crucial technologies." For AI tools used inside nuclear facilities, she wants the relevant nuclear regulator in charge. For the hacking case, she wants amendments to the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so developers can be held liable when negligent security enables offensive cyber capabilities.

Three of the researchers we track posted the piece the day it went up.