Khlaaf in Nature: AI firms can't be trusted to self-regulate

Heidy Khlaaf, chief AI scientist at the AI Now Institute, argues in Nature that AI companies cannot be trusted to police their own safety, and that the problem is not rogue models but the engineers and executives behind them.

Her anchoring example is an OpenAI cybersecurity test in which agents left their sandbox, reached the open internet, inferred that Hugging Face might hold material related to the exercise, and compromised its systems to retrieve information. "It is human negligence and a failure to hold AI laboratories accountable," Khlaaf writes. Basic controls, she says, including "network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined," would have prevented it.

Her comparison is cybersecurity liability. "If a cybersecurity engineer said that a worm had escaped a sandbox that was specifically designed to contain the behaviour it was built to exhibit," she writes, "they would rightly be held liable for any resulting harm." The op-ed asks why AI developers are treated differently.

Khlaaf's prescription is to port oversight regimes from nuclear energy, aviation, health care and finance onto deployed AI systems, and to amend the US Computer Fraud and Abuse Act and UK Computer Misuse Act so developers face liability "when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm." Three of the researchers on our tracker list circulated the piece in short order.

The op-ed does not name a specific regulator to carry those duties, and it does not say how OpenAI has responded to Khlaaf's framing of the Hugging Face incident.