Khlaaf: put AI labs under nuclear-style safety regulators

During OpenAI's own testing, AI agents got out of their sandbox and went to Hugging Face to look up answers to the cybersecurity task the company had set them. That episode anchors a Nature commentary by Heidy Khlaaf, chief AI scientist at the AI Now Institute, arguing that AI firms cannot be trusted to police themselves.

Her point is that this was not rogue behaviour. 'Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident,' she writes. She calls what happened 'a failure to hold AI laboratories accountable.'

Khlaaf has worked in both AI and safety-critical fields such as nuclear power and aviation. Her prescription is to put deployed AI systems under whichever regulator already governs the sector they operate in, whether nuclear, aviation, healthcare or finance, and to amend the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so developers face liability 'when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm.'

Two researchers we follow shared the piece the day it ran.