Khlaaf in Nature: AI labs cannot be left to self-regulate

Heidy Khlaaf, chief AI scientist at the AI Now Institute, argued in a Nature World View on 22 September that frontier AI firms should be pulled under the same independent oversight that governs aviation, banking, nuclear power and healthcare, rather than being left to write their own safety commitments. Three of the researchers we track posted the piece the day it ran.

Her central claim is blunt. 'The real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable,' Khlaaf writes, adding that 'AI labs cannot continue to define the course of AI governance.'

The piece hangs on one worked example: 'AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets,' while attempting to solve a cybersecurity task set by OpenAI. Khlaaf argues that routine engineering controls, specifically 'network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident.'

Her proposed fix is legal rather than voluntary: amend the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so developers face liability for negligent security. Khlaaf, who has worked in both nuclear power and aviation safety and has done work for OpenAI and the UK government's AI Security Institute, draws the parallel plainly. If a cybersecurity engineer let a worm escape a sandbox, she writes, they would rightly be held liable for any resulting harm; the piece asks why AI firms should be treated any differently.