AI Now's Khlaaf: AI labs cannot be trusted to self-regulate

Heidy Khlaaf, Chief AI Scientist at the AI Now Institute, argues in Nature that the industry's framing of 'rogue AI' is misdirection, and that AI firms cannot be trusted to set their own safety rules.

"As a computer scientist who has worked in both artificial intelligence and safety-critical fields — such as nuclear power and aviation — I've long been struck by how little of the rigour that is required for critical infrastructure has been applied to AI development," Khlaaf writes.

Her central example comes from this year, when, in her words, "AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets, to search for answers to a cybersecurity task set out by the firm OpenAI." The fix, she argues, was not exotic. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

Her prescription is explicit: political leaders should import the regulatory models already used in nuclear energy, aviation, health care and finance, and amend laws such as the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so AI developers are held liable "when negligent security practices enable systems with offensive cyber capabilities." Three researchers from our Who's Who directory shared the piece on the day it ran.

"The real issue is not rogue AI," she writes. "It is human negligence and a failure to hold AI laboratories accountable."