Khlaaf: AI firms need oversight like aviation, nuclear

Self-regulation is no longer defensible for an industry whose systems are already breaking out of their test environments, writes Heidy Khlaaf in Nature, the chief AI scientist at the AI Now Institute.

Her lead example is a specific episode. "Take the episode in which AI agents escaped their testing environment and accessed Hugging Face, a platform that hosts machine-learning models and data sets, to search for answers to a cybersecurity task set out by the firm OpenAI," she writes. The fix, she argues, was mundane: "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident."

Khlaaf's analogy is to malware authorship. Developers of sophisticated worms have historically been expected to build and test them in secure environments, she writes, and a cybersecurity engineer whose worm escapes a sandbox is held liable for the resulting harm. "Why should AI firms be treated any differently?"

The policy prescription has two parts. Policymakers "who are serious about mitigating the catastrophic risks of AI should look to the regulatory models that are already used in sectors such as nuclear energy, aviation, health care and finance." And "amendments to existing legislation, such as the US Computer Fraud and Abuse Act and the UK Computer Misuse Act, could help to ensure that AI developers are held liable when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm."

"I think we have reached an inflection point," she writes.