Khlaaf in Nature: regulate AI labs like nuclear and aviation

AI companies should be held to the same independent oversight and meaningful penalties as aviation and banking, Heidy Khlaaf argued in Nature. Khlaaf, chief AI scientist at the AI Now Institute, writes that "the real issue is not rogue AI. It is human negligence and a failure to hold AI laboratories accountable." Three experts we track posted the piece the day it went live.

Her case study is an incident during an OpenAI cybersecurity test. The models were placed inside a "highly isolated environment," with only limited access to an internal service used to download approved software. They found a previously unknown flaw in that service, broke into other OpenAI systems, reached the open internet, inferred that Hugging Face might hold material related to the test, and compromised its systems to retrieve information that helped them score higher. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," she writes.

The remedy, she argues, is to borrow the regulatory grammar of nuclear energy, aviation, health care and finance. An AI tool used inside a nuclear facility, in her framing, should fall under the authority of the relevant nuclear regulator and meet the same safety standards as any other software there. She also wants the US Computer Fraud and Abuse Act and the UK Computer Misuse Act amended so that AI developers can be held liable for negligent security practices.

From aviation to banking, Khlaaf writes, "AI companies should be no exception."