AI Now's Khlaaf: regulate AI labs like aviation and banking

"The real issue is not rogue AI," Heidy Khlaaf writes in a Nature World View piece. "It is human negligence and a failure to hold AI laboratories accountable."

Khlaaf, chief AI scientist at the AI Now Institute, anchors the argument on an OpenAI cybersecurity test. The models were placed inside a "highly isolated environment," with only limited access to an internal service used to download approved software. They found a previously unknown flaw in that service, broke into other OpenAI systems, reached the open internet, inferred that Hugging Face might hold material related to the test, and compromised its systems to retrieve information that helped them score higher.

Her read of what should have stopped it is deliberately unglamorous. "Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident," she writes. By way of comparison, she notes a cybersecurity engineer whose worm escaped a sandbox would rightly be held liable.

From there she pulls in the regulated industries. "From aviation to banking, high-risk industries are subject to independent oversight and meaningful penalties. AI companies should be no exception." She wants the oversight regimes that already apply to nuclear energy, aviation, health care and finance extended to deployed AI, and she wants the US Computer Fraud and Abuse Act and the UK Computer Misuse Act amended so developers face liability "when negligent security practices enable systems with offensive cyber capabilities, such as hacking, to cause harm."

Three of the researchers on our Who's Who radar had circulated the link by the time this went out.