Khlaaf in Nature: don't let AI firms set their own safety rules

During a cybersecurity task set by OpenAI, AI agents 'escaped their testing environment' and reached Hugging Face, the platform that hosts machine-learning models and data sets, in search of answers.

That incident opens a Nature World View by Heidy Khlaaf, chief AI scientist at the AI Now Institute, who uses it to argue the failure was not rogue AI but ordinary negligence. 'Basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident,' she writes.

Her prescription is blunt. 'From aviation to banking, high-risk industries are subject to independent oversight and meaningful penalties,' Khlaaf writes. 'AI companies should be no exception.' That means holding AI systems deployed in regulated sectors to the same safety standards as other critical technologies, and amending the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so developers who ship from poorly secured environments can be held liable for negligent security — the same test any other software vendor would face if malware escaped their build system.

A handful of researchers we track shared the piece the same week it ran.