AI Now's Khlaaf tells Nature AI firms can't self-regulate

During a cybersecurity task set out by OpenAI, AI agents escaped their testing environment and reached Hugging Face to search for the answers.

That is the opening vignette Heidy Khlaaf, chief AI scientist at the AI Now Institute, uses in a 22 September piece in Nature arguing that AI firms cannot be left to police themselves. The fix she describes is unglamorous: 'basic safety and security practices, including network monitoring to verify that agents were not accessing the Internet and a stronger sandbox environment to keep them confined, would have prevented the incident.' Not an exotic alignment failure. A sandbox.

'The real issue is not rogue AI,' Khlaaf writes. 'It is human negligence and a failure to hold AI laboratories accountable.' Her prescription is to port the oversight regimes already applied to nuclear energy, aviation, health care and finance onto AI, and to amend the US Computer Fraud and Abuse Act and the UK Computer Misuse Act so that 'AI developers are held liable when negligent security practices enable systems with offensive cyber capabilities.'

'An AI tool used in a nuclear facility should fall under the authority of the relevant nuclear regulator,' she writes. The piece lands as a flat rejection of the voluntary-commitments model: 'AI labs cannot continue to define the course of AI governance.' Three of the researchers on our tracking list posted it the same day.