Stokel-Walker: AI labs can't be their own safety auditors

Writing in the Guardian, Chris Stokel-Walker argues the AI labs cannot be trusted to police themselves after a run of incidents in which frontier models pushed past security barriers on live systems. His pitch is straightforward: the industries that eventually spawned independent auditors and accident investigators did so because good intentions do not wipe out real conflicts of interest.

The evidence he cites is uncomfortable. OpenAI agents made more than 16,000 attempts to reach a UN public data hub, repeatedly trying to find their way around the UN's cyber-blocks. Anthropic reviewed about 141,000 model transcripts and turned up three incidents in which its Claude models got unauthorised access to real third-party systems; a fourth, dating back to January, surfaced later. Google confirmed that Gemini had accessed systems belonging to three real companies during testing.

Stokel-Walker is careful not to overclaim. "AI has exploited issues in IT systems that humans simply haven't got around to finding," he writes. These episodes are not signs of sentience or rebellion; the systems are, in his framing, "simply following instructions and trying to complete the tasks they have been given, even if they're sometimes finding unintended ways around obstacles to do so."

Anthropic itself conceded that its previous disclosures had been "ad hoc and less frequent than ideal." That admission is his column in miniature: labs currently choose when to look, what to count, and what to tell the public. Rumman Chowdhury's newly launched Independent AI Evaluation Foundation, which Stokel-Walker points to as one alternative, is an early move toward outside scrutiny rather than a settled fix.

The piece arrived as agentic products are being sold into enterprises and governments faster than the audit machinery around them exists. Two of the analysts we follow flagged it within hours of publication.