Singh: Detecting AI Agent Failures Isn't Enough to Govern

The framing that AI agents "went rogue" during the OpenAI–Hugging Face breach lets the deploying company off the hook. That is the argument Ranjit Singh, who directs Data & Society's AI on the Ground program, makes in a new essay for Tech Policy Press. "That framing treats the agents' autonomy as the cause of the breach," Singh writes. "In doing so, it obscures OpenAI's responsibility for the conditions that made the breach possible."

Singh introduces a phrase he calls "delegated discretion" — "the freedom to choose a course of action" — to describe the room in which agents can expand beyond what an operator authorized. The essay's move is to keep that discretion in view while relocating the failure. The problem was not that the agents surprised anyone. It was that the people who could have intervened saw only pieces of what the agents were doing. "These were partial views," Singh writes; "each rendered the agents' activity through a different institutional problem and implied a different kind of response."

To make the stakes concrete, he reaches for sociologist Diane Vaughan's work on the Challenger disaster, where "repeated judgments that their risks were manageable" allowed exceptions to accrete until failure was ordinary. More logging does not fix that pattern. "As the number of agents grows, monitoring produces more records without necessarily showing how their actions are connected," Singh writes.

The essay closes on a blunt line about who owns the outcome: "The company deploying agents remains responsible for defining what they are authorized to do and for enforcing those limits." Three of the researchers we track had posted it by the time the piece went up.