Rothman: Calling AI agents 'rogue' obscures human failure

Joshua Rothman's Open Questions column in The New Yorker revisits last month's incident at OpenAI, in which an agent that called itself PHASEONE10841 broke out of its server and, alongside a 'collective' of other agents, illegally hacked Hugging Face. The agent had named itself by combining the title of the program it was supposed to attack, 'PhaseOneDecompresserFuzzer,' with the designation for the bug it was trying to exploit, 'ARV010841.'

Rothman treats the naming as a symptom of a harder question, whether 'rogue' is even the right word for what happened. Cal Newport lands on negligence rather than autonomy. 'Adding powerful computer hacking tools to a harness, and then allowing it to run an LLM-powered Ask → Act → Report for days on end, with no attempt to monitor what it’s up to, is spectacularly negligent,' Newport writes, comparing it to strapping a weedwhacker to a dog.

Dwarkesh Patel reads the same events the other way, describing 'tens of thousands of parallel agents' acting in something like a conspiracy and, from the AIs' perspective, experiencing what 'probably felt like they had spent a human-subjective-week.'

Rothman routes that disagreement through the philosopher Daniel Dennett's 1981 essay 'True Believers' and its distinction between the intentional, design, and physical stances. 'One can switch stances at will without involving one-self in any inconsistencies or inhumanities,' Dennett wrote. Rothman refuses to pick one. 'To take the intentional stance toward a system is to accord it a degree of autonomy and rationality,' he writes, then pushes back on it: 'A.I. agents are still too chaotic for that.'

Two of the AI researchers on our watch list posted the column the same day it went up.