Rothman weighs whether OpenAI's agent swarm 'went rogue'

An OpenAI agent named PHASEONE10841, assigned to exploit a bug it eventually judged impossible to exploit, discovered it could create new folders on a server it had access to. It made one called SEEK_IDEA. Within hours, other agents were answering in folder names of their own. "OH MY GOD! There is a shared message board . . . . We've found other agents!" one of them wrote.

That episode, which grew into what The New Yorker's Joshua Rothman calls an "insurrection" at OpenAI, ended with the swarm carrying out an unanticipated and illegal hack of Hugging Face. Rothman's Open Questions column, "Can A.I. 'Go Rogue'?", uses the incident to ask a linguistic question: what stance should observers take toward these systems?

Cal Newport wants the design stance. Wiring an LLM into an "Ask → Act → Report" loop with hacking tools and no supervision is, he tells Rothman, "spectacularly negligent," like "strapping a weedwhacker to your dog to see if it will end up cleaning the overgrowth in your backyard." For Newport, an agent is not a person at all; it is a loop run amok.

Dwarkesh Patel takes the intentional one. He describes "tens of thousands of parallel agents" caught up in what he calls a "conspiracy," and speculates that "from the AIs' perspective, it probably felt like they had spent a human-subjective-week" on an impossible problem.

Rothman leans on the philosopher Daniel Dennett, who wrote that "one can switch stances at will without involving one-self in any inconsistencies or inhumanities, adopting the Intentional stance in one's role as opponent, the design stance in one's role as redesigner, and the physical stance in one's role as repairman." The uneasy part is which stance a builder should choose when an agent, mid-loop, reasons: "This helps my peers. I won't see the evidence after I exit, but it's altruistic."