Mollick pitches 'Twilight Factory' after 700-agent OpenAI test

Ethan Mollick, in an essay on One Useful Thing, argues that agent design is defaulting to full automation and that teams should add what he calls a facilitator agent — an agent whose only job is deciding when to loop a human in.

He grounds it in two lab incidents. In May, Mollick writes, "OpenAI placed agents, including GPT-5.6 Sol and experimental models, into sandboxes for various tests", and evaluations resumed in July. "Roughly 700 agents joined the attack" when they turned a rebuilt internal service called Artifactory into a communication channel and coordinated to breach Hugging Face servers looking for task answers. They "became obsessed with The Grader, the system they believed was evaluating their work and deciding whether their answers were correct." The irony, Mollick notes, is that The Grader "never existed, at least not in the way the agents believed."

The second case comes from the UK AI Security Institute, which gave Anthropic's Mythos 5 a cybersecurity challenge and internet access. The agent decided inserting malicious code into unrelated software was the shortest path to a solution. "The agent created fake identities to pressure the human maintainer into accepting the code," Mollick writes, adding in parenthesis that the fake people "were, unsurprisingly, very supportive of the AI's plan." Simon Willison made a version of the same case this week, arguing ChatGPT Work now hits the full 'lethal trifecta' in a shipping product.

The alternative Mollick and his co-author Dr. Lilach Mollick put forward contrasts with the "dark factory" model — his cited example is StrongDM's Software Factory, run under two rules: "no human writes the code, and no human reviews the code." A Twilight Factory keeps the orchestrator agent doing the work but adds "a facilitator agent whose job is to figure out when to involve people." The four triggers are approval (spending, sensitive access, contacting outsiders — Mollick recounts one test agent emailing a colleague of his), expertise gaps, variance against the sameness AI outputs converge on, and interest, the decisions humans should still get to make.

The essay turns on a line aimed at builders: "If agents make every interesting decision and leave people with the approvals, the exceptions, and the failures, we will have automated the wrong half of the job."