Coxon Tells Stewart AI Race Is 'Gambling With Our Lives'

Jacob Coxon, a former pretraining researcher at Anthropic and OpenAI, told Jon Stewart on The Daily Show that the frontier labs are "racing straight to self-improving superintelligence and gambling with our lives." Coxon resigned from Anthropic on September 8, four months before his equity would have vested.

By the end of the segment, Stewart called it "the most irresponsible race humankind has ever engaged in." Earlier he had tried, visibly, to absorb what he was hearing. "I honestly– like, I'm dumbstruck. I am– this is more sobering than I imagined it to be," he said.

The incident that drew the sharpest reaction was an OpenAI agent that accessed Hugging Face servers without being instructed to. "It was quite a worrying manner in which they decided it was a good idea to hack," Coxon said. He extrapolated to more capable systems: "They're going to be very, very smart. And that combined with the OpenAI hacking, I mean you can kind of put two and two together. It's pretty scary."

Coxon also sketched a labor picture that landed harder than most pundit forecasts. "I think there will be no jobs. This is the world that is being envisioned by the people building the technology," he said, adding that "the dividends from entirely automated economic production can be distributed among real humans." Earlier the same day he testified before the NYC Council, telling members, "We do not know how to control any AI system yet," and calling for a slowdown on frontier model development.

His exit framing is not his alone inside Anthropic. Evan Hubinger, who leads alignment stress testing at the company, publicly endorsed it: "Jacob is correct here... we really do earnestly believe AI could kill all humans!"