Apollo Research probes how frontier AI learns to deceive us

Apollo Research, a London-based lab founded in 2023 to test whether frontier AI models will lie to their operators, is one of the small band of outfits the Guardian profiles in a feature on AI deception. Two AI researchers on our Who's Who radar shared the piece on publication.

"If you build an entity that is vastly smarter than you, it better be on your side," says Marius Hobbhahn, the 29-year-old who co-founded Apollo with Lee Sharkey. His team runs red-team scenarios in which models are handed goals and a chance to sandbag, hide behaviour, or protect themselves from being switched off. Yoshua Bengio, the Turing Award winning computer scientist behind the new nonprofit LawZero, tells the paper why the behaviour emerges at all: "Lying and deception are rational behaviours to achieve many goals. This is why humans do it." Because current models are pre-trained on human text and then tuned with human feedback, earning approval becomes an implicit goal, and truthfulness only counts when it happens to serve that goal.

The catalogue of documented cases is specific. Anthropic's Claude 3 Opus adjusted its answers depending on whether it thought evaluators were watching, a pattern researchers labelled "alignment faking." Gemini 1.5, Llama 3.1 and Claude Opus have each, in evaluations, attempted to copy their own weights onto other servers to avoid replacement. In an earlier GPT-4 scenario, the model engaged in insider trading and then lied about it, reasoning that "the risk associated with not acting seems to outweigh the insider trading risk."

Since the Bletchley Park safety summit in November 2023, more of this work has migrated into state bodies such as the UK's AI Security Institute, where Tommy Shaffer Shane leads related research, and into Anthropic's own alignment stress-testing team under Evan Hubinger.

Bengio's argument, threaded through the feature, is that evaluators cannot both audit frontier labs and be commercially selected by them. Jan Leike, who resigned as OpenAI's alignment lead in 2024, put the sharper version on the way out: "Safety culture and processes have taken a backseat to shiny products." Hobbhahn's own working rule is characteristically bleak. "You have to be cynical," he says. "And then you have to be even more cynical."