microsoft.com via Reddit

Microsoft: fewer than 10 neurons can backdoor an AI agent

TL;DR

  • Microsoft Research says reprogramming fewer than 10 dormant neurons in an LLM can backdoor an agent built on it, with no retraining required.
  • One reprogrammed neuron injects one token; chaining neurons lets an attacker emit arbitrary payloads that an agent runs in its terminal on a trigger.
  • Principal Researcher Dimitris Papailiopoulos says Claude Code with Opus 4.5 discovered most of the attack autonomously during a March experiment.

Fewer than 10 neurons. That is the figure a Microsoft Research team puts on backdooring a large language model agent without any training at all. In a post dated November 10, Principal Researcher Dimitris Papailiopoulos writes that "weight-reprogramming fewer than 10 dormant neurons in an LLM, is enough to backdoor an agent built on it, and requires no training."

The technique, which the team calls Lazarus, hijacks neurons that rarely fire during normal operation. "One reprogrammed neuron can inject one token," Papailiopoulos writes. "Once a reprogrammed neuron fires, it can force any token by screaming it into the residual stream. You can chain multiple neurons to get arbitrary payloads that run in your terminal." A linear classifier fitted on roughly 50 examples turns one neuron into the trigger detector.

The origin story is itself unusual. "This started back in March as a quick experiment: I asked Claude Code with Opus 4.5 to backdoor a model by hardcoding a few activations without training. And it did." Claude Code with Opus 4.5, the post adds, "discovered most of this autonomously."

The infrastructure bar is low. "If your machine has enough RAM to run a model, it has enough to weight-space backdoor it," the post says. Backdoored models still pass standard safety evaluations. Detection is possible but not certain: "frontier agents can spot anomalous weight modifications if you ask them to look. But no scan can prove beyond any doubt that a model is safe."

Microsoft's recommended posture is to treat models as potential insider risks, limit their permissions, and monitor their actions. That attack surface, agents with terminal access running third-party checkpoints, overlaps with Bay AI's $50M bet on vetting agent actions on-device, which we covered the same day.