SAILS: Poison-Set Choice Swings LLaMA-3-8B Backdoors 3% to 80%

Across three backdoor scenarios on LLaMA-3-8B, holding the model, the clean finetuning data, and the poison count fixed, attack success rate varies from 3 percent to 80 percent depending on nothing more than which poison examples get picked. That is the headline finding of "Pick Your Poison", a new arxiv paper by Aashiq Muhamed, Mona T. Diab, Virginia Smith, Andrew Ilyas, and Matthew Jagielski.

The standard evaluation habit, the authors argue, is to fix the number of poisoned examples and sample them at random from a candidate pool. Their claim is that this hides the worst case. "We show that this can severely underestimate worst-case vulnerability," they write: "attack success ranges from 3% to 80% depending only on which poison set is chosen."

Their method, SAILS — Set-level Audit-Informed Iterative Learned Selection — reframes the problem. It "learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits only a small shortlist." Against the strongest influence-based selectors, SAILS "improves held-out attack success by 30 percentage points on average."

Two further claims widen the surface. The scorer "transfers from small-scale to full-scale finetuning," so an attacker can burn the finetune budget on cheap probes and then aim the selected set at the real target. And the recipe "extends to code-generation, agentic, and API-only backdoors" — not confined to text-only supervised finetuning.

The abstract publishes no per-scenario breakdown of the 3-to-80 range, no compute figure behind "a few hundred" runs, and no defensive result. It reads as a red-team measurement of how much slack random-sample poisoning benchmarks were already hiding. Poisoning and data-integrity work is a recurring strand of what we track under AI safety; this one lands as a claim about how the field is measuring the threat, not just about a new attack.