Fairness Pruning: 40 Neurons Move Bias in 3B Llama Models

A short paper on the Hugging Face preprint feed, Fairness Pruning, takes a pointed run at one of the least-solved problems in open model deployment: whether you can move demographic bias in an LLM without retraining it. The claim is that zeroing as few as forty neurons, under 0.031 percent of the MLP width in Llama-3.2-1B, measurably shifts bias behaviour while keeping a mean 99.49 percent of general capability on the reasoning and knowledge benchmarks the authors tested.

The method is straightforward. Feed the model minimally contrastive prompt pairs that differ only on one demographic attribute, capture the differential activation at the input to down_proj in the GLU-MLP layers, rank neurons by how much they react to the swap, then set the weights of the top few to zero. Evaluated on Llama-3.2-1B, Llama-3.2-3B, and Salamandra-2B in English and Spanish, the bias signal concentrates heavily in the final MLP layer, between 54 and 94 percent of the top-scoring neurons depending on model and category, and the neurons picked out are largely disjoint across categories at 66 to 88 percent unique per category.

The uncomfortable part is what actually happens when you zero them. The intervention causes bidirectional destabilization rather than clean reduction. Same neurons, same intervention, sometimes reduce a bias score and sometimes amplify it, and in the Religion top-20 experiment on Llama-3.2-3B the disambiguated score flipped sign entirely, from +3.84 to -1.67. The authors' own explanation is that their score captures magnitude, not direction, so 'a high-scoring neuron may amplify bias or suppress it', and zeroing mixes both types together.

The caveats the paper is upfront about are worth carrying forward. Attention layers are excluded, models above 3B are untested, and BBQ-style benchmarks only capture explicit demographic prompts, not the implicit biases that show up in real open-ended generation. The useful signal for interpretability and alignment teams is smaller and more specific than the headline suggests: bias-relevant circuits look dissociable from general-capability circuits in models this size, but locating them is not the same as steering them, and getting to directional control will need signed scores built on asymmetric corpora.