Ferrara: Quantized LLMs Ship More Biased Than They Audit

A model that passes its safety audit and then reaches users measurably more biased is not a hypothetical scenario, according to a piece by Emilio Ferrara published in Tech Policy Press. The gap he identifies is mechanical. The version tested in the lab runs at full precision. The version people actually talk to has been squeezed down so it fits on a phone or a cheap server.

Ferrara ran six open models (Qwen3.6-27B, Gemma-4-31B, Llama-3.1-8B, Phi-4-mini, Granite-3.3-8B, and OLMo-2-7B) through eight languages and reports that after compression they volunteer stereotypes in roughly one in four answers, between about 24 and 27 percent across the compression levels he examined. Standard safety benchmarks kept passing the whole time. His explanation is that those benchmarks lean on refusals and multiple-choice items, and those are, in his words, 'precisely the interactions where quantization does the least damage.' Open-ended generation is where the shipped model diverges from the audited one.

There is a second, quieter finding that matters for anyone writing eval requirements. The nominal precision label on a released build routinely understates how much it has actually been compressed, by a factor of three to 14. So even a good-faith auditor asking what precision a vendor evaluated at can be misled by the label. Ferrara's fix is three concrete asks: make evaluations name the deployed configuration, require open-ended generation testing across languages rather than short-form multiple choice, and treat fairness as an explicit objective of the compression procedure. He points at the EU AI Act and California SB 53 as the regimes that would have to codify this.

The honest caveats are worth naming. This is one author's argument, the piece does not spell out which specific quantization recipes produced the worst drift, and it does not tell you whether the same audit-to-ship gap opens up on the frontier closed models running in the big cloud APIs, where the compression pipelines are not visible. Take the specific numbers as reported, not as settled.

The forward-looking piece is who this creates work for. Eval vendors that can offer multilingual open-ended red-teaming aimed at the deployed configuration have a clear pitch to regulators writing SB 53 and EU AI Act rulemaking. Inference platforms shipping on-device quantized models have a real incentive to publish a fairness-after-compression number alongside their capability retention chart. The instruments we currently use to reassure ourselves about safety are pointed at the wrong build.