SimpleLoRIF Flags Fine-Tuning Data That Erodes LLM Safety

"Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful." That is the opening sentence of a new preprint from Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab and Pratyush Maini, proposing SimpleLoRIF, a gradient-based detector for the examples responsible.

The authors score every instruction example by how closely its gradient aligns with gradients produced on unsafe responses. On Mistral fine-tuned on misaligning data, the detector reaches "AUC as high as 0.865 in the gradient space of misaligned models." On a fine-tuned Qwen, the figure is 0.604.

Then they used it to clean a real corpus. SafeTulu removes "the 20% of examples most gradient-aligned with unsafe responses" from Tulu 3's 939,343 examples, using 43,469 unsafe prompts from SafeRLHF as the signal. GCG attack success drops from 47.7% to 37% on Qwen, 38% to 30% on Llama, and 48% to 41% on Olmo. An LLM-as-Judge filter run on the same corpus leaves the Qwen rate at 47.3%.

The embedding step took "approximately 1 hour on 8 H100 GPUs" by using rank-1 factorization and limiting backpropagation to the final four transformer blocks. Scores on MMLU Pro, IFEval, GSM8K and MATH 500 held or improved after filtering, which the paper describes as "largely preserving or even improving capability."

The headline AUC comes from one fine-tuned model, and the attack result is a single attack class. The paper does not break out per-detector numbers across every setting.