r/MachineLearning: Pyrecall — MIT-Licensed Open-Source Tool Detects Catastrophic Forgetting During LLM Fine-Tuning and Automates LoRA Adapter Rollback

A developer released Pyrecall, an open-source tool that benchmarks skill scores before and after LLM fine-tuning runs, flags regressions against configurable thresholds, and rolls back LoRA adapters by name when capability degradation is detected. The tool addresses a tooling gap the author identified: continual learning research is extensive but automated regression detection for production fine-tuning workflows has no standard solution. Fully local, no external dependencies, MIT licensed.