LASAGNA Finds Three RAG Techniques Deliver +17%, Not +47%

Three retrieval techniques that each look strong in isolation predicted a +47% combined improvement over baseline in a new 450-configuration study, but delivered only +17% when actually stacked together. That is the headline finding of a paper released October 22 introducing LASAGNA, an open-source framework for controlled RAG evaluation across eight benchmarks and corpora ranging from 5K to 200K documents.

"Individually strong techniques from different phases of a RAG pipeline do not necessarily compose," the paper states. The median shortfall across 17 technique combinations tested was 41%, with individual shortfalls ranging from 7% to 72%.

The study pre-declared three hypotheses; all three were confirmed. The second finds that retrieval quality and answer faithfulness are essentially uncorrelated at Pearson r = -0.01 (p = 0.91), with different pipeline stages driving each metric. Query augmentation and agentic gating dominate faithfulness; embedding, enrichment and reranking dominate retrieval quality. "Optimizing one can hurt the other," the authors write.

The third finding is about scale. A rank crossover between RAG profiles occurs somewhere between 15K and 25K documents, and at 200K the "Balanced" profile exceeds "Standard" by 13.7% relative (p = 0.003). "The combination that wins on a small corpus may not remain best at large scale," the paper says.

The run spanned 358,215 individual query evaluations and cost roughly $5,200 across 310 wall-clock hours. One surprising sub-finding: a 2019 MS-MARCO MiniLM-L6 cross-encoder ranked above all four recent rerankers the authors tested. The framework is on GitHub with 53 built-in components across 30 methods and six vector backends. It is one of 14 RAG studies we have tracked in the last 90 days.