r/ArtificialInteligence: Production Team Documents That RAG App Failures Come From Retrieval Quality Not the LLM — Version-Mismatch Chunks Were Root Cause

A practitioner who built an internal knowledge base tool documents a weeks-long debugging path where the team adjusted prompts and model settings before discovering the actual root cause of wrong answers: retrieval was returning version-mismatched documentation chunks where two incompatible versions of the same content coexisted without metadata to distinguish them. The post argues that RAG failures diagnosed at the prompt engineering layer are almost always retrieval problems — wrong chunks in, wrong answers out — and that chunking strategy, metadata filtering, and content freshness signals deserve investigation before any model-level tuning. Community responses surface a consistent pattern where developers systematically eliminate retrieval quality last because it's harder to instrument than prompt adjustments.