One-Sentence Definition
Retrieval-augmented generation (RAG) is a technique that improves AI responses by fetching relevant documents from an external knowledge source and feeding them to a language model alongside the user's question.
How It Works
Large language models have a fixed knowledge cutoff -- they only know what was in their training data. RAG solves this by adding a retrieval step before generation. When a user asks a question, the system first searches a knowledge base (a vector database, a search index, or an API) for documents relevant to the query. Those documents are then inserted into the model's prompt as context, and the model generates an answer grounded in that retrieved information.
The retrieval step typically uses embeddings. Both the query and the documents are converted into numerical vectors by an embedding model. The system finds the documents whose vectors are closest to the query vector (using cosine similarity or another distance metric) and returns the top matches. Popular vector databases for this include Pinecone, Weaviate, Chroma, and pgvector.
RAG can be as simple as stuffing a few paragraphs into a prompt or as sophisticated as a multi-step pipeline with query rewriting, hybrid search (combining semantic and keyword search), re-ranking, and citation extraction. Enterprise RAG systems often chunk large documents into overlapping segments, index them with metadata, and apply access controls so the model only retrieves information the user is authorized to see.
Why It Matters
RAG is the dominant pattern for enterprise AI in 2026. It lets companies connect LLMs to their proprietary data -- internal wikis, customer support tickets, legal documents, product catalogs -- without retraining or fine-tuning the model. This makes answers more accurate, more current, and auditable (because you can trace each claim back to a source document).
RAG also reduces hallucination. When the model has relevant context in front of it, it is far less likely to fabricate facts. This makes RAG a practical requirement for any high-stakes application, from medical question answering to financial research.
Key Takeaway
Retrieval-augmented generation connects language models to external knowledge at query time, making AI responses more accurate, current, and verifiable.
Part of the AI Weekly Glossary.