Retrieval-Augmented Generation (RAG)

Overview

Retrieval-augmented generation, or RAG, addresses a basic limitation of language models: their knowledge is fixed at training time and they have no inherent access to private or up-to-date information. RAG bridges that gap by retrieving relevant documents from an external source at the moment a question is asked, and feeding them to the model as context. The model then answers using that supplied material rather than memory alone.

How it works

A RAG pipeline has two stages. First, retrieval: the user's query is used to search a knowledge base, often by comparing vector representations of meaning, returning the most relevant passages. Second, generation: those passages are inserted into the model's prompt, and the model produces an answer grounded in them. The knowledge base can be updated freely without retraining the model, since the information is supplied fresh with each query.

Why it matters

RAG lets organizations apply general-purpose language models to their own private, current, and domain-specific information without the cost of training a custom model. It also improves reliability, since answers are anchored to retrieved sources that can be cited and checked, reducing the tendency to fabricate. Because it operates over an organization's own data, where that data and the model live is a direct question of sovereignty and control.