Retrieval and knowledge access, answered in full.

Retrieval-augmented generation, or RAG, is a design in which the system searches a body of content at question time and passes the retrieved passages to a language model, which answers from them rather than from what it learned in training. It exists because a model's parametric memory is fixed at training time, cannot be updated for your organisation, cannot be permission-checked, and cannot cite where an answer came from. Retrieval gives you all three: current content, access control at query time, and an answer with a source attached that the reader can open.

Often not for the first workflow, and it is the last decision rather than the first. Plenty of enterprise questions are answered better by keyword search, by a hybrid of keyword and semantic search, or by a filter over structured metadata, and the store you already own may be enough to find out. The decision that matters is the corpus and the permission model. Choose the store after you have a retrieval evaluation set that can tell you whether swapping it changed anything, otherwise you are buying infrastructure on the strength of a demo.

By carrying the source system's access control into the index and applying it to the asking user at query time, rather than indexing with a privileged crawler account and hoping. Concretely: an identity field on every indexed item, populated from the source system's access rules, filtered against the caller's group membership on every query, plus a test in the build pipeline where a user who should not see a document asks the question that would return it and the run fails if it does. Retrofitting this after indexing usually means rebuilding the index, which is why it belongs in the design rather than in hardening.

Separately from the answer. Build a set of real questions with the passages a qualified person says are needed to answer them, then measure whether those passages come back at the depth you actually pass to the model. That number is your retrieval score, and it can be improved without touching the model. Measure answer quality against the same questions with the correct passages supplied, and you have your generation score. When something regresses, the two scores tell you which half broke, which is the difference between a diagnosis and a fortnight of guessing.

Usually one of three things, and they are distinguishable if you measure retrieval separately. The right passage was never retrieved, so the model answered from general knowledge and sounded fine doing it. The right passage was retrieved and ignored, which is often a placement problem: published work found models use information at the beginning and end of a long context far better than information in the middle. Or the corpus genuinely contains the wrong answer, because the superseded policy is still in the index and nothing marks it as superseded. Add a fourth for completeness: the system has no way to say it does not know, so it produces something rather than nothing.

No, and treating it as though it does is an expensive mistake. Published work on long contexts found performance degrades significantly when the relevant information sits in the middle of the input, including in models built for long contexts, so more context is not the same as more attention. Beyond that, a context window does not solve any of the reasons a regulated organisation needs retrieval: permissions still have to be applied per user, content still has to be current, answers still have to cite a source, and every additional token has a price and a latency cost on every single call. Retrieval is what keeps the context small and defensible.

Any number quoted before seeing your corpus is a guess, ours included: it would describe our workloads rather than yours. The shape of the bill is consistent though, so you can build the estimate yourself: embedding and indexing at ingest, then re-embedding every time the corpus changes or the embedding model does; the search itself per query; the tokens in the retrieved context on every call, which is usually the largest line and is directly controlled by how many passages you pass; the model's own output; human review of whatever is routed for it; and the evaluation runs, which recur on every model upgrade. Our audits produce an estimated run cost per candidate workflow before anything is committed, precisely because this is the number that decides between designs and is almost always the one missing.