Retrieval and knowledge access, answered in full.

Retrieval-augmented generation, or RAG, is a design in which the system searches a body of content at question time and passes the retrieved passages to a language model, which answers from them rather than from what it learned in training. It exists because a model's parametric memory is fixed at training time, cannot be updated for your organisation, cannot be permission-checked, and cannot cite where an answer came from. Retrieval gives you all three: current content, access control at query time, and an answer with a source attached that the reader can open.

Often not for the first workflow, and it is the last decision rather than the first. Plenty of enterprise questions are answered better by keyword search, by a hybrid of keyword and semantic search, or by a filter over structured metadata, and the store you already own may be enough to find out. The decision that matters is the corpus and the permission model. Choose the store after you have a retrieval evaluation set that can tell you whether swapping it changed anything, otherwise you are buying infrastructure on the strength of a demo.

By carrying the source system's access control into the index and applying it to the asking user at query time, rather than indexing with a privileged crawler account and hoping. In practice that is an identity field on every indexed item, populated from the source system's access rules and filtered against the caller's group membership on every query, plus a test in the build pipeline where a user who should not see a document asks the question that would return it and the run fails if it does. Retrofitting this after indexing usually means rebuilding the index, which is why it belongs in the design rather than in hardening.

Separately from the answer. Build a set of real questions with the passages a qualified person says are needed to answer them, then measure whether those passages come back at the depth you actually pass to the model. That number is your retrieval score, and it can be improved without touching the model. Measure answer quality against the same questions with the correct passages supplied, and you have your generation score. When something regresses, the two scores tell you which half broke, which is the difference between a diagnosis and a fortnight of guessing.

Usually one of three things, and they are distinguishable if you measure retrieval separately. The right passage was never retrieved, so the model answered from general knowledge and sounded fine doing it. The right passage was retrieved and ignored, which is often a placement problem, since published work found models use information at the beginning and end of a long context far better than information in the middle. Or the corpus genuinely contains the wrong answer, because the superseded policy is still in the index and nothing marks it as superseded. Add a fourth for completeness. The system has no way to say it does not know, so it produces something rather than nothing.

No, and treating it as though it does is an expensive mistake. Published work on long contexts found performance degrades significantly when the relevant information sits in the middle of the input, including in models built for long contexts, so more context is not the same as more attention. Beyond that, a context window does not solve any of the reasons a regulated organisation needs retrieval: permissions still have to be applied per user, content still has to be current, answers still have to cite a source, and every additional token has a price and a latency cost on every single call. Retrieval is what keeps the context small and defensible.

Any number quoted before seeing your corpus is a guess, Tenhaw's included. The shape of the bill is consistent though, so you can build the estimate yourself: embedding and indexing at ingest, then re-embedding whenever the corpus or the embedding model changes; search per query; the tokens in the retrieved context on every call, usually the largest line and directly controlled by how many passages you pass; the model's output; human review of whatever is routed for it; and evaluation runs, which recur on every model upgrade. Tenhaw publishes its method in full and free to adopt, so this estimate is one you can build without hiring anyone, and it is the number that decides between designs and is almost always the one missing.

Chunk on the document's own structure rather than a fixed character count, so a section or a table survives intact instead of being cut in half, and treat the size as something you tune rather than a default you inherited. Two related settings matter as much. One is where a retrieved passage sits in the prompt, and published work on long contexts found models use information at the beginning and the end far better than anything buried in the middle. The other is how many passages you pass, which has a measured cost as well as a measured benefit. All three are cheap experiments once a retrieval evaluation set exists, and guesswork before it does.

Yes, and internal corpora are the awkward case, because the instruction does not have to come from an attacker to work. Indirect prompt injection was demonstrated against real deployed applications in 2023 by planting text in content the application would later fetch, and prompt injection is the first entry in the OWASP risk list for language model applications. In an enterprise corpus a well-meant instruction sitting in a document template does the same job by accident. So retrieved passages are handled as data rather than as instructions, the tools the agent can reach are scoped so a successful injection cannot do much, and the evaluation set carries adversarial documents alongside the ordinary ones.

Decide the lifecycle before the first ingest rather than after the first incident. Documents get amended, reclassified and deleted, people change teams and leave, and an index rebuilt on a schedule with permissions copied in at ingestion time spends the whole gap between rebuilds answering from a world that no longer exists, including for people whose access has been withdrawn. Two things carry most of the load. Evaluate permissions against the source system at query time instead of trusting what was copied in, and set re-index cadence, deletion and reclassification deliberately. Agree on day one how long a document deleted this morning can still be answered from, and whether your data protection officer accepts that number.

No. Tenhaw builds first retrieval paths over client corpora and puts a fortnight on the corpus and the permission model ahead of any purchase: which content is authoritative, who owns it, what its access rules actually are in the source system, and whether those rules can be read at query time. That work needs no supplier and it decides most of the architecture, including whether a platform would have helped. Buying first usually means paying to discover the same answers in a worse order, because the questions a platform cannot settle for you are exactly the ones that matter. Enterprise search across the whole estate is a different job from one workflow proved out.

The corpus needs a named owner, and the job is deciding which document is authoritative. Enterprise corpora hold the policy, the superseded policy, three drafts of the next one, and a slide that summarises all of them incorrectly. Retrieval returns the wrong one with complete composure, because relevance is not authority, so someone has to mark what is current and what has been withdrawn. You also need whoever controls access in each source system, whose rules have to reach query time, and your data protection officer, who has to accept how long a document deleted this morning can still be answered from. Until those people exist, tuning chunk sizes is work on the wrong problem.

Fifty real ones settle almost every architectural argument you are about to have. Collect them from the people who will actually ask, and write each row as three things: the question in their words, the answer they would accept, and the document that answer lives in. That list is the design brief and the first evaluation set at the same time, and it is cheap to label because nothing needs a model in the loop to grade it. Write it before you pick a store, an embedding model or a chunking strategy, and those stop being preferences defended in a meeting and become decisions with a number attached.

Only partly, and the literature is candid about it. A study of three production retrieval systems concluded that validating one is only feasible during operation, and that robustness evolves rather than being designed in at the start. That is not a reason to skip testing. It is a reason to ship something narrow to real users early and treat the first months of running it as part of the test. A retrieval score kept separate from the generation score still belongs in the build from week one, and so does a permission test that is meant to fail, where someone who should not see a document asks the question that would surface it.

Make every answer carry the passage it came from, as a link the reader can open. That one change alters the risk profile of the whole system more than any single accuracy improvement, because a checkable answer fails safely and an unchecked one does not. It also lets a reader see that the source was the superseded policy, which no confidence number would have told them. On a live engagement in the London insurance market, Tenhaw scored confidence from provenance, which source each extracted value was drawn from, combined with model certainty and an independent cross-check. Where a passage came from is a stronger signal than how sure the model sounds, and it is the one a reviewer can argue with.

Enough to settle the architecture. On a retrieval problem the target is one real path over your own content: a defined corpus, permissions carried from the source system to query time, and answers that cite the passage they came from. Tenhaw took a London specialty insurance business from PDFs to business intelligence on Azure in a fortnight, a proof of concept over ground it had circled for roughly a year. The artefact that outlasts the code is the evaluation set, real questions with the passages that answer them, scored separately from the generated answer. That is the thing almost nobody has, and the thing that still tells you the truth after you change the model, the embedding or the store.

Start with one, and choose the one where the questions actually are. Tenhaw builds retrieval paths one workflow at a time rather than standing up enterprise search across a whole estate. Every repository has its own access model, so each system you add is fresh work rather than a copy of the last. The rules have to be read from that source, carried onto the indexed item and matched to the person asking at query time. Doing that once, properly, on the content that matters most tells you what the second system will cost. Doing it everywhere at once means meeting the hardest permission model late, with an index already built on the wrong assumption and a rebuild in front of you.

Wherever you decide, and decide it explicitly, because embeddings are derived from your documents and inherit their sensitivity. Tenhaw works to UK data residency by default, with the EU available, and builds on your infrastructure under your policies. The genuinely provider-specific choices are whether document-level access control is native to the search service or something you build on top of it, how identity federates from your source repositories, and where the vectors physically live. It has delivered retrieval components on Azure; on AWS and Google Cloud the shape is the same and the services differ. OWASP lists vector and embedding weaknesses among its top risks for language model applications, so treat the index as in scope for classification and review.