Pattern guide

Retrieval, RAG and permission-aware knowledge access

Answering from your own knowledge, without answering from documents the person asking is not allowed to see.

ways these programmes stall
5
steps in how we would address it
7
questions answered in full
7
In one paragraph

Retrieval-augmented generation, usually shortened to RAG, is the pattern where a system searches your own content at question time and hands the retrieved passages to a model to answer from, rather than relying on what the model absorbed in training. It is the default way to make an agent answer from enterprise knowledge. In a regulated organisation the hard part is not the search: it is that retrieval has to respect the permissions of the person asking rather than the privileges of whatever account built the index, and that most enterprise RAG failures are retrieval failures that get blamed on the model. The right document was never fetched, or the wrong one was, and nothing in the system could tell you which.

James Rooney, Founder

Updated

Our approach

We have not delivered this at enterprise scale, and here is exactly what we are drawing on

We have built retrieval into delivered work and we have not built an enterprise knowledge platform. On a live engagement in the London insurance market we built the semantic layer over extracted document fields, with provenance recorded per field and confidence scored from where each value came from. On the HSBC voice proof of concept our founder led, call transcripts were landed in a vector database and queried for recurring themes. Neither of those is a permission-aware retrieval estate serving thousands of users across systems that each have their own access model. We have not built one of those, the flagship agentic build is a regulated-estate proof of concept now being productionised, and the parts of this guide about permission propagation, index lifecycle and adversarial content are method rather than a write-up of a delivery.

So what should you do about it

Do not buy a knowledge platform as the first move. The cheapest useful thing is a fortnight spent on the corpus and the permission model: which content is authoritative, who owns it, what its access rules actually are in the source system, and whether those rules can be read at query time. That answer decides most of the architecture and it does not need us. Where we are useful is building the first real retrieval path against it and leaving behind the evaluation set that says whether it works, because that is the artefact almost nobody has and the one that survives every subsequent change of model. If what you need is an enterprise search programme stood up rather than one workflow proved out, say so on the call and we will tell you whether we are the right firm for it.

Why this matters now

Retrieval is the oldest idea in this stack and the least well measured. The paper that named the pattern, published in 2020, argued that models storing factual knowledge in their parameters have a limited ability to access and precisely manipulate that knowledge, and that an explicit non-parametric memory addresses it. The engineering literature since is less romantic: a study of three production retrieval systems concluded that validating one is only feasible during operation and that robustness evolves rather than being designed in at the start. Both are arguments for building the measurement before the corpus.

Ask a technical questionanswers from all 12 guides
Ask anything technical about retrieval, rag and permission-aware knowledge access and I will answer from this guide, and tell you first whether this is work we have delivered or an approach we would be taking.

Prefer to talk it through? Ask us on a discovery call →

Diagnosis

Where these programmes actually stall

Not the risks a vendor lists. The ones that stop the work.

01

The index was built with the ingestion account's privileges, so the assistant will answer anyone from anything

This is the most common serious failure in enterprise retrieval and it is nearly always found after the corpus has been indexed. A crawler runs as a service principal with broad read access, everything it can see goes into one index, and the assistant then answers any user from any document in it. The fix is not a filter added at the end. It is carrying the source system's access control into the index and applying it to the asking user at query time, which usually means rebuilding the index. Design it in, or budget for doing it twice.

02

Nobody decided which document is the authoritative one

Enterprise corpora contain the policy, the superseded policy, three drafts of the next policy, and a slide that summarises all of them incorrectly. Retrieval will return the wrong one with complete composure, because relevance is not authority. Until someone owns the corpus and marks what is current, accuracy work is being done on the wrong problem, and every hour spent tuning chunk sizes is an hour not spent on the thing that is actually wrong.

03

Chunking and placement decide the answer, and neither was measured

How a document is split, and where the retrieved passage sits in the prompt, change the answer materially. Published work on long inputs found that performance is highest when the relevant information appears at the beginning or the end of the context and degrades significantly when a model has to use something in the middle, including in models built for long contexts. A team treating chunking as a default setting is leaving a large and cheap accuracy gain unclaimed, and cannot explain its own failures when asked.

04

There is one score, at the end, so nobody can tell a retrieval failure from a generation failure

If the only measurement is whether the final answer was good, every regression is a mystery. Did the search miss the document, did it return the document and the model ignore it, or did the model have everything it needed and answer badly. Those three have different fixes and different owners, and one end-to-end number cannot separate them. This is the cheapest thing most teams are not doing, and the reason their accuracy conversations go in circles.

05

The index is a snapshot and the business is not

Documents get amended, reclassified and deleted, and people join teams and leave the organisation. Where the index is rebuilt on a schedule and permissions are copied at ingestion time, then for the whole gap between rebuilds the system is answering from a world that no longer exists, including for people whose access has been withdrawn. Nobody notices until an audit or an incident, and both are expensive ways to learn it.

Prescription

How we would address it

Grounded in The Tenhaw Way and the engagements written up in our case studies.

  1. 01

    Write the questions down before choosing a store

    The first artefact is a list of real questions from the people who will ask them, with the answers they would accept and the documents those answers live in. Fifty of those settle almost every architectural argument, and the same list becomes the first evaluation set. Choosing a vector database before it exists is answering a question nobody has written down.

  2. 02

    Propagate source permissions to query time, and prove it with a test that is meant to fail

    Retrieval respects the permissions of the asking user against the source system, not the privileges of whatever indexed the corpus. In practice that means carrying an access identifier onto every indexed item and filtering on the caller's group membership at search time, which is the security filter pattern the cloud search services document in their own guidance. The control is not the filter, it is the test: a named user who should not see a document asks the question that would surface it, and the pipeline fails if it comes back. That test belongs in the build from week one, not in a penetration test at the end.

    Why the agent needs its own identity too
  3. 03

    Measure retrieval separately from generation

    Two evaluation sets, scored separately. Retrieval is measured on whether the right passages came back at all, at the depth you actually pass to the model. Generation is measured on whether the answer is right given those passages. Keeping them apart turns a regression from an argument into a diagnosis, and we would build the retrieval set first: it is cheaper to label, it needs no model in the loop to score, and it survives you changing the model.

    What an evaluation harness consists of
  4. 04

    Treat chunking, ordering and context budget as tuned parameters

    Chunk boundaries follow the structure of the document rather than a character count, retrieved passages are ordered with the strongest at the edges of the context rather than buried in the middle of it, and the number of passages passed is a decision with a measured cost and a measured benefit rather than a default. All three are cheap experiments once the retrieval evaluation set exists, and guesswork before it does.

  5. 05

    Make every answer carry its source, and score confidence from provenance

    An answer with a link back to the passage it came from is checkable by the person reading it, which changes the risk profile of the whole system more than any single accuracy improvement. On our insurance engagement, confidence came from the provenance of the data, which source each value was drawn from, combined with model certainty and an independent cross-check, rather than from the model's own score. The same principle holds here: where a passage came from is a stronger signal than how sure the model sounds.

  6. 06

    Design the index lifecycle before the first ingest

    Re-index cadence, deletion at source, reclassification, and what happens to a leaver's access are design decisions rather than operational ones. The question to answer on day one is how long a document deleted this morning can still be answered from, and whether that number is acceptable to your data protection officer. If nobody can state the number, it is not a system yet.

  7. 07

    Treat retrieved content as untrusted input

    Anything the system retrieves can carry instructions aimed at the model rather than at the reader. Indirect prompt injection was demonstrated against real deployed applications in 2023 by planting text in content the application would later fetch, and prompt injection is the first entry in the OWASP risk list for language model applications. In an enterprise corpus the injected text does not have to come from an attacker: a well-meant instruction in a document template will do it. So retrieved passages are handled as data, the tools the agent can reach are scoped so a successful injection cannot do much, and the evaluation set carries adversarial documents alongside the ordinary ones.

    Scoping what an agent can actually do
Generation is measured on whether the answer is right given those passages.
Provider notes

Where the cloud matters for retrieval, and where it does not

The hard parts here are provider-independent: who owns the corpus, which document is authoritative, what the permission model is, and whether an evaluation set exists at all. The genuinely provider-specific decisions are whether document-level access control is native to the search service or something you build on top of it, how identity federates from the source repositories, and where the index and the embeddings physically sit. We have delivered retrieval components on Azure. On AWS and Google Cloud the shape is the same and the services differ; there we would be applying the method rather than repeating a delivery.

Where this usually starts

Agentic Proof of Concept

Pick the workflow. Two to four weeks later, look at a working thing. Fixed price · £20k–£55k · 2–4 weeks.

What that engagement covers

Retrieval, RAG and permission-aware knowledge access: your questions

What is retrieval-augmented generation?

Retrieval-augmented generation, or RAG, is a design in which the system searches a body of content at question time and passes the retrieved passages to a language model, which answers from them rather than from what it learned in training. It exists because a model's parametric memory is fixed at training time, cannot be updated for your organisation, cannot be permission-checked, and cannot cite where an answer came from. Retrieval gives you all three: current content, access control at query time, and an answer with a source attached that the reader can open.

Do we need a vector database?

Often not for the first workflow, and it is the last decision rather than the first. Plenty of enterprise questions are answered better by keyword search, by a hybrid of keyword and semantic search, or by a filter over structured metadata, and the store you already own may be enough to find out. The decision that matters is the corpus and the permission model. Choose the store after you have a retrieval evaluation set that can tell you whether swapping it changed anything, otherwise you are buying infrastructure on the strength of a demo.

How do we stop RAG answering from documents a user is not allowed to see?

By carrying the source system's access control into the index and applying it to the asking user at query time, rather than indexing with a privileged crawler account and hoping. Concretely: an identity field on every indexed item, populated from the source system's access rules, filtered against the caller's group membership on every query, plus a test in the build pipeline where a user who should not see a document asks the question that would return it and the run fails if it does. Retrofitting this after indexing usually means rebuilding the index, which is why it belongs in the design rather than in hardening.

How do you measure whether retrieval is working?

Separately from the answer. Build a set of real questions with the passages a qualified person says are needed to answer them, then measure whether those passages come back at the depth you actually pass to the model. That number is your retrieval score, and it can be improved without touching the model. Measure answer quality against the same questions with the correct passages supplied, and you have your generation score. When something regresses, the two scores tell you which half broke, which is the difference between a diagnosis and a fortnight of guessing.

Why does our RAG assistant give confident wrong answers?

Usually one of three things, and they are distinguishable if you measure retrieval separately. The right passage was never retrieved, so the model answered from general knowledge and sounded fine doing it. The right passage was retrieved and ignored, which is often a placement problem: published work found models use information at the beginning and end of a long context far better than information in the middle. Or the corpus genuinely contains the wrong answer, because the superseded policy is still in the index and nothing marks it as superseded. Add a fourth for completeness: the system has no way to say it does not know, so it produces something rather than nothing.

Does a bigger context window remove the need for retrieval?

No, and treating it as though it does is an expensive mistake. Published work on long contexts found performance degrades significantly when the relevant information sits in the middle of the input, including in models built for long contexts, so more context is not the same as more attention. Beyond that, a context window does not solve any of the reasons a regulated organisation needs retrieval: permissions still have to be applied per user, content still has to be current, answers still have to cite a source, and every additional token has a price and a latency cost on every single call. Retrieval is what keeps the context small and defensible.

What does a retrieval system cost to run?

Any number quoted before seeing your corpus is a guess, ours included: it would describe our workloads rather than yours. The shape of the bill is consistent though, so you can build the estimate yourself: embedding and indexing at ingest, then re-embedding every time the corpus changes or the embedding model does; the search itself per query; the tokens in the retrieved context on every call, which is usually the largest line and is directly controlled by how many passages you pass; the model's own output; human review of whatever is routed for it; and the evaluation runs, which recur on every model upgrade. Our audits produce an estimated run cost per candidate workflow before anything is committed, precisely because this is the number that decides between designs and is almost always the one missing.

Sources

Where the checkable claims came from

Every source below was opened and read before it was attached. Where nothing survived that check, the claim on this page was softened rather than given a plausible-looking link.

  1. 01
    Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)

    The paper that named the pattern. Source of the argument that models storing knowledge in their parameters have a limited ability to access and precisely manipulate it.

  2. 02
    Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System (2024)

    Three production case studies. Source of the finding that validating a retrieval system is only feasible during operation and that robustness evolves rather than being designed in.

  3. 03
    Liu et al., Lost in the Middle: How Language Models Use Long Contexts (2023)

    Performance is highest when the relevant information is at the beginning or end of the context and degrades significantly in the middle, even for models built for long contexts.

  4. 04
    Microsoft, security filter pattern for Azure AI Search

    One provider's documented pattern for document-level access control: an identity field on each indexed document, filtered against the caller's group membership at query time. Named as a worked example, not as a recommendation of that provider.

  5. 05
    Greshake et al., Not what you've signed up for: compromising real-world LLM-integrated applications with indirect prompt injection (2023)

    Demonstrates injecting instructions into data an application is likely to retrieve, against real deployed systems. The reason retrieved content is treated as untrusted input above.

  6. 06
    OWASP Top 10 for Large Language Model Applications (2025)

    LLM01 Prompt Injection and LLM08 Vector and Embedding Weaknesses are the relevant entries.

Keep reading

The closest guides to this one

Nearest first, then the rest. Each one carries the same label: written from delivery, or the method we would bring. All 12 are on the hub.

Our approach

MCP, tool calling and integrating agents with your systems

The moment an agent can call your systems, integration stops being plumbing and becomes an access decision.

Our approach

Guardrails, hallucination and accuracy control

You will not stop a language model being wrong. You can decide in advance what happens when it is.

Our approach

RAG, fine-tuning or prompting: how to choose

Sorting the requirement into knowledge, behaviour and cost, so the decision survives a finance review.

Delivered

Document intelligence to business intelligence

Getting information out of PDFs and into something the business can decide with.

Our approach

Agent evaluation and assurance

Why AI pilots never reach production, and what it takes to get one through the gate.

Our approach

AI governance and regulatory evidence

Building the evidence as a by-product of the work, rather than assembling it under deadline.

All 12 pattern guides, grouped

Talk to us about retrieval, rag and permission-aware knowledge access.

A 30-minute call with James Rooney. We'll tell you honestly which parts of this we have done before and which we would be doing for the first time, and you'll leave with a rough scope either way.

30 minutesWith James personallyNo obligation
See the proof first

Most organisations start with a fixed-price Agent-Readiness Audit · £30k–£90k · 6–8 weeks