Pattern guide

Document intelligence to business intelligence

Getting information out of PDFs and into something the business can decide with.

ways these programmes stall
4
steps in how we would address it
7
questions answered in full
5
In one paragraph

Document intelligence turns unstructured source material (PDFs, scans, email, forms) into structured data that lands in a warehouse and drives business intelligence. It is where most enterprises meet agentic AI first, and the pattern most likely to stall between a convincing extraction demo and a number a business will actually act on. Tenhaw has delivered this end to end on Azure: a working proof of concept in two weeks, on a live engagement in the London insurance market, covering ground that had previously taken roughly twelve months.

James Rooney, Founder

Updated

Delivered

We have done this, and here is exactly how far that goes

We have delivered this pattern end to end, on Azure, on a live engagement. What follows is written from that work rather than from vendor documentation. We have not delivered the equivalent on AWS or Google Cloud; the provider notes below set out where they genuinely differ.

Why this matters now

Document work is usually where an organisation meets agentic AI first, because the source material already exists, nothing is customer-facing, and the value is easy to describe to a board. It is also the pattern where the published engineering literature is bluntest about the gap between a demo and a system: a study of three production retrieval implementations concluded that validating one is only feasible during operation, and that the robustness of such a system evolves rather than being designed in at the start. That is the four failures below, arrived at from the other direction.

Ask a technical questionanswers from all 12 guides
Ask anything technical about document intelligence to business intelligence and I will answer from this guide, and tell you first whether this is work we have delivered or an approach we would be taking.

Prefer to talk it through? Ask us on a discovery call →

Diagnosis

Where these programmes actually stall

Not the risks a vendor lists. The ones that stop the work.

01

The demo extracted beautifully, and it became a proof of concept that never shipped

Extraction accuracy on a curated sample tells you almost nothing about accuracy on the long tail: the scanned fax, the amended schedule, the document where the important number is in a footnote. This is the commonest shape of a proof of concept that never shipped. The programme discovers that 95% accuracy is unusable for a process that requires a defensible number, nobody designed the exception path, and the work stops in the gap between impressive and usable.

02

Nobody agreed what a correct answer is, so accuracy became an argument

Two experienced underwriters will disagree about what a document says. If you have not established ground truth with the people who own the decision, you cannot measure the system, and every accuracy conversation becomes an argument about the benchmark rather than the model. Pilots stuck in that argument do not fail a test, they simply never get one, which is why they can sit unresolved for quarters.

03

The extraction works, the BI layer is not ready, and the programme stalls there

This is the most common failure and the least discussed. You now have structured data with no semantic layer, no agreed definitions, and no governance, so the business gets confident answers drawn from the wrong table. The bottleneck moves from the model to the data foundations, and the programme is not resourced for it, so a technically successful pilot sits waiting on a data programme nobody has funded.

04

Expert review became the bottleneck, because it was built as a queue

Every serious implementation keeps humans in the loop. Most implement it as a review queue that is worked in order, which means scarce expert time is spent uniformly across easy and hard cases. Routing by confidence and consequence is what makes the economics work, and it is usually retrofitted after the queue becomes the bottleneck. Until it is, the business case shows a saving that the operation cannot feel.

Prescription

How we would address it

Grounded in The Tenhaw Way and the engagements written up in our case studies.

  1. 01

    Extract to markdown first, then narrow to the fields that matter

    The pipeline we ran starts by using a language model to turn each source document into a markdown representation of itself, and only then narrows to the key fields. Extracting to an intermediate readable form first means the extraction is inspectable by a human and re-runnable when the field list changes, rather than being a black box from PDF to database column.

  2. 02

    Normalise, then enrich, then add semantics

    Extracted fields are normalised into consistent formats, enriched against third-party APIs, and then given semantic enhancement (context and thematic grouping) so downstream logic is working with meaning rather than strings. Each stage is separable, which matters because the enrichment and semantic stages are where accuracy problems are usually diagnosable.

  3. 03

    Requirements as a machine-readable corpus

    The document types, the fields, the validation rules and the edge cases go into structured markdown before any build, and we run the gap-and-contradiction pass over them. On the insurance engagement this surfaced ambiguities the business had not realised were ambiguous, and resolving them took a conversation rather than a rebuild.

  4. 04

    Build against the whole requirement set, paired with your engineers

    The build follows our published AI-engineering-first method: a blank repository, high-level prompts against the full corpus, a security review roughly every fifth prompt, and pair-programming throughout. On the insurance engagement the client's own engineer finished a two-week build 70% confident they could run the process unaided.

  5. 05

    Score confidence by source, not just by model certainty

    On the same engagement we scored confidence using the provenance of the data, which third-party enrichment source it came from, alongside model certainty and a search-based cross-check. A model's own confidence is a weak signal on its own; combining it with where the data came from is what makes routing decisions defensible to the people who own the outcome.

  6. 06

    Land it in business logic and a dashboard, not a database

    The final stages apply business logic to interpret the enriched, semantically grouped data and surface it in an internal dashboard. This is the step that distinguishes document intelligence from document extraction, and it is the one most often descoped when a programme runs late, which is how organisations end up with a populated table nobody uses.

  7. 07

    Treat the semantic layer as in scope, not someone else's problem

    Structured output is worthless if the BI layer answers confidently from the wrong table. Agreed definitions, a governed semantic layer and lineage back to the source document are part of the work, because without them the programme delivers extraction and not intelligence.

Structured output is worthless if the BI layer answers confidently from the wrong table.
Provider notes

Where the cloud actually matters, and where it does not

The hard parts of this pattern (ground truth, exception design, human routing, semantic layer, lineage) are provider-independent, and they are where programmes fail. The provider-specific decisions are the document-understanding service and its handling of layout and tables, how identity and permissions propagate from the source repository through to the warehouse, and what the residency position is for the documents themselves. We have delivered this on Azure. On AWS and Google Cloud the architecture shape is the same and the service choices differ; an engagement there applies the method rather than repeats a delivery, and we say so at the point of engagement.

Where this usually starts

Agentic Proof of Concept

Pick the workflow. Two to four weeks later, look at a working thing. Fixed price · £20k–£55k · 2–4 weeks.

What that engagement covers

Document intelligence to business intelligence: your questions

How long does a document intelligence proof of concept take?

Two to four weeks for a working proof of concept against one document type and one real workflow. On a live engagement in the London insurance market, Tenhaw delivered a working proof of concept extracting information from PDFs into business intelligence on Azure in two weeks, ground the business had been circling for roughly a year. Productionising it is a separate phase, scoped at four to six weeks with a dedicated team.

Why do document extraction projects stall after the demo?

Usually for one of four reasons: accuracy on a curated sample does not survive the long tail and no exception path was designed; nobody established what a correct answer is, so every accuracy discussion becomes an argument about the benchmark; the extraction works but the BI layer has no semantic layer or governance, so the business gets confident answers from the wrong table; or human review was built as a queue rather than routed by confidence and consequence, so expert time becomes the bottleneck.

Our document AI proof of concept worked and never shipped. What now?

Start by working out which of the three gaps you are actually in, because they need different money and different people. If accuracy collapsed on the long tail, the missing piece is exception routing rather than a better model. If nobody can agree what a correct answer is, the next piece of work is a ground-truth set built with the people who own the decision, and it is a fortnight rather than a phase. If the extraction is fine and the numbers are not trusted, you are waiting on a semantic layer and lineage, which is a data programme with a different sponsor and a different budget line. A proof of concept that never shipped is rarely blocked on the model, and the diagnosis is cheap to do before anyone commits to a rebuild.

What accuracy is good enough for document intelligence?

There is no universal number, and quoting one is a warning sign. The right question is what the exception path costs. A process that tolerates review can run at accuracy that would be unacceptable for straight-through processing. Design the routing first, by extraction confidence and business consequence, and the accuracy target falls out of it rather than being asserted up front.

Do we need to fix our data platform before doing this?

Not before a proof of concept, and yes before production. A proof of concept establishes whether the extraction is viable at all, which is the cheaper question to answer first. But structured output with no semantic layer, agreed definitions or lineage produces confident answers from the wrong table, so data foundations belong in the productionisation scope rather than being discovered during it.

Sources

Where the checkable claims came from

Every source below was opened and read before it was attached. Where nothing survived that check, the claim on this page was softened rather than given a plausible-looking link.

  1. 01
    Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System (2024)

    Three case studies across research, education and biomedical domains. The finding quoted above, that validation is only feasible during operation and robustness evolves rather than being designed in, is from the abstract.

  2. 02
    Liu et al., Lost in the Middle: How Language Models Use Long Contexts (2023)

    Why the position of an extracted passage in the prompt changes the answer, and why a long context window is not a substitute for narrowing to the fields that matter.

Keep reading

The closest guides to this one

Nearest first, then the rest. Each one carries the same label: written from delivery, or the method we would bring. All 12 are on the hub.

Delivered

AI-native SDLC and product delivery lifecycle

Changing how software gets specified, built and shipped once AI is in the room.

Delivered

Voice agents and conversation intelligence

Turning conversations into structured intelligence, and holding conversations that take real actions.

Delivered

End-to-end agentic workflow implementation

Taking one whole business process agentic, rather than assisting the humans doing it.

Our approach

Agent evaluation and assurance

Why AI pilots never reach production, and what it takes to get one through the gate.

Our approach

AI governance and regulatory evidence

Building the evidence as a by-product of the work, rather than assembling it under deadline.

Our approach

Agent identity and access

Agents are not users, and giving them a service account is how this goes wrong.

All 12 pattern guides, grouped

Talk to us about document intelligence to business intelligence.

A 30-minute call with James Rooney. We'll tell you honestly which parts of this we have done before and which we would be doing for the first time, and you'll leave with a rough scope either way.

30 minutesWith James personallyNo obligation
See the proof first

Most organisations start with a fixed-price Agent-Readiness Audit · £30k–£90k · 6–8 weeks