Pattern guide

Voice agents and conversation intelligence

Turning conversations into structured intelligence, and holding conversations that take real actions.

ways these programmes stall
4
steps in how we would address it
7
questions answered in full
4
In one paragraph

Voice work splits into two quite different problems. Conversation intelligence takes recorded voice (calls, meetings) and turns it into structured, searchable intelligence: summaries, themes, sentiment, entities and compliance signals. Real-time voice agents hold a live conversation, take an action and hand off to a human. The first is a data and privacy problem; the second is a latency, interruption and handoff problem. Tenhaw's founder led an AI Voice Insights proof of concept at HSBC projected to remove 1.5M+ hours of manual administration annually, and Velocity84 has shipped real-time agentic voice products.

James Rooney, Founder

Updated

Delivered

We have done this, and here is exactly how far that goes

Both halves rest on real delivery, and each has a limit. The HSBC AI Voice Insights work was a proof of concept, led by our founder; its 1.5M+ hour figure was projected rather than realised, and it was not taken to production rollout. The real-time agentic voice experience comes from products shipped through Velocity84, a separate venture of our founder's: startup-scale products without enterprise regulatory constraints. We have not delivered a production real-time voice agent inside a regulated enterprise.

Why this matters now

Voice is the channel with the most data already collected and the least structure imposed on it. Most contact centres hold years of recordings captured for quality or regulatory purposes, and can still only answer what customers were calling about anecdotally. That makes conversation intelligence over the recorded estate the lower-risk entry point, because the data already exists and nothing is customer-facing. It also makes the first question a legal one rather than a technical one: recordings collected for one specified purpose cannot simply be reused for a purpose incompatible with it, which is the purpose limitation principle in Article 5(1)(b) of the UK GDPR and the reason these programmes stall in month four rather than month one.

Ask a technical questionanswers from all 12 guides
Ask anything technical about voice agents and conversation intelligence and I will answer from this guide, and tell you first whether this is work we have delivered or an approach we would be taking.

Prefer to talk it through? Ask us on a discovery call →

Diagnosis

Where these programmes actually stall

Not the risks a vendor lists. The ones that stop the work.

01

The recordings exist and the consent position does not

Most organisations hold years of call recordings captured for quality or compliance purposes. Repurposing them for AI analysis is a different processing purpose, and programmes stall when that is discovered late. It is answerable, but it is a question for month one rather than month four, and it is the commonest reason a voice proof of concept sits finished, demonstrated and unshipped while a data-protection review it should have started with catches up.

02

Latency budgets are discovered in production

Real-time voice is unforgiving. Transcription, reasoning, tool calls and synthesis all have to fit inside a window where a human would have started speaking. Architectures that test fine asynchronously fall apart the moment someone interrupts, and interruption handling is rarely designed up front.

03

The handoff is treated as an edge case

It is the main event. What the human receives when the agent gives up (the context, the transcript, the reason) determines whether customers experience the agent as helpful or as an obstacle. Most implementations design the happy path thoroughly and the handoff barely at all.

04

Sentiment is trusted more than it deserves

Sentiment scoring on real calls is noisy, culturally variable, and frequently wrong on exactly the calls that matter. Using it as a headline metric rather than as one weak signal among several is a reliable way to lose the confidence of the operational team that has to act on it.

Prescription

How we would address it

Grounded in The Tenhaw Way and the engagements written up in our case studies.

  1. 01

    Start with the recorded estate, not the live channel

    Conversation intelligence over existing recordings is lower risk, uses data you already hold, and produces business value without touching a customer interaction. It also tells you how well the models handle your accents, jargon and line quality before anything is live. At HSBC this meant integrating with the existing contact-centre system and processing inbound handler calls rather than building anything customer-facing.

  2. 02

    Transcribe into a retrievable store, then classify for themes

    The architecture we built transcribed calls and landed them in a vector database, then assessed each call for common themes, account issues and similar recurring categories. That ordering matters: a searchable store first means later analytical questions can be asked of the same corpus without reprocessing, rather than each new question requiring a new pipeline.

  3. 03

    Settle the data-protection position in month one

    Processing purpose, consent basis, retention and residency for voice data are established before the build, with your DPO involved as a co-author. On any regulated engagement this is a design constraint, not a compliance review at the end.

  4. 04

    Design the latency budget as an architecture requirement

    Each stage (transcription, reasoning, tool calls, synthesis) gets an explicit budget, and the design is tested against interruption from the start. This is the single most transferable lesson from shipping consumer voice products, where users are far less forgiving than internal testers.

  5. 05

    Make the handoff a first-class deliverable

    What the human receives, how the agent decides to escalate, and what the customer experiences at the boundary are specified and tested as carefully as the automation itself. The routing logic here is the same confidence-and-consequence pattern we use for document review.

  6. 06

    Aim at two outputs: the operational one and the analytical one

    The HSBC work produced both, and they serve different people. Operationally, automated agent notes remove manual write-up from the handler's day. Analytically, thematic classification lets the business see what customers were actually calling about, which is a question most contact centres can only answer anecdotally. Programmes that pursue only the analytical output tend to struggle for operational buy-in, because nothing changes for the people whose calls are being analysed.

  7. 07

    Land it somewhere governed, with lineage

    Summaries and themes are only useful if they land somewhere governed, with lineage back to the source conversation. Otherwise you have replaced one unsearchable estate with another, which we would consider a failed engagement.

Operationally, automated agent notes remove manual write-up from the handler's day.
Provider notes

Where the cloud matters for voice

Speech recognition quality on your specific accents, jargon and line conditions varies meaningfully between providers and is worth testing rather than assuming, it is the one place we would insist on a bake-off against your own recordings. Real-time orchestration primitives and telephony integration also differ substantially. Data residency for voice is frequently the binding constraint in regulated environments and should be established before any provider selection.

Where this usually starts

Agentic Proof of Concept

Pick the workflow. Two to four weeks later, look at a working thing. Fixed price · £20k–£55k · 2–4 weeks.

What that engagement covers

Voice agents and conversation intelligence: your questions

What is conversation intelligence?

Turning recorded voice (calls, meetings) into structured, searchable intelligence: summaries, recurring themes, entities, and compliance or risk signals. It is distinct from real-time voice agents, and it is usually the lower-risk starting point because the data already exists, nothing is customer-facing, and it tests how well models handle your actual accents, jargon and line quality before anything goes live.

Can we use our existing call recordings to train or run AI analysis?

Often yes, but not automatically. Recordings captured for quality monitoring or regulatory purposes were collected under a specific processing purpose, and analysing them with AI is generally a different one. The consent basis, retention position and residency need establishing before the build rather than during it, because it is answerable, and programmes that leave it to month four lose months.

What makes real-time voice agents hard?

Latency and interruption. Transcription, reasoning, tool calls and speech synthesis all have to complete inside the window where a human would have started speaking, and users interrupt constantly. Architectures that work asynchronously fail immediately under those conditions. The handoff to a human is the other hard part, and it is usually designed as an edge case when it is the main event.

Has Tenhaw delivered voice AI?

Partly. At HSBC our founder led an AI Voice Insights proof of concept that integrated with the contact-centre system, transcribed inbound handler calls into a vector database, classified each call for recurring themes such as account issues, and drove two outputs: automated agent notes and business intelligence on what customers were actually calling about. The 1.5M+ hours of manual administration it was projected to remove annually is a projection, never realised, and the work was a proof of concept rather than a production rollout. Real-time agentic voice experience comes from products shipped through Velocity84, a separate venture, at startup rather than enterprise scale. We have not delivered a production real-time voice agent inside a regulated enterprise.

Sources

Where the checkable claims came from

Every source below was opened and read before it was attached. Where nothing survived that check, the claim on this page was softened rather than given a plausible-looking link.

  1. 01
    UK GDPR, Article 5(1)(b): purpose limitation

    Personal data must be collected for specified, explicit and legitimate purposes and not further processed in a manner incompatible with those purposes. This is the provision behind the month-one question above. Whether your recordings clear it is a matter for your DPO and, where needed, counsel, not for us.

Keep reading

The closest guides to this one

Nearest first, then the rest. Each one carries the same label: written from delivery, or the method we would bring. All 12 are on the hub.

Delivered

Document intelligence to business intelligence

Getting information out of PDFs and into something the business can decide with.

Delivered

AI-native SDLC and product delivery lifecycle

Changing how software gets specified, built and shipped once AI is in the room.

Delivered

End-to-end agentic workflow implementation

Taking one whole business process agentic, rather than assisting the humans doing it.

Our approach

Agent evaluation and assurance

Why AI pilots never reach production, and what it takes to get one through the gate.

Our approach

AI governance and regulatory evidence

Building the evidence as a by-product of the work, rather than assembling it under deadline.

Our approach

Agent identity and access

Agents are not users, and giving them a service account is how this goes wrong.

All 12 pattern guides, grouped

Talk to us about voice agents and conversation intelligence.

A 30-minute call with James Rooney. We'll tell you honestly which parts of this we have done before and which we would be doing for the first time, and you'll leave with a rough scope either way.

30 minutesWith James personallyNo obligation
See the proof first

Most organisations start with a fixed-price Agent-Readiness Audit · £30k–£90k · 6–8 weeks