How to measure AI coding productivity
Measure AI-assisted delivery through outcomes, cycle time and quality evidence. Read the PPL comparison with its scope and limits intact.
Measure the outcomes shipped per cycle, not lines of code or licence usage, and compare like with like on the same piece of work. At PPL, an outcome ticket shipped 54 front-end changes against 1 for a prescriptive story on the same epic, in a similar cycle time. That is one reported comparison, not a general benchmark, and the two ticket shapes covered different scopes of work.
30 minutes with James · how this would work in your organisation
The basis for this approach
This is our measurement approach, drawing on The Tenhaw Way and one reported PPL epic comparison. The PPL result is not a controlled study or a general productivity benchmark.
The demand signal
The PPL operating-model case compares an outcome ticket and a prescriptive story on the same front-end epic. Similar cycle time produced different scope: 54 changes versus 1. It is one reported comparison with different ticket scopes, not a repeatable uplift to put into a forecast.
Bring the reason this is on your agenda to a 30-minute call with James, and we can explore what it would take in your organisation.
Book a 30-minute callWhy it stalls
2 failure modes we keep meeting
The metric counts activity
Licence use and code volume show activity. The question for a buyer is what changed in the product, how long it took, what passed review and whether the result had the intended value.
The comparison ignores scope
A larger outcome ticket and a smaller story are not identical units. Keep the requirement, scope and quality evidence beside the timing so the reader can see what is actually comparable.
If one of these sounds familiar, bring it to a 30-minute call with James and we can explore a way past it.
Book a 30-minute callHow we approach it
3 moves, in order
- 01
Agree the outcome and its value
Write the work as an outcome with an acceptance decision. Record a baseline and give the value measure an owner before interpreting delivery activity.
- 02
Keep the comparison inspectable
Report outcomes, scope and cycle time together. Include the review and testing records used to accept the work, and retain failures or rework that change the interpretation.
- 03
Forecast from your own delivery data
Use the programme’s actual throughput and confidence intervals. A published example can suggest a method to try; it cannot substitute for the data from your own workflow.
Record a baseline and give the value measure an owner before interpreting delivery activity.
Prefer to talk it through? Ask us on a discovery call →
Sources
Every source below was opened and read before it was attached. Where nothing survived that check, the claim on this page was softened rather than given a plausible-looking link.
PPL, the six-month Embedded Agentic Lead engagementThe 54 vs 1 front-end changes are one reported epic comparison, not a general productivity benchmark.Tenhaw, LearnThe Tenhaw Way, measure valueThe Tenhaw Way, forecast with confidence intervalsThe closest guides to this one
Nearest first, then the rest. Each one carries the same label: written from delivery, or the method we would bring. All 34 are on the hub.
What does an AI-native software delivery lifecycle look like?
Connect requirements, AI engineering and human release decisions.
Our approachSetting up an AI centre of excellence
Build an AI centre of excellence around a practical method, accountable workflow owners and learning your teams can reuse. See the Learn cohort scope.
Our approachCopilot adoption in the enterprise: getting from licences to use
Build Copilot into a routine your team already owns. Read the scoped Microsoft EMEA example from James Rooney’s contractor role and the Learn options.
Our approachRetrieval, RAG and permission-aware knowledge access
Answer from useful sources while keeping access tied to the person asking.
Our approachMCP, tool calling and integrating agents with your systems
Connect the tools with clear permissions, approvals and operating evidence.
Our approachDesign guardrails for uncertain AI results
Decide what continues, what needs review and which actions are unavailable.
Or bring the problem to a call instead of reading three more of these.
Book a 30-minute callWork out the next step for your team.
A 30-minute call with James Rooney to discuss your workflow, people and budget. Leave with a rough scope and a useful next step.
Learn: engineering programme · £24,000 fixed · 6 weeks · excluding VAT
No time that suits? Email hello@tenhaw.com. Calendar not loading? Open it on cal.com.
Describe the work
Describe the work in a paragraph. James replies with a rough scope and budget range within two working days.
Questions this guide answers
What should we measure?
Measure outcomes accepted, the scope completed, cycle time and the quality evidence used for release. Connect the result to the value agreed for the workflow and retain the baseline so changes can be checked.
Why not lines of code?
Code volume does not establish that the requirement works or that the buyer received value. Review the completed outcome and its tests; extra generated code can still require correction and review.
How do we compare before and after?
Keep the work and measurement definition visible: outcome, scope, time window, acceptance criteria and review evidence. Report meaningful differences instead of treating unlike tickets as equal units.
How do we forecast with AI in the loop?
Use your own throughput and forecast completion as confidence intervals, then update the forecast as the delivery data changes. The Tenhaw Way publishes the method in its forecasting how-to.
What does good look like?
The team can show what useful outcome changed, how it was accepted and what it cost in time and review. PPL’s 54 vs 1 front-end changes are one reported epic comparison with different scope, not a target or general benchmark for another team.