How to forecast with confidence intervals
Replace the single invented date with a probability the business can plan against, derived from your own throughput.
- steps
- 8
- named failure modes
- 5
- definition-of-done criteria
- 6
How to forecast with confidence intervals, in one paragraph
Forecasting with confidence intervals replaces the invented date with two numbers taken from your own delivery history: a p50 the team plans against and a p85 the business commits to. You get there by counting completed items per week, inflating the remaining scope by your historical split rate, and simulating ten thousand possible quarters against that history in a spreadsheet. Good looks like a forecast refreshed weekly, published as a range with the unit and window written beside it, and converted into the currency at risk the moment the range misses the quarter end.
What these are. The delivery operating model our engagements install alongside client teams: the method underneath the agentic work rather than the agentic work itself, published in full and free to use. It is written for the person running a quarter, not for a buyer, so if you are evaluating us, read the five priced engagements or the case studies instead.
Run it the week a quarterly roadmap opens, then again every week until the quarter closes. Reach for it the moment anyone asks whether an epic will land, or asks for a date before the work has been counted.
- Time to run it once
- About two hours, then half an hour a week to refresh it
- What you need
- A spreadsheet
- Twelve weeks of finished-per-week throughput history
Step by step
Each step is deep-linkable, so you can send a colleague the one that is in dispute.
- 1
Pick one countable unit and right-size it
Pick the thing you will count finished per week and do not change it mid-quarter.
AI-augmented teams count stories closed. AI-native teams count key user journeys demonstrated, because outcome tickets are too few and too large to sample. Count items, never story points: points get re-baselined every time a team re-points, so rising velocity can mean nothing changed except the scale. Then check the items are interchangeable enough to sample. If single items routinely take more than two weeks, split them at refinement first, because the arithmetic assumes one item is much like another. Write the unit at the top of the forecast.
- 2
Pull twelve weeks of throughput history
Export the finished-per-week count for the last twelve weeks.
Eight is the minimum that gives a usable spread. Use the date an item met done, not the date it merged or demoed. Include everything that consumed capacity: stories, bugs linked to the Bug Budget epic, tech debt linked to the Tech Debt epic. Keep the zero weeks and the holiday weeks. They are the tail, and deleting them is the most common way a forecast turns optimistic. If the system changed materially, a team split, a two-week freeze, a switch of delivery mode, shorten the window and accept a wider interval rather than editing the numbers.
- 3
Count the remaining scope and inflate it
Count the items left in the roadmap, including the Tech Debt and Bug Budget epics, which are the first things people forget and the last things that stop consuming throughput. Count anything already in progress as remaining, because half-counting it flatters the answer. Then apply your split rate: the items an epic finally shipped divided by the items it carried when it left Ready for Dev, averaged over your last five closed epics. Most teams land between 1.2 and 1.5. If you have never measured it, use 1.3 this quarter and measure it properly for the next one. Forecast against the inflated number.
- 4
Simulate ten thousand quarters in a spreadsheet
A spreadsheet is enough.
Put the twelve weekly counts in A1:A12. For the fixed-date question, fill B1 across to J1, one cell per remaining week, with =INDEX($A$1:$A$12, RANDBETWEEN(1,12)), total the row in K1, then fill the block down ten thousand rows. Column K is now ten thousand plausible quarters. For the when-is-it-done question, extend the draws to thirty columns, running-total across the row, and record the first column that reaches your inflated scope. Copy and paste the results as values before reading percentiles, or the sheet resamples every time you touch it. Never average the twelve weeks. The spread is the whole point.
- 5
Read p50 and p85, commit at p85
Read the percentiles off the results column with PERCENTILE.INC.
For a date, 0.5 and 0.85 give the week half the simulations finished by and the week 85% of them did. For an item count against a fixed date, the 85% confident number is the 15th percentile of the totals: being 85% sure of delivering at least N items means only 15% of simulated quarters came in below N. Publish one sentence carrying both, for example forty-three items at p50 and thirty-eight at p85. Plan against p50, commit against p85. The gap between them measures how variable your delivery is, and closing it beats pushing the average up.
- 6
Convert the shortfall into currency
Rank the remaining epics in the order you will really deliver them, draw a line at the p85 count, and name every epic that falls below it.
Each carries a planned share of its outcome's currency target, so sum those shares and the shortfall stops being a mood and becomes a number. Take it to the outcome owner with three options and no fourth: defer this value to next quarter, cut something above the line to pull it up, or accept that the outcome target moves. Going faster is not on the list, and offering it wastes the meeting and the credibility of the forecast.
- 7
Refresh weekly and plot the trend
Re-run it the same morning every week with the new throughput week and a fresh scope count, and keep every version in one sheet: date, sample window, split rate, inflated scope, p50, p85. The useful artefact is not this week's p85, it is the line of p85s across the quarter. Three consecutive weeks drifting later is a signal you can act on in week five, whereas one bad week is noise. Put the current range on the daily dashboard next to the quarter end date, so divergence surfaces before stand-up rather than at a steering meeting in week eleven.
- 8
Check your calibration at roadmap close
At roadmap close, write down three comparisons: the p85 count you published against what finished, the forecast completion week against the real one, and the split rate you assumed against the one you got. Four quarters of that gives you a realisation rate, and if your organisation reliably realises 70% of what it plans, 70% is the number next quarter's planning uses rather than optimism. If actuals fall below p85 more often than roughly one quarter in six, the sample window is flattering you: too short, too recent, or missing the bad weeks.
A worked example: one team, nine weeks left
Take a generic mid-market grocer, call it Northwind Foods. One AI-augmented team, nine weeks left in the quarter. Their last twelve weekly story counts were 5, 3, 7, 4, 6, 2, 5, 8, 4, 5, 3 and 6: fifty-eight items, an average of 4.8 a week, and a spread wide enough to matter. The roadmap has 40 items left in it, Tech Debt and Bug Budget epics included, and their last five closed epics shipped 1.2 items for every item counted at Ready for Dev, so they forecast against 48 rather than 40. Ten thousand simulated nine-week runs give a p50 of 43 items and a p85 of 38. The coin-flip number misses 48 by five. The number they would commit to misses it by ten. Asked as a date instead, 48 items lands at p50 after ten weeks and at p85 after twelve, against nine weeks of quarter remaining. So ten items leave the quarter. Ranked in delivery order, the ten below the line are the whole of one epic carrying £180k of a £1.2m outcome target. Nobody asks for overtime. With nine weeks still to run, the outcome owner chooses between deferring that £180k to next quarter and dropping something above the line to pull it up. The same conversation with two weeks left has no options in it.
Where this goes wrong
- 01
Forecasting from velocity in story points.
Points get re-baselined every time a team re-points, so the series measures the scale as much as the work, and a rising line can mean nothing moved. Counted items cannot be quietly inflated.
- 02
Ignoring scope growth.
Teams forecast accurately against the backlog they counted and miss anyway, because the epics grew by a third between Ready for Dev and release. If you are not measuring your split rate, you are forecasting the wrong quantity precisely.
- 03
Averaging the history instead of sampling it.
An average assumes every remaining week is an average week. The interval exists because they are not, and the tail is where the misses live.
- 04
Deleting the weeks that look unrepresentative.
The zero week, the week of the incident, the week between Christmas and New Year: strip those out and you have removed exactly the variability the interval is meant to price, leaving a confident forecast of a quarter you have never had.
- 05
Publishing p50 and calling it the date.
Half your quarters miss it by definition. The business hears a commitment, the team hears a stretch, and when it slips the method loses credibility rather than the choice of percentile.
Done means
- The forecast is published as p50 and p85 together, and no single date appears anywhere in it.
- The unit counted, the sample window and the split rate used are written next to the numbers, so anyone can reproduce them.
- The scope count includes the quarter's Tech Debt and Bug Budget epics and everything currently in progress.
- Every epic falling below the p85 line is named with its planned currency share, so the value at risk is a number rather than a warning.
- The forecast was refreshed within the last seven days, and the trend of p85 across the quarter is visible on the daily dashboard.
- The outcome owner has made an explicit call on the shortfall, or there is no shortfall to call.
For an AI-augmented team the countable unit is stories closed and twelve weeks of history is plenty. For an AI-native team the outcome ticket is the unit of work, and outcome tickets are too few and too large to sample well, so count key user journeys demonstrated instead and pull sixteen weeks rather than twelve. Expect a wider p50 to p85 gap and publish it rather than smoothing it. Two further differences bite. Throughput history goes stale faster, because a change of model, harness or verification standard changes the system as much as a team split does, so treat those as reasons to shorten the window. And measure the split rate rather than assuming it: outcome tickets tend to grow more between approval and done than stories do, because more of the decomposition happens after the ticket is written.
The two delivery modes, side by side →Questions
How much history do I need before I can forecast?
Eight weeks is the working minimum and twelve is comfortable. What matters more than the length is whether the window describes the system you are in now. If the team doubled, the delivery mode changed, or the quarter opened with a two-week freeze, use the weeks since that change and accept a wider interval rather than padding the sample with data from a team that no longer exists. A wide honest range beats a narrow invented one.
The team is brand new and has no throughput at all. What then?
Borrow a reference class for the first six weeks: take the weekly throughput of a comparable team in the same organisation, label the forecast clearly as borrowed history, and publish a deliberately wide range. Replace one borrowed week with a real one every week until the sample is entirely yours. What you must not do is fall back on a single date because you have no data, since that is the situation in which invented dates are least defensible.
The business will not accept a range. What do I give them?
Give them p85 as the committed number and keep p50 as the internal plan, then say the remaining 15% out loud so nobody is surprised later. The range is not there for comfort, it is there to produce the currency number. Once the value at risk is on the table, the conversation stops being about whether the date is right and becomes a decision about which value gets deferred, which is the only version anyone can act on.
Do I need a forecasting tool to do this?
No. Ten thousand rows and two formulas in a spreadsheet produce the same answer as any Monte Carlo tool, and doing it by hand for a quarter teaches you where the forecast is fragile. Tools earn their place later, when you want the weekly refresh and the trend of p85 maintained without someone remembering. Buying one first does not help, because the arguments are always about the inputs, the unit, the window and the split rate, not about the maths.
More on forecasting and team health
These guides are written to be read in order.
02How to run a team health check
Monthly, per team, reviewed by both the team and management. The trend is the signal; any single month is noise.
← All seventeen guidesWant help installing this?
These guides are free and you owe us nothing for using them. If you would rather have operators install the operating model alongside your teams and stay until it sticks, that is what our engagements do.
Most organisations start with a fixed-price Agent-Readiness Audit · £30k–£90k · 6–8 weeks