Part V · Landing it · Chapter 14
Rollout and handover
Demo, pilot, controlled production and full production are four different promises. How to move between them and when to stop.
12 min read
"Launch" means different things to everyone in the room. The sponsor pictures the whole company using it next month. Security pictures a limited trial with tight controls. Engineering pictures a deploy. Most rollout trouble starts from that confusion, not from the technology. This chapter is about naming the stages and moving between them on evidence.
What you'll take away
- 01Five stages from showcase to general release, and what each one can honestly claim.
- 02Why shadow mode is the cheapest risk reduction you can buy.
- 03How to agree, before a trial starts, what would widen it and what would halt it.
- 04How to widen a rollout safely, and why adoption must be designed.
- 05What a proper handover contains, and why 'done' is the customer's word to say.
Name the stage you are promising
Software teams have long used words like alpha, beta and general availability to keep these expectations apart. Use whichever words the client already knows; what matters is that everyone means the same thing. This handbook uses five stages.
| Stage | Who touches it | What you can honestly claim afterwards |
|---|---|---|
| Showcase | You, driving | It can be done |
| Shadow | Nobody acts on it; you compare its output with what people did | How often it would have been right, on real work |
| Trial | A named team, checking everything it produces | Whether it helps, and what people need from it |
| Limited release | More people, with limits, monitoring and support | Whether it holds up day to day, and what it really costs |
| General release | Everyone it was meant for, run by the client | The business result, at scale |
When a sponsor says "live by March", ask which row they mean. Then say which row you can stand behind by March. Squeezing a general release into a trial's timeline means skipping work that shows up later, usually at the worst possible moment.
Shadow first, then a trial
In shadow mode the system runs on real inputs alongside the existing process, and nobody acts on its output. You compare its decisions with what people actually did. It costs almost nothing in risk and shows you real error rates, real latency and real edge cases before anyone depends on it. Two weeks of shadow running often reveals more than two months of testing on curated data.
Then choose the trial team deliberately.
- Representative work: include the awkward cases, not just the easy ones.
- A willing champion in the group who will give honest feedback and push through early friction.
- Measurable: you need a baseline for this group, from before the pilot.
- Bounded: a named list of users, document types, regions or values, written in the pilot plan.
- Time-boxed, typically four to six weeks, with a decision meeting already in the diary.
Agree the widen and halt rules first
Before the trial starts, sit down with the sponsor and the people who will use the system and write two short lists. What would we need to see to widen this? What would make us halt it? Decided now, they are a plan. Decided after the trial, they are an argument.
| Ask | A widen rule might be | A halt rule might be |
|---|---|---|
| What do the people using it say? | Most of the trial team would be unhappy to lose it | Staff are working around it to get their job done |
| What do the numbers say? | The agreed outcome measure has moved, for each kind of case | Errors on any one kind of case pass the agreed limit |
| Has anything happened that must never happen? | No, and we tested for it | Yes, even once |
| Can the client run it without us? | An owner, alerts and a tested way to switch it off | Nobody will take ownership |
| Is it worth what it costs? | Cost per case, including people checking it, beats today | It costs more than the old way with nothing to show for it |
Decide what would halt the trial before it starts. Afterwards, nobody is neutral.
Widening the rollout
- 1
Put it behind a switch
A feature flag or configuration setting that turns the system off, or sends everyone back to the old process, in minutes and without a deploy. Test it before you need it. - 2
Widen in steps
By team, region, case type or a percentage of traffic. Google's SRE writing on canary releases is a good guide: expose a small slice, compare it with the rest, and widen only when the comparison is clean. - 3
Watch the right signals
Business outcome, error and correction rates by case type, latency, cost per unit, and complaints. Not just whether the service is up. - 4
Keep the way back open
The old process should keep working until the new one has proven itself at the current step. Don't let anyone decommission it early.
Adoption is designed, not hoped for
A system nobody uses has an accuracy of zero. Adoption depends far more on how the tool fits people's work than on how good the model is.
- Put it where the work happens: inside the tool people already have open, not in a new tab they have to remember.
- Remove a step, don't add one. If using the tool takes longer than not using it, people won't.
- Make feedback one click, and show people that it changes things.
- Train with their own cases, including what to do when the tool is wrong.
- Watch for workarounds: copy-pasting out of the tool, ignoring suggestions, keeping a private spreadsheet. Each is a design problem to fix.
Usage is not the goal. A tool can be used heavily and change nothing. Keep reporting the business outcome you agreed in the problem brief.
Handover, and who says it's done
Eventually the client's team owns the system. A good handover means they can run it, change it safely and know when it is going wrong, without calling you.
The handover pack
- Architecture and decision log: what was built and why, including what was rejected.
- Runbook: how to restart, roll back, rotate keys, and handle the top five likely incidents.
- Monitoring: dashboards, alerts, who receives them, and what each alert means.
- The eval set and how to run it, so they can change prompts or models safely.
- Known limitations and open risks, written plainly.
- Owners for the system, the data sources, the rules and the budget.
- Service levels agreed for support after handover.
Then wait. Done is not the day of full rollout. It is when the client has watched the system run for an agreed period, compared it with the baseline, and confirmed the outcome. Agree that period and that comparison in advance. The client saying so is what ends the engagement, until the next policy change opens it again.
Try it
Write the pilot plan
A regional bus operator will pilot an assistant that drafts responses to passenger complaints, for staff to edit and send. About 2,000 complaints a month arrive. Today each takes about 15 minutes, and complaints about accessibility have a legal response deadline.
Write the trial scope, two widen rules, two halt rules, and how you'll know the pilot worked.
Write your answer down first, then reveal the worked answerWorked answer
Scope: one depot's complaints team for six weeks, all complaint types except accessibility, which stays fully manual for the pilot because of the legal deadline. Staff edit and send every draft.
Widen rules: quality, meaning at least 70% of drafts sent with only light edits, measured per complaint type; and operations, meaning a named owner in customer services, a runbook, and the switch back to manual drafting tested.
Halt rules: any draft that commits to compensation or makes a factual claim about a specific incident that staff can't verify; or the light-edit rate falling below 50% for two consecutive weeks.
Success: median handling time down from about 15 to under 8 minutes, with no rise in repeat complaints compared with the same weeks last year. A strong answer protects the legally sensitive category from the start rather than averaging it in.