Part III · Designing the system · Chapter 6
The lightest tool that works
Rules, classifiers, models, workflows, agents: climb the ladder only as far as the problem needs, and price every rung.
11 min read
There is a ladder of tools from fixing a form field to deploying an autonomous agent. Each rung up buys flexibility and costs money, predictability and upkeep. The skill is climbing only as far as the problem needs, and knowing what each rung will cost the customer every month.
What you'll take away
- 01Seven rungs from process fixes to agents, and what each one is good for.
- 02The running costs each rung brings, including the ones that are not on the invoice.
- 03How to compare a unit of work against what the customer pays for it today.
- 04Signals that justify climbing a rung, and why the best systems mix several.
The ladder
Start at the bottom and ask, rung by rung, whether this level solves enough of the problem. Most of the time you stop sooner than the customer expected.
The bottom rung deserves more respect than it gets. Plenty of "AI projects" are really a missing field on a form, a notification nobody sends, or two systems that disagree about a customer's ID. Fixing those is not glamorous. It is often the highest-return work in the engagement, and it makes everything you build afterwards more reliable.
Every rung sends a bill
Customers tend to compare options on build cost. The running cost is what decides whether a system survives its first budget review.
| Option | Running costs | Hidden costs |
|---|---|---|
| Rules and queries | Close to zero per item | Someone must update rules when policy changes, and know they exist |
| Small trained model | Very cheap per call | Labelled data, retraining, a hosted endpoint, monitoring for drift as live data moves away from training data |
| One model call | Tokens per item, scaling with input length | Evals, output validation, latency, provider outages and price changes |
| Fixed workflow | Several calls per item | Each step needs its own checks and tracing; failures compound across steps |
| Agent | Unbounded unless you cap it | Variable cost and latency per item, wide tool permissions, harder audits, more ways to fail |
Neither column is a reason to avoid models. A trained classifier needs MLOps that a team may not have. A model call might replace a pile of brittle rules that nobody understands. The point is to list both bills, recommend one, and be honest about how your own company earns its fee.
Price one unit of work
Pick the unit the customer already thinks in: a ticket, an invoice, a claim, an application. Work out what one unit costs today, then what it would cost with each option, including the people still needed to review or correct it.
| Triage of one support ticket | Cost per ticket | Notes |
|---|---|---|
| Today: agent reads and routes | $0.60 | About 75 seconds at a loaded cost of $29 an hour |
| Keyword rules | $0.00 + $0.21 | Correct on 65% of tickets; the other 35% still need a person |
| Small classifier | $0.001 + $0.09 | Correct on 85%; plus a share of training and hosting |
| Model call, short prompt | $0.003 + $0.05 | Correct on 92%; the human cost is reviewing the uncertain 8% |
| Agent with tools | $0.04 + $0.05 | Similar accuracy; more tokens per ticket and harder to audit |
The figures are illustrative, but the shape is typical. The machine cost is often tiny next to the human cost of the cases the machine gets wrong. A cheaper model that is wrong more often can cost more overall. So can an expensive agent that is no more accurate than one well-designed call.
Always put your number next to theirs. Customers rarely keep a solution that costs more per unit than the process it replaced, unless it buys something they can name. If yours does cost more, say so, and say what the difference pays for.
Your cost per unit, next to their cost per unit, is the most persuasive slide you will ever show.
When to climb a rung
- The input is free text, images or speech, and the meaning matters. Rules on raw language break constantly.
- The rules would run to hundreds of cases and change often. A model can be cheaper to maintain than a rulebook nobody understands.
- You have no labelled data and can't get it soon. A model that works from instructions can stand in until you can.
- The steps genuinely vary per case in a way you can't enumerate. That is the narrow territory where an agent earns its cost.
And signals to stay where you are: the decision must be exactly reproducible for an auditor; the input is already structured; a wrong answer is expensive and hard to reverse; or the team that will run the system can't support the next rung.
Most good systems are hybrids
The question is rarely "rules or AI?". It is which part of the work goes to which tool. A pattern that holds up well: rules decide what they can decide, models handle meaning, people handle the exceptions, and rules also act as guardrails around the model. Thresholds, eligibility checks, value limits and anything a regulator might ask about belong in code. Reading a messy email, extracting fields from a scan or drafting a reply belong to a model.
A hybrid also degrades gracefully. If the model provider has an outage, the rules still handle most of the volume, and the rest queues for people. A system that sends everything through a model stops completely.
Try it
Pick the lowest rung
For each task, choose the lowest rung that would handle most of it, and say why.
- Flag invoices where the total doesn't equal the sum of the line items.
- Route incoming emails to one of eight teams, with two years of correctly routed history.
- Pull the policy number, incident date and description from scanned claim forms in three layouts.
- Tell customers when their parcel is out for delivery.
- Research a prospect across their website, filings and news, then draft a briefing.
Write your answer down first, then reveal the worked answerWorked answer
1. Rules. It is arithmetic. A model would be slower, cost more and occasionally get it wrong.
2. A small trained model. Two years of labelled history is exactly what a classifier needs. Keep a confidence threshold and send uncertain emails to a person.
3. One model call with structured output, or a document-extraction service. Validate the fields with rules, such as the policy number format and the date being in the past.
4. Fix the process. Trigger a message from the courier's status event. No AI needed.
5. A fixed workflow, possibly a small agent. The steps are fairly predictable, so start with a fixed sequence of search, read and summarise. Use an agent only if research paths genuinely vary, and cap its budget.