Measurement & experimentation
Know which plays create pipeline, and run tests big enough to tell.
Roadmap: AI agents & measurement, Portfolio & interview readiness
Why it matters
GTM engineering is judged on pipeline and revenue, not on how impressive the tables look. Leaders will ask which campaign, signal or channel produced the meetings, and whether the new copy really beat the old. Small samples, vanity metrics and loose attribution make it easy to fool yourself. Being the person with honest numbers and clear tests earns you a seat in planning conversations.
What it's made of
- The outbound funnel: sent, delivered, replied, positive reply, meeting, opportunity, won
- Choosing a primary metric for each test
- Sample sizes and how long a test must run
- Attribution rules and their limits
- Holdout groups to measure true lift
- Cost per meeting and cost per opportunity
- Dashboards that lead to decisions
- Writing up results, including the ones that failed
Do the funnel math first
Suppose you contact 1,000 people. A 3% positive reply rate gives 30 interested replies. If half book and 70% show, about 10 meetings happen. If half of those become qualified opportunities at an average $30,000 deal size, that's about $150,000 of pipeline. Now you can see which lever matters: doubling positive replies does more than a 10% lift in show rate, and you can estimate what a campaign is worth before running it.
Tests need more volume than you think
Low base rates need large samples. To reliably detect a lift from 2% to 3% positive reply rate (with the usual 95% confidence and 80% power), you need roughly 3,800 contacts per version. Detecting 2% to 2.5% takes several times that. With small lists, test bigger changes (a different angle or segment) rather than subject-line tweaks, and treat early results as directional.
Attribution: pick a rule and say it out loud
There's no perfect attribution. What matters is a consistent, written rule. Common choices:
- Sourced: outbound created the opportunity if the first meeting came from an outbound sequence and the account had no open deal in the prior 90 days.
- Influenced: outbound touched any contact on the account in the 90 days before the opportunity was created.
- Holdout: keep a random 10–20% of a target segment out of a play, then compare pipeline rates. This is the most honest way to measure real lift.
Report for decisions
A weekly report should answer: what did we run, what happened, what are we changing? Show cost per meeting and per opportunity next to volume, call out tests still too small to read, and include failures. A short "we stopped signal X because it produced 0.3 meetings per 100 accounts versus 1.1 for the baseline" is more valuable than ten charts.
Mid-level looks like
Tracks reply rate, positive reply rate and meetings per campaign, and runs simple A/B tests on copy.
Senior looks like
Ties activity to pipeline and revenue, sizes tests before running them, uses holdouts to measure lift, reports cost per opportunity, and is open about uncertainty.
Common pitfalls
- Calling a winner after 200 sends because one version got 3 replies more.
- Reporting open rates as success when they're inflated by privacy features and bots.
- Counting a meeting as created by outbound when the account was already in an active deal.
- Changing the list, copy and timing at once, then not knowing what caused the result.
Practice drills
Drill 1
Given a quarter's campaign data, calculate positive reply rate, meeting rate, cost per meeting and pipeline per campaign, and recommend which two campaigns to scale and which to stop.
A strong answer: Correct funnel math, costs included, small-sample results flagged, and recommendations tied to the numbers.
Drill 2
Plan an experiment comparing two messaging angles for a segment of 6,000 contacts. State the hypothesis, metric, sample size, duration and decision rule.
A strong answer: One variable changed, a realistic detectable lift for the volume, randomised assignment, and a decision written before results come in.
Drill 3
The VP of Sales says outbound "isn't working". Design a holdout test that would settle the question within a quarter.
A strong answer: Random holdout at the account level, a pipeline metric rather than replies, enough accounts to detect a meaningful lift, and agreed rules for counting.
Read next
Traction
Gabriel Weinberg, Justin Mares
Surveys nineteen customer acquisition channels and a process for testing them cheaply. A good map before you decide which channel to automate.
Hacking Growth
Sean Ellis, Morgan Brown
Describes the cross-functional growth team and its rapid test cadence. Useful for setting up how your experiments get proposed, prioritized and reviewed.
Lean Analytics
Alistair Croll, Benjamin Yoskovitz
Helps you pick the one metric that matters for a given business model and stage. Read it before building dashboards so you measure what drives decisions.
Trustworthy Online Controlled Experiments
Ron Kohavi, Diane Tang, Ya Xu
The rigorous reference on A/B testing pitfalls: sample ratio mismatch, novelty effects and bad metrics. Read it before claiming that a new sequence or scoring model won.