What each number can and cannot say
| Number | Good for | Structurally blind to |
|---|---|---|
| Platform ROAS | Ranking campaigns and creatives within one platform | Incrementality, cross-channel comparison, its own modelling error |
| Warehouse funnel | What actually happened: registrations, FTDs, cohort value per attributed source | What would have happened anyway; upper-funnel influence |
| Experiments | Causal lift of a channel or tactic, in a defined window | Everything outside the tested cell; needs volume and patience |
| MMM | Cross-channel allocation over quarters, including effects experiments can't isolate weekly | Fast feedback; anything the input data never varied |
The dysfunction in most operations is not a missing number — it is one of these four being asked a question that belongs to another. The stack below assigns the questions.
The three-clock decision stack
- Daily — steer. Platform metrics plus your own funnel counts (clicks → registrations → FTD) manage creative fatigue, bid drift and broken landing flows. Decisions at this clock are small and reversible; that is what platform numbers are fit for.
- Monthly — calibrate. Incrementality tests on the biggest budget lines produce channel-level truth multipliers: how much of the claimed conversion volume is caused. The multipliers travel — they discount tomorrow's platform claims until the next test.
- Quarterly — allocate. MMM, anchored by the experiment results, moves money between channels and markets. Allocation is the only question at this clock, and it is the only layer entitled to answer it.
The operator's own funnel: non-negotiable properties
Everything above assumes a warehouse-side acquisition funnel with four properties: one precedence rule with the affiliate ledger (no player funded twice), registration-anchored cohorts with the four choices declared, value read at honest windows (censoring respected — the payback page is entirely about this), and RG-suppressed players excluded from every optimisation export exactly as from every audience. Without these, the stack calibrates noise.
Anti-patterns worth naming
- Channel league tables on platform numbers. Each platform's ROAS uses different modelling and windows; ranking them against each other compares methodologies, not channels.
- Brand-term triumph. Paid search on your own brand name claiming players who typed your name into a search box. The cheapest incrementality test there is usually the most sobering.
- Day-7 payback verdicts. The deposit cycle matures over weeks and months; killing a campaign on an immature cohort read is how portfolios drift toward instant-gratification traffic with worse long-run value.
- The blended CAC comfort blanket. One number over all channels and markets hides every problem worth finding — the same argument as the blended approval rate, in a different department.
Continue reading: Incrementality testing — the calibration layer, designed properly. CAC, LTV and payback — the unit economics underneath the verdicts.