The design menu, honestly priced
| Design | Answers | Cost and constraint |
|---|---|---|
| Geo holdout (dark markets) | Does this channel cause registrations/FTDs at all, and roughly how much? | Revenue forgone in holdout geos; needs matched markets and weeks of patience |
| Geo lift (incremental spend) | What does the next unit of budget buy? | Cheaper politically (no channel goes dark); answers marginal, not total, lift |
| Switchback (time-based) | Fast read on tactics inside one market | Carryover effects blur it; fit for short-decay tactics, wrong for brand |
| User-level holdout (owned channels) | CRM, bonus and lifecycle causal lift | The gold standard where you control delivery — which paid media is not |
Match the design to the question and the volume. The recurring institutional mistake is running the cheap design because the right one requires telling a channel owner their market goes dark for six weeks.
Pre-registration: the one-page contract
Before launch, one page states: the unit and the cells (which geos, matched how), the window (including the maturation tail), the primary metric and its exact dictionary definition, the minimum detectable effect the volume supports, and the decision rule (what result moves what budget). The page is the countermeasure to every post-hoc temptation — and the reason a null result is a finding rather than an embarrassment. A test that cannot state its decision rule in advance is not a test; it is content for a slide.
iGaming-specific hazards
- Bonus contamination. A promo calendar that differs between test and control geos measures the promo, not the channel. Freeze the offer surface across cells for the window — coordination that must be agreed before launch, not discovered in week three.
- Affiliate spillover. Affiliates don't respect your geo cells; their traffic keeps flowing into dark markets. Decide upfront whether the test measures the channel alone or the channel's interaction with the affiliate baseline, and read accordingly.
- Regulatory asymmetry. Cells must share a regime — comparing a market mid-regulatory-change against a stable one measures the regulator.
- Seasonality and the sporting calendar. Sports-led acquisition lives on fixtures; a test spanning a major tournament in some geos and not others is measuring the fixture list. The same held-out-tournament discipline as the personalization case.
Reading results without wishful maths
Three habits close the loop. Report the interval, not just the point — a lift of "somewhere between nothing and substantial" is a real result that honest meetings can still use. Convert the result into the channel's truth multiplier — caused conversions over platform-claimed conversions — and apply it to that channel's claims until the next test. And log every test, including the nulls, in a registry: the institution's calibration memory is worth more than any single result, and it is what keeps the same argument from being re-run every quarter on vibes.
Continue reading: MMM for iGaming — where anchored truths become allocation. The measurement stack — the cluster overview.