Incrementality and marketing mix modelling

Attribution tells you what happened before a sale. Incrementality asks the question the finance team really cares about: how many extra sales did the advertising cause? Incremental sales are the ones that would not have happened without the ad. Everything else was going to happen anyway.

A farmer wants to know if a new fertiliser works. Measuring how much grew in the fertilised field is not enough; it might have been a rainy year. So the farmer leaves one strip of the field unfertilised and compares. That untreated strip is a holdout group, and the difference in harvest is the incremental effect.

Lift studies and holdout groups

A lift study randomly splits a target audience into a test group that can see the ads and a holdout group (or control group) that cannot. After the campaign, you compare outcomes. If 2.0% of the test group buys and 1.6% of the holdout group buys, the lift is 0.4 percentage points, or 25%. Only that 0.4 was caused by the ads.

1. Define the outcomeSales, sign-ups, app installs, store visits or brand awareness measured by survey.
2. Randomise fairlyAssign people, households or regions to test and control by chance, so the only systematic difference is the ads.
3. Keep the control cleanControls may see a charity ad (a PSA) or nothing, or platforms use "ghost ads" that record who would have been shown the ad.
4. Run long enoughCollect enough outcomes for the difference to be statistically meaningful, not noise.
5. Compute incremental returnDivide incremental revenue by spend to get incremental ROAS, often far lower than attributed ROAS.

Geo experiments

When individual users cannot be tracked or randomised, advertisers randomise places. A brand might advertise in some cities or postal regions and not in comparable ones, then compare sales. Geo tests work for TV, radio, out-of-home and retail, and they survive signal loss because they use aggregate sales data. They are common in large markets such as the US, India and Brazil where many similar regions exist.

Marketing mix modelling (MMM)

Marketing mix modelling uses statistics on years of weekly or daily data (spend per channel, prices, promotions, seasonality, competitors, even weather) to estimate how much each channel contributed to sales. It was invented for TV-era brands and is enjoying a revival because it needs no user-level tracking. Open-source tools have made it more accessible: Meta released Robyn, and Google made its Meridian model generally available in early 2025.

The strongest measurement programmes combine all three layers: attribution for day-to-day optimisation, experiments to test what attribution claims, and MMM calibrated with those experiments to set long-term budgets. This is sometimes called a unified or triangulated measurement approach.

Invalid traffic ruins experiments too

If bots make up part of your test group, they dilute the measured lift, because bots never buy. If fraud targets are uneven between regions, geo tests become biased. MMM can be fooled when a channel's spend rises alongside organic demand. Filtering invalid traffic before measurement keeps every method honest.

Common pitfalls

• Contaminated controls: if the holdout group sees the ads anyway through another platform or device, measured lift shrinks toward zero.
• Too little data: small tests produce noisy results that look precise. Estimate the needed sample size first.
• Short windows: brand-building effects can take months to show in sales, so a two-week test may undervalue video and TV.
• Marking your own homework: a platform measuring its own lift has an interest in the result. Independent or advertiser-run tests are more credible.