Incrementality testing for DTC and ecommerce
Incrementality testing measures the sales a channel actually caused, not the ones it merely touched. The classic way to do it is a holdout. There is a faster way.
€99 once, excl. VAT. Full refund within 30 days, no questions asked. You keep the read.
By Joris van Huët, Founder & CEOUpdated
The classic method is a holdout: switch a channel off for some users or regions, wait, and compare. It works, but it is slow, it costs you real revenue while the test runs, and many ecommerce brands cannot hold a clean control group.
Causality Engine answers the same question from data you already have. Upload a GA4 export and it estimates what each channel caused, in 1–2 minutes, with a data-health score on every channel, and without turning anything off.
The three ways to measure incrementality
| Method | How it works | The trade-off |
|---|---|---|
| Geo holdout and lift tests | Suppress a channel in some regions and compare against control regions. | Rigorous, but slow, expensive, and fragile when your regions are not comparable. |
| Conversion lift studies | Experiments the ad platform runs on its own inventory. | Useful, but the platform designs and grades its own test. |
| Causal modeling on historical data | Estimate the counterfactual from variation already in your sales history. | No test to run and no revenue sacrificed. It relies on the variation in your data, which is why our read scores data health on every channel. |
A platform's lift study is a real experiment: Meta's lift studies randomise accounts into a test group that sees your ads and a control group that does not. It is still Meta's experiment on Meta's inventory, graded by Meta, and it says nothing about the channels around it.
Why many brands cannot run clean holdouts
A trustworthy holdout needs comparable test and control groups, enough volume to detect an effect, and weeks of patience while you deliberately under-spend a working channel. Many DTC brands have none of those to spare. That is why incrementality testing so often gets talked about and so rarely gets done. Modeling incrementality from existing data removes the test itself: no control group to hold, no weeks of under-spending.
How to get incrementality from data you already have
Your spend, sales and seasonality already contain natural experiments: weeks you spent more, weeks you spent less, channels that paused, promotions that ran. Causal inference reads that variation to estimate what sales would have been without each channel; the gap is what that channel caused. Every channel also gets a data-health score from 0 to 100, so you know when to move budget in steps.
Reading a lift test's confidence intervals
When a holdout or lift test reports incremental ROAS, a tight interval well above 1.0x means the channel is almost certainly paying for itself. An interval that straddles 1.0x means the channel might not be incremental at all, which is exactly the situation where platform dashboards quietly mislead you. The interval is the point: it tells you how hard to lean on the number.
Frequently asked questions
What is incrementality testing?
Incrementality testing measures the conversions a marketing channel caused that would not have happened otherwise. It separates incremental sales from sales the channel merely received credit for.Do I have to run a geo holdout?
No. Causality Engine estimates incrementality from your existing GA4 history using causal modeling, so there is no holdout to design and no revenue lost to a test.How is this different from a platform's conversion lift study?
The platform designs and grades its own lift study. Causal modeling on your first-party data is independent of any single platform's incentives.How long does it take?
1–2 minutes on a GA4 export. A holdout test typically runs for weeks.
Related on causal attribution
- Causal attributionWhat it is, and how it differs from last-click and multi-touch.
- Marketing mix modelingModern MMM for SMB and DTC brands.
- Holdout testThe classic incrementality experiment, explained.
- Uplift modelingModeling incremental effect per customer.
- Potential outcomes frameworkThe Rubin causal model: what would have happened if not for X.
- Regression to the meanWhy naive lift studies overstate the effect of an intervention.
- ExogeneityThe identifying assumption every credible incrementality estimate rests on.
- D-separationHow to read a causal graph to know which channel effects can be identified.
Your platforms guess.
We run the math.
Upload a GA4 export and see what each channel caused, next to last-click, in 1–2 minutes. The read is yours to keep.
The free finding runs in your browser. Your file never leaves it. The full read is €99, refundable within 30 days, no questions asked. Prices exclude VAT.