Incrementality Testing
for DTC & Ecommerce

The three ways to measure incrementality

• Geo holdout and lift tests

  Suppress a channel in some regions, compare against control regions. Rigorous, but slow, expensive, and fragile when your regions are not comparable.

• Conversion lift studies

  Platform-run experiments. Useful, but the platform designs and grades its own test.

• Causal modeling on historical data

  Estimate the counterfactual from variation already in your sales history. No test to run, no revenue sacrificed, and you are not grading your own homework.

Why most brands cannot run clean holdouts

A trustworthy holdout needs comparable test and control groups, enough volume to detect an effect, and weeks of patience while you deliberately under-spend a working channel. Most DTC brands have none of those to spare. That is why incrementality testing so often gets talked about and so rarely gets done. Modeling incrementality from existing data removes every one of those blockers.

How to get incrementality from data you already have

Your spend, sales, and seasonality already contain natural experiments. Weeks you spent more, weeks you spent less, channels that paused, promotions that ran. Causal inference reads that variation to estimate what sales would have been without each channel, and reports the gap as incremental ROAS. Every estimate comes with a confidence interval so you can tell a real signal from noise.

Reading the confidence intervals

A tight interval well above 1.0x means the channel is almost certainly paying for itself. An interval that straddles 1.0x means the channel might not be incremental at all, which is exactly the situation where platform dashboards quietly mislead you. The interval is the point. It tells you how hard to lean on the number.

Frequently asked questions

Keep reading

Related on causal attribution