How to run an incrementality test, step by step
Pick one channel and one question. Check how long buyers take in GA4 and whether Google Ads will run Conversion Lift. Split regions with Shopify's billing location report, switch the channel off in the test regions, and compare sales once the window closes.
By Joris van Huët, Founder & CEOUpdated 7 min read
Run the numbers for your store: the free holdout test planner.
To run an incrementality test, pick one channel and split your regions or audience into a test group and a control group. Switch the channel off for one group, run it longer than your buyers usually take to decide, then compare sales. If Google Ads offers you Conversion Lift, it does the splitting for you.
This is the do-it-yourself route, with each menu path taken from the platform's own help pages. Google Ads flips the switch, GA4 sets the timing and Shopify keeps score. Meta runs lift studies too: a paper on lift test design notes that Facebook provides them to measure incrementality. This guide sticks to the three tools whose menus are spelled out below.
First, match the test to the question. Google's geo experiment guide names three kinds. Go-dark switches off spend you already run, to check it still earns its keep. Holdback launches something new in some regions only. Heavy-up adds budget in test regions, to see whether more money still pays. The steps below run a go-dark test on a channel you already pay for.
Step by step
- Name one channel and one question. Pick the channel with the most credit and the least certainty. Write the question down: does it add sales, or collect them? In GA4, open Advertising, then Attribution models under Attribution, and compare each channel's revenue under data-driven and paid and organic last click.
- Measure how long your buyers take. The test has to outlast the decision, or late buyers land after it ends. In GA4, open Advertising, then Key event attribution paths under the Key events dropdown, and read Days to key event. Google's advice is to set a study length that captures your average conversion lag.
- Check whether Google will run it for you. In Google Ads, open Campaigns, then Experiments, then the Lift studies tab. Select the plus button and choose Conversion Lift. Google lists two entry tickets for the user-based version: at least 1,000 observed conversions and a $5,000 USD minimum campaign budget. For a regional study, open Lift measurement in the Goals menu instead.
- Choose test and control regions from your own sales. In Shopify admin, go to Analytics, then Reports, and filter by the Sales category. Open Total sales by billing location. Pick two groups of regions whose sales rose and fell together before the test. Google's geo guide asks for pre-test data at least 3 times the length of the test.
- Switch the channel off in the test regions. In Google Ads, open the campaign's settings, expand Locations, remove country-level targeting and select Enter another location. Target the control regions only, then expand Location options and select Presence. Keep the campaign out of any shared budget.
- Read the result where the money lands. When the test ends, compare each group's sales in Total sales by billing location against its pre-test pattern. In GA4's Attribution models report, an include filter on Region under User shows channel credit by region. If Google ran the study, open Goals, then Measurements, then Lift measurement.
A worked example
In one store's Journeys sheet, journeys with 1 touch took 0.5 days to buy. The same Journeys sheet puts journeys with two or three touches at 12.5 days, and four to nine touches at 16.9 days. Google allows lift studies as short as 7 days but typically recommends more than 14. In this store, a one-week test would end well before the 12.5 days that two-to-three-touch journeys took. Plan for three weeks or more.
Now the arithmetic, with round numbers made up for the example. For illustration, run a three-week go-dark test on one paid channel, with nine weeks of pre-test data to meet Google's three-times rule.
For illustration, say your test regions sold €30,000 in those nine weeks and your control regions sold €60,000. So the test regions normally sell half of what control sells. Say the control regions sell €24,000 during the test. If the channel did nothing, the test regions should sell about €12,000.
If the test regions sell €11,000 with the channel off, the channel was adding about €1,000 there. For illustration, that is a lift of about 9%, because €1,000 on top of €11,000 is roughly 9%. If the platform had credited itself with €3,000 of sales in those regions, two thirds of its credit was sales you would have had anyway.
Then the money. If the channel would have cost €800 in the test regions over those three weeks, each euro brought back about €1.25 of caused sales. One store's Break-even sheet gives the yardstick: at a 40% margin, break-even ROAS is 2.5x, which is 1 divided by 0.40. For illustration, €1.25 back per euro sits well under 2.5x, so at that margin the tested spend lost money. It still caused sales. It just caused too few for what it cost.
What to check when the report looks wrong
- The control regions jumped on day one. If the campaign is limited by budget, removing test regions pushes the unspent money into control regions. Google warns this inflates the baseline. Cut the daily budget to the control regions' usual share before the start.
- Traffic dipped in both groups. Google notes that switching from country to city or postcode targeting reduces overall traffic. Judge the test on the gap between the groups, not on the totals.
- One region went its own way. A local promotion, a launch or a pop-up in one group breaks the comparison. Google's guide says to watch for major regional launches during the test and drop those regions.
- Sessions moved but sales didn't. Shopify's session measurement rollout runs from 21 to 23 September 2026. Treat session and conversion rate shifts across those dates as a measurement change, and judge the test on sales.
- No lift showed up. Google's geo report marks a study as having no significant lift when the two groups don't differ enough to tell apart. That can mean the channel adds little, or that the test was too short or too small to see it.
- Sales kept arriving after the end date. Late buyers still count. Google's geo results offer an optional cooldown period after the test, recommended when your conversion cycle runs longer than a few weeks. Campaigns go back to normal during the cooldown, but keep counting sales until it closes.
What to do this week
- Put a number on your buying time. In GA4, open Key event attribution paths under Advertising. Filter the path length to more than one touchpoint and read Days to key event. Pass: you have a day count to set the test length by. Fail: the report is empty, which usually means no purchase is marked as a key event yet.
- Shortlist your regions. In Shopify, open Total sales by billing location for the last three months. Pass: you can form two groups of regions whose weekly sales rise and fall together. Fail: one or two regions hold nearly everything, so a user-based study fits better.
- Check the campaign's budget setup. In Google Ads, open the campaign's settings and expand Budget under Budget and bidding optimization. Pass: it has its own daily budget. Fail: it shares a budget with other campaigns, which Google's guide says a go-dark campaign should not do.
Check the homework. Your GA4 Attribution paths export already holds the evidence. Causality Engine reads that one file and shows what each channel caused next to what last-click gave it, in 1 to 2 minutes, for €99 once (excluding VAT), refundable within 30 days. Check the homework
Sources, 1 October 2026: Designing Experiments to Measure Incrementality on Facebook (arXiv). Implement campaigns for geo experiments (Google Ads Help). Key event attribution models report (Analytics Help). Key events attribution paths report (Analytics Help). Set up Conversion Lift based on users (Google Ads Help). Set up Conversion Lift based on geography (Google Ads Help). Understand your Conversion Lift based on geography measurement data (Google Ads Help). Sales reports (Shopify Help Center). Acquisition reports (Shopify Help Center).
Related answers
Frequently asked questions
How long should an incrementality test run?
Longer than your buyers take to decide. Google's minimum is 7 days and its usual advice is more than 14, with pre-test data at least 3 times the test length. Read Days to key event in GA4 before you pick the dates.Can I run an incrementality test without Google's Conversion Lift?
Yes. A do-it-yourself go-dark test needs only campaign location settings and a sales report by region. Switch the channel off in the test regions, keep it on in the control regions, and compare sales against their pre-test pattern.What if my incrementality test shows no lift?
Check the test before you blame the channel. No lift can mean the channel adds little, or that the test was too short or too small to see it. Review the study power or feasibility estimate, the test length and any regional promotions before you cut spend.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Keep reading
Terms in this article
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Attribution ModelAn Attribution Model defines how credit for conversions is assigned to marketing touchpoints. It dictates how marketing channels receive credit for sales.
- Control GroupControl Group is a segment of an audience intentionally not exposed to a marketing campaign, used to measure the campaign's true causal impact.
- Conversion rateConversion Rate is the percentage of website visitors who complete a desired action out of the total number of visitors.
- ExperimentsExperiments are scientific procedures that test hypotheses or demonstrate facts. In marketing, experiments like A/B tests determine the causal effect of campaign changes, enabling data-driven decisions.
- IncrementalityIncrementality measures the true causal impact of a marketing campaign. It quantifies the additional conversions or revenue directly from that activity.
- Incrementality TestingIncrementality Testing measures the additional impact of a marketing campaign. It compares exposed and control groups to determine causal effect.
- Lift MeasurementLift Measurement: A method to determine the incremental impact of a marketing campaign by comparing exposed and control groups.