Measure Shopify channel lift in a single afternoon
An afternoon, four stages, and a checkpoint at each. What to pull, what to check it against, what to read first in the output, and what to write down at the end.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 3 min read
Run the numbers for your store: the free holdout test planner.
An observational lift read on a Shopify brand is genuinely an afternoon's work, and the afternoon has four stages with a checkpoint each. Skipping a checkpoint is how people end up with a number they cannot defend a week later.
The afternoon
| Stage | The work | Checkpoint |
|---|---|---|
| Pull | Export the acquisition window from analytics | Retention still covers the whole window |
| Anchor | Compare export order counts to Shopify order counts | The two agree within a few percent |
| Read | Run the causal read, look at the data-health scores first | You can name the lowest data-health score from memory |
| Record | Write the decision, the number and the date | A colleague could reconstruct why |
Pull
Choose a window long enough that spend actually varied within it. A fortnight where everything was flat gives the method nothing to work with, and the honest output will be wide intervals across the board. Check your analytics retention setting reaches back far enough before you export, because it expires quietly; the retention trap has the detail.
Anchor
This is the stage most people skip and it is the one that catches real problems. Total the orders in your export and compare them against the same window in Shopify admin. A gap of a few percent is normal. A gap of thirty percent means your analytics is missing a large share of purchases, and every channel estimate built on it inherits that gap.
If the anchor fails, stop. Fix the collection before you read anything, because a lift number computed on sixty percent of your orders is a number about sixty percent of your business. The related idea, coverage, is explained in the unassigned traffic problem.
Read
Look at the confidence intervals before the estimates. The intervals tell you which channels the window could actually resolve. Channels below the level of spend where any method can separate an effect are named as unmeasured rather than scored, which is covered in the measurability floor.
Record
One paragraph: what you decided, the number it rested on, the interval, and the date.
What it costs
A €99 one-time read on a Google Analytics export, refundable within 30 days, no questions asked, covers the afternoon described here.
What the afternoon does not buy
It does not buy you a randomised result. If the decision at stake is large enough that being wrong is expensive, use the afternoon to identify the channel worth testing properly, then run a geo holdout on that one. The afternoon is the triage, not the verdict.
Related answers
Frequently asked questions
How do I check my analytics data against Shopify orders?
Total the orders in your export for the window and compare them against the same window in Shopify admin. A few percent difference is normal. A large gap means analytics is missing purchases, and every channel estimate built on that export inherits the gap.How long a window should I export for a lift read?
Long enough that spend actually varied within it. A flat fortnight gives the method nothing to separate, and the honest result is wide intervals on every channel.What should I read first in a lift report?
The confidence intervals, before the estimates. The intervals tell you which channels the window could actually resolve. A tight interval below break-even is an instruction; a wide interval spanning it is a request for more data rather than a weak channel.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AnalyticsAnalytics is the systematic computational analysis of data. It reveals customer behavior and measures campaign performance.
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Confidence IntervalConfidence Interval is a statistical range of values that likely contains the true value of a metric. In marketing analytics, it quantifies uncertainty around estimates, indicating the precision of an outcome or causal effect.
- Google AnalyticsGoogle Analytics is a web analytics service that tracks and reports website traffic.
- IncrementalityIncrementality is the part of an outcome, such as orders or revenue, that a marketing activity caused: the extra results compared with what would have happened without it. It is a causal effect, not the number of conversions credited to a channel.
- Lift TestA lift test is an experiment that measures lift, the extra conversions or sales a campaign causes, by comparing a group that could see it with a control group that could not. Lift is reported as a difference or as a ratio to the control group's result.
- Multi-Touch AttributionMulti-Touch Attribution assigns credit to multiple marketing touchpoints across the customer journey. It provides a complete view of channel impact on conversions.
- ShopifyShopify is an ecommerce platform for creating online stores and selling products. Attribution modeling shows which marketing channels drive traffic and conversions within Shopify.