Marketing ROI measurement: the order to do things in
Reconcile platform totals to your store, watch MER weekly, run a holdout on one channel, and only then build a model. Each step fixes an input the next one needs.
By Joris van Huët, Founder & CEOPublished 5 min read
Run the numbers for your store: the free marketing ROI calculator.
Reconcile first, watch MER second, run holdouts third, and model last. Each step uses what the one before it settled: the revenue definition, then the trend, then a causal number, then a model calibrated on that number. Skip ahead and the later step has nothing to stand on. Google's Meridian documentation works an example where 104 weekly data points for 26 parameters are "too low to estimate the model reliably", and calls incrementality experiments "perhaps the strongest basis" for calibrating one.
Why does the order matter?
Because the first step fixes the numbers the others use, starting with which revenue you count. Shopify's total sales is "gross sales - discounts - sales reversals + taxes + shipping + fees", while net sales is "gross sales - discounts - sales reversals". MER, total revenue over total ad spend, should use net sales, or it counts tax, shipping and fees as if they were revenue. Shopify also records a return "as a negative number on the date the return was processed", so a week of sales and a week of returns are not the same orders.
The platforms disagree with each other and with GA4 before any of that: Google Ads counts 30 days after a click by default and GA4 looks back 90 days for most key events (windows, side by side). So platform revenue is a claim, and the store's net sales is the ceiling to check it against. This order is an argument from dependencies, not a tested protocol. The evidence below supports the parts.
What does each step decide?
| Step | Question it answers | What it can decide | What it cannot decide |
|---|---|---|---|
| Reconcile | Do the platform totals fit the store? | Whether a platform's revenue column can be read as it stands, and which revenue you use | Which channel caused a sale |
| MER | Is total efficiency moving? | Whether to hold, spend more or test, once a move is bigger than the usual wobble | Which channel to change |
| Holdout | Did this channel's spend cause revenue? | Keep, cut or scale one channel, on your own data | Channels you did not hold out, or spend levels you did not test |
| Model | How should budget split across channels and spend levels? | An allocation, once calibrated with holdout results | Anything, if the data is thin or spend barely varied |
Why do holdouts come before the model?
A holdout is the only step here that creates its own control group. In 15 Facebook experiments, observational methods "often fail to produce the same effects as the randomized experiments" (Gordon et al., Marketing Science, 2019). The abstract describes observational models run on "user-experiment observations" and does not mention media mix models, so it is the case for a control group, not a verdict on a model.
Holdouts are noisy, which is why you size them. Across 25 field experiments, "The median confidence interval on return on investment is over 100 percentage points wide" (Lewis and Rao, Quarterly Journal of Economics, 2015). Check the smallest lift your test can see with the holdout test calculator before you start.
What does a model need before it is worth building?
Meridian's own page on data says its guidance is "rough and directional", then does the arithmetic. With 12 media channels, six controls and eight knots there are 26 parameters, and "With two years of weekly data (104 data points), you have four data points per parameter." Cut the model to 10 parameters and use three years of weekly data, and 156 data points gives "roughly 15 data points per parameter", enough for "directional information" at best. It adds that "insufficient variation in the media spend adversely impacts national models". Meridian's guidance on priors says "Incrementality experiments are perhaps the strongest basis for formulating your intuition", and a holdout result is that kind of input. See marketing mix modeling for the method.
What do you run this week?
- Reconcile. Put each platform's purchases for one closed week next to Shopify orders by order date. The pass is a platform total at or below the order count. The fail is a total above it, so stop reading platform ROAS as it stands.
- Watch MER. Divide net sales by ad spend each week. The pass is a move larger than its usual week-to-week wobble: something changed, so test it. A flat line means there is nothing to decide, and platform ROAS alone is not a reason to move budget.
- Hold out one channel. Choose the one you would cut first and size the test before you run it. The 30-day version of this plan has the calendar.
- Model only when the count works. Add up channels, controls and time knots, then divide your weekly data points by that number. By Meridian's own worked examples, nearer four than 15 means the model is not ready.
Later, if you want a second opinion on a channel before you spend a test on it, a causal attribution read like Causality Engine's takes one GA4 export and shows what each channel group caused next to last-click.
Sources, 30 September 2026: Amount of data needed (Meridian documentation, Google for Developers, updated 2026-06-03); Calibrate treatment priors (Meridian documentation, Google for Developers, 2026); A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook (Gordon, Zettelmeyer, Bhargava and Chapsky, Marketing Science, 2019); The Unfavorable Economics of Measuring the Returns to Advertising (Lewis and Rao, Quarterly Journal of Economics, 2015); Finance reports (Shopify Help Center, 2026); Sales report (Shopify Help Center, 2026); About conversion windows (Google Ads Help, 2026); Select attribution settings (Google Analytics Help, 2026).
Related answers
Frequently asked questions
In what order should I measure marketing ROI?
Reconcile platform totals to your store first, then watch MER weekly, then run a holdout on one channel, and build a model last. Each step fixes an input the next one needs, and each can be checked against your own numbers.Do I need marketing mix modeling?
Only when the data supports it. Google's Meridian documentation works an example where 104 weekly data points for 26 parameters is too low to estimate the model reliably. Fewer channels, more history and holdout results to calibrate it make a model worth building.Why use MER instead of platform ROAS?
MER divides your total net sales by total ad spend, so it never assigns a sale to a platform and cannot be double counted. It cannot say which channel to change. A holdout answers that.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Keep reading
Terms in this article
- Causal AttributionCausal Attribution uses causal inference to determine which marketing touchpoints genuinely cause conversions, not just correlate with them.
- Confidence IntervalConfidence Interval is a statistical range of values that likely contains the true value of a metric. In marketing analytics, it quantifies uncertainty around estimates, indicating the precision of an outcome or causal effect.
- Control GroupControl Group is a segment of an audience intentionally not exposed to a marketing campaign, used to measure the campaign's true causal impact.
- Google AnalyticsGoogle Analytics is a web analytics service that tracks and reports website traffic.
- IncrementalityIncrementality measures the true causal impact of a marketing campaign. It quantifies the additional conversions or revenue directly from that activity.
- Incrementality TestingIncrementality Testing measures the additional impact of a marketing campaign. It compares exposed and control groups to determine causal effect.
- Marketing MixThe marketing mix is the set of actions a company uses to promote its brand or product. It traditionally includes product, price, place, and promotion.
- Marketing Mix ModelingMarketing Mix Modeling (MMM) is a statistical analysis that estimates the impact of marketing and advertising campaigns on sales. It quantifies each channel's contribution to sales.