Platform reported ROAS vs causal ROAS
Platform reported ROAS and causal ROAS answer different questions. Here is how to put them on the same window, and what the gap between them actually means.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 6 min read
Run the numbers for your store: the free ad platform over-reporting checker.
Use the same dates and the same spend. Platform-reported ROAS is the revenue an ad platform claims, divided by spend. Causal ROAS is the revenue that would not have happened without the ads, divided by spend, from a holdout test or a causal read. Divide one by the other to see how far the platform's claim is off.
Platform reported ROAS answers "how much revenue can this platform claim." Causal ROAS answers "how much revenue would not have happened without this spend." They are not competing estimates of one quantity. They are two quantities.
The two questions, side by side
| Platform reported ROAS | Causal ROAS | |
|---|---|---|
| Question answered | What can this platform claim? | What would not have happened otherwise? |
| Who computes it | The platform selling the ads | A method you can inspect |
| Counts an order the buyer would have placed anyway | Yes | No |
| Same order claimable by two channels | Yes | No |
The row that does the damage is the third one. A returning customer who was going to reorder, who sees a retargeting ad on the way, produces a claimed conversion and zero incremental revenue. Multiply that across a retargeting budget and you get a channel that reports beautifully and contributes very little.
How to compare platform-reported ROAS against causal ROAS
Putting the two numbers on the same window is mostly a data exercise, and it is worth doing by hand once before buying anything.
- Pull each platform's reported revenue for a fixed date range, with the attribution window written down.
- Pull the store's actual revenue for the same range from Shopify or your order table.
- Sum the platform claims. If the sum exceeds actual revenue, note the overlap.
- Get a causal estimate per channel for the same range, from a holdout test or a causal read.
- Put all of it in one table, by channel, with the window stated on every row.
- Divide reported ROAS by causal ROAS for each channel. Above one, the platform claims more than it caused. Below one, it caused more than it can see.
Step five is where most comparisons fall apart, because the platforms are not using the same window. A seven day click and one day view window is not comparable to a twenty eight day click window, and a table that mixes them is not a comparison. For the causal column from the GA4 export you already have, start with the free check; the full read per channel is €99.
What the research shows
Two studies speak to this, and they point in different directions. That is the useful part.
A platform can under-count as well as over-claim. Liang, Chen, Han, Zhang and Chen (2023) ran a randomized field experiment with an automobile brand's campaign on ByteDance:
"In a large-scale randomized experiment on ByteDance in collaboration with an automobile brand, we show a significant effect of the advertising campaign on ByteDance, compared with effects reported in other studies."
"Exposed users are eight times more likely to convert from outside than from within ByteDance, which raises the importance of information sharing between the platform and brands."
"When considering conversions outside ByteDance, the average cost per conversion, which brands commonly use to evaluate the cost of campaigns, shrinks by 5 or 25 times, depending on the methods used to calculate it."
Source: Liang, Y., Chen, X., Han, S., Zhang, J. and Chen, Y. (2023). Effectiveness of Advertising Campaigns on Short-Form Video Social Platforms: An Empirical Analysis through a Large-Scale Randomized Field Experiment on ByteDance. SSRN working paper; a second version is SSRN 4641827.
What it means here: the platform's own data saw the conversions that happened inside it. The experiment also counted the ones that happened outside. So a platform-reported number can be too low, not only too high, and the comparison needs a causal column, not a rule of thumb.
A model built on observational data needs correcting before it matches an experiment. Chen, Chan, Perry, Jin, Sun, Wang and Koehler (2018) studied search ads inside media mix models:
"Evaluating the return on ad spend (ROAS), the causal effect of advertising on sales, is critical to advertisers for understanding the performance of their existing marketing strategy as well as how to improve and optimize it."
"In this paper, we study the challenge associated with measuring the impact of search ads in MMM, namely the selection bias due to ad targeting."
"We use case studies to show that the method provides promising results by comparison with results from randomized experiments."
Source: Chen, A., Chan, D., Perry, M., Jin, Y., Sun, Y., Wang, Y. and Koehler, J. (2018). Bias Correction For Paid Search In Media Mix Modeling. arXiv:1807.03292.
What it means here: search ads are shown to people who are already searching, so a model that ignores the targeting credits the ad with demand that existed anyway. The authors check their correction against randomized experiments, the same yardstick the causal column uses.
Where the research stops
- Neither study measures the gap between a platform's dashboard ROAS and causal ROAS for an online store. Liang et al. report cost per conversion for one automobile brand on one platform. Chen et al. study search ads inside media mix models.
- Neither gives a ratio you can apply to your own numbers. The "5 or 25 times" belongs to one campaign's cost per conversion. It is not a benchmark.
- Chen et al. limit their own claim: "in some common, practical scenarios, one may be able to obtain an approximately unbiased estimate of search ad ROAS."
- The versions linked here are a working paper (SSRN) and a preprint (arXiv).
Your own ratio comes from your own data, channel by channel, on one window.
What the gap means
A large gap is not proof a channel is worthless. It has three ordinary explanations, and they imply different actions.
- The channel is genuinely harvesting demand created elsewhere. Cut it and revenue moves less than the reported number suggests.
- The channel creates demand that converts through another path. The claim is low and the contribution is real.
- The window is wrong, and the comparison is not one.
Ruling out the third before acting on the first two is the whole discipline. See causal versus last click attribution for what separates a claim from a cause.
Related answers
Frequently asked questions
How do I compare platform-reported ROAS against causal ROAS?
Use the same dates and the same spend for both. Platform-reported ROAS is the revenue an ad platform claims, divided by spend. Causal ROAS is the revenue that would not have happened without the ads, divided by spend, measured with a holdout test or a causal read. Divide one by the other per channel. Above one, the platform claims more than it caused. Below one, it caused more than it can see, which a randomized ByteDance experiment found when most conversions happened off the platform.facebook says i made 5000 euros shopify says 2000 which one is right
Both can be right about different things. Facebook counts every sale it can link to an ad in its attribution window, including sales that would have happened anyway and sales another channel also claims. Shopify counts the sales that happened, by its own rules. Neither tells you how many of those sales the ads caused. A holdout test answers that, and so does a causal read of your GA4 export. The free check at causalityengine.ai/start is the place to begin.What tools compare platform reported ROAS with causal ROAS?
Any method that can produce a per channel causal estimate on the same date range as the platform's own report will do it, including a geo holdout, a marketing mix model, or a causal read on a GA4 export. The hard requirement is that every row states its attribution window, otherwise the two columns are not comparable.Why is platform reported revenue higher than my actual revenue?
Because several platforms can each claim the same order. Summing self reported revenue across channels double counts every order that touched more than one channel, which is most of them.Does a big gap mean the channel is worthless?
Not on its own. It can mean the channel harvests demand created elsewhere, that it creates demand converting through another path, or simply that the two numbers use different attribution windows. Rule out the window problem before acting on the other two.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Attribution WindowAttribution Window is the defined period after a user interacts with a marketing touchpoint, during which a conversion can be credited to that ad. It sets the timeframe for assigning conversion credit.
- CorrelationCorrelation is a statistical measure showing a relationship between variables; it does not imply causation.
- Holdout TestA holdout test is an experiment that keeps a random part of an audience from seeing a campaign and compares its results with those of the part that could see it. The held-out part is the control group, so the gap in outcomes estimates what the campaign added.
- Last Click AttributionLast Click Attribution: Assigns all credit for a conversion to the final marketing touchpoint before that conversion.
- Media Mix ModelingMedia Mix Modeling is a statistical technique that measures the collective impact of marketing and advertising on sales. It uses historical data to inform budget allocation.
- Return on Ad Spend (ROAS)Return On Ad Spend (ROAS) measures the total revenue generated for each dollar spent on advertising. It indicates campaign profitability and effectiveness.
- Selection BiasSelection Bias occurs when data points selected for analysis do not represent the target population. This leads to distorted findings about marketing campaign impact.