Own the test results, not just the winners: Most teams keep the winning variant and discard the rest. The discarded results are the part that would have made the next decision cheaper.
Read the full article below for detailed insights and actionable strategies.
The attribution problem
One sale. Four channels. 400% credit claimed.
Reported revenue: €400 · Actual revenue: €100 · Gap: €300
The losing variants are data you paid for. Throwing them away means paying again for the same lesson.
What usually survives a test
The winner ships. The loser is deleted. The interval, the sample size, the date range and the hypothesis are in a chat thread that scrolls away.
Six months later someone proposes the losing idea again, in good faith, because there is no record that it was tried.
What a kept result looks like
| Field | Why |
|---|---|
| The hypothesis, as written before the test | Stops post-hoc reinterpretation |
| Both variants, not just the winner | So the comparison can be rerun or revisited |
| Sample size and date range | Tells a later reader how much to trust it |
| The interval, not just the direction | An unresolved result is different from a null one |
| What else was running | Promotions and launches that could confound it |
The fourth row is the one that changes behaviour. "Variant B lost" and "Variant B was not resolved" lead to different next actions, and most records preserve only the first.
The compounding argument
A team with two years of kept results can answer new questions from history: has this been tried, at what size, with what outcome. A team without them runs every test fresh.
The cost difference is not in the tests. It is in the tests you did not need to run.
Where to keep it
Somewhere you own and can query, not in a tool whose export you have never tried. The check is simple: try exporting it once, now, before you need it.
The same argument applies to channel data, and a causal read on a Google Analytics export works from a file you keep rather than a dashboard you rent. It returns an estimate, a confidence interval and a coverage share for 99 euro once, refunded if it does not move a budget decision. The interactive demo shows the output with no signup.
Related answers
Key Terms in This Article
Analytics
Analytics is the systematic computational analysis of data. It reveals customer behavior and measures campaign performance.
Attribution
Attribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
Causality
Causality is the relationship where one event directly causes another, essential for identifying specific actions that drive desired outcomes in marketing.
Confidence Interval
Confidence Interval is a statistical range of values that likely contains the true value of a metric. In marketing analytics, it quantifies uncertainty around estimates, indicating the precision of an outcome or causal effect.
Dashboard
A dashboard is a visual display of key information required to achieve specific objectives. It consolidates data onto a single screen for quick review.
Google Analytics
Google Analytics is a web analytics service that tracks and reports website traffic.
Incrementality
Incrementality measures the true causal impact of a marketing campaign. It quantifies the additional conversions or revenue directly from that activity.
Related Articles
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Ready to see your real numbers?
Own the budget? Upload your GA4 export and see which channels drive incremental sales, with confidence intervals, in minutes. Have to defend it? Start with the live demo and take the read to your CFO.
Full refund if you don't see value.
Stay ahead of the attribution curve
Weekly insights on marketing attribution, incrementality testing, and data-driven growth. Written for the person who owns the budget and the person who has to defend it.
No spam. Unsubscribe anytime. We respect your data.
Frequently Asked Questions
Why keep losing test variants?
Because they are data you already paid for. Without a record, the same idea gets proposed again in good faith six months later and the test is run twice.
What should a kept test result contain?
The hypothesis as written beforehand, both variants, sample size and date range, the interval rather than just the direction, and what else was running that could confound it.
Why does the interval matter in the record?
Because lost and not resolved lead to different next actions. Most records preserve only the direction, which makes an underpowered test look like a settled negative.