Own the test results, not just the winners
Most teams keep the winning variant and discard the rest. The discarded results are the part that would have made the next decision cheaper.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 2 min read
The losing variants are data you paid for. Throwing them away means paying again for the same lesson.
What usually survives a test
The winner ships. The loser is deleted. The interval, the sample size, the date range and the hypothesis are in a chat thread that scrolls away.
Six months later someone proposes the losing idea again, in good faith, because there is no record that it was tried.
What a kept result looks like
| Field | Why |
|---|---|
| The hypothesis, as written before the test | Stops post-hoc reinterpretation |
| Both variants, not just the winner | So the comparison can be rerun or revisited |
| Sample size and date range | Tells a later reader how much to trust it |
| The interval, not just the direction | An unresolved result is different from a null one |
| What else was running | Promotions and launches that could confound it |
The fourth row is the one that changes behaviour.
The compounding argument
A team with two years of kept results can answer new questions from history: has this been tried, at what size, with what outcome. A team without them runs every test fresh.
The cost difference is not in the tests. It is in the tests you did not need to run.
Where to keep it
Somewhere you own and can query, not in a tool whose export you have never tried. The check is simple: try exporting it once, now, before you need it.
The same argument applies to channel data, and a causal read on a Google Analytics export works from a file you keep rather than a dashboard you rent. It shows what each channel caused next to what last-click gave it, with a data-health score rather than a confidence interval, for 99 euro once, refundable within 30 days, no questions asked.
Related answers
Frequently asked questions
Why keep losing test variants?
Because they are data you already paid for. Without a record, the same idea gets proposed again in good faith six months later and the test is run twice.What should a kept test result contain?
The hypothesis as written beforehand, both variants, sample size and date range, the interval rather than just the direction, and what else was running that could confound it.Why does the interval matter in the record?
Because lost and not resolved lead to different next actions.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AnalyticsAnalytics is the systematic computational analysis of data. It reveals customer behavior and measures campaign performance.
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- ClickClick is the action a user takes to interact with a digital advertisement, redirecting them to a website or landing page. Clicks are a fundamental metric for measuring ad engagement and a primary input for click-based attribution models.
- Confidence IntervalConfidence Interval is a statistical range of values that likely contains the true value of a metric. In marketing analytics, it quantifies uncertainty around estimates, indicating the precision of an outcome or causal effect.
- DashboardA dashboard is a visual display of key information required to achieve specific objectives. It consolidates data onto a single screen for quick review.
- Google AnalyticsGoogle Analytics is a web analytics service that tracks and reports website traffic.
- IncrementalityIncrementality is the part of an outcome, such as orders or revenue, that a marketing activity caused: the extra results compared with what would have happened without it. It is a causal effect, not the number of conversions credited to a channel.