Cut a Channel the Right Way: Run the Holdout First
The safest way to cut a channel is to prove, first, that revenue survives without it.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 2 min read
Run the numbers for your store: the free holdout test planner.
Cutting on ROAS Is Cutting Blind
Killing a channel because its reported ROAS looks weak assumes the dashboard measured cause, when a low-ROAS channel can be quietly creating demand that other channels convert, so the honest test is to turn it off for a slice of your market and watch total revenue. Cut on evidence, not on a ratio.
We have all seen it: a channel posts a soft ROAS, someone proposes cutting it, and the number is treated as proof. But reported ROAS is a correlation with a platform's own ads. It cannot tell you whether the channel was propping up conversions credited elsewhere. Cut it and you might be fine. You might also watch three other channels sag.
The Holdout Answers the Real Question
A holdout test is the clean way through. Turn the channel off for a randomly chosen portion of your audience or geography, keep it on for the rest, and compare total revenue between the two. The gap is the channel's real incremental contribution, cause and all, with nothing for a platform to overclaim.
If revenue holds when the channel goes dark, cut with confidence. If it drops, you just avoided an expensive mistake.
When You Cannot Run a Clean Test
Holdouts are the gold standard, but they take time and traffic.
Either way, the rule holds: prove revenue survives the cut before you make it.
Frequently asked questions
How do I know if it is safe to cut a channel?
Killing a channel because its reported ROAS looks weak assumes the dashboard measured cause, when a low-ROAS channel can be quietly creating demand that other channels convert, so the honest test is to turn it off for a slice of your market and watch total revenue.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- ConversionConversion is a specific, desired action a user takes in response to a marketing message, such as a purchase or a sign-up.
- CorrelationCorrelation is a statistical measure showing a relationship between variables; it does not imply causation.
- DashboardA dashboard is a visual display of key information required to achieve specific objectives. It consolidates data onto a single screen for quick review.
- Holdout TestA holdout test is an experiment that keeps a random part of an audience from seeing a campaign and compares its results with those of the part that could see it. The held-out part is the control group, so the gap in outcomes estimates what the campaign added.
- IncrementalityIncrementality is the part of an outcome, such as orders or revenue, that a marketing activity caused: the extra results compared with what would have happened without it. It is a causal effect, not the number of conversions credited to a channel.
- RevenueRevenue is the total income generated by the sale of goods or services related to a company's primary operations.