When to re-test a channel you already ruled out

A test result is a reading of one period, with one product, one offer and one market. When those change enough, a "does not work" verdict expires, and the only way to know is to test again.

A recent ecommerce video is a public reversal. An established creator revisits advice they gave months earlier, that a business model's margins were too thin, after finding a supplier they say changes the economics (video). Whatever you think of the model, the move itself is sound measurement practice. If you keep a list of channels, offers and products you tested once and ruled out, look at how many of those verdicts carry an expiry date.

Why old verdicts go stale

What changes underneath a verdict:

• The product. A new supplier, a better photo set, a lower price or a new bundle changes what the channel was selling.
• The offer and the margin. A channel that could not pay back at one margin can pay back at another.
• The market. Competitors enter and leave, auction prices move, and seasons shift demand.
• The platform. Targeting options, placements, attribution windows and delivery systems change without asking you.
• Your measurement. The first test may have been read on last-click numbers, over a short window or on a small sample.

The last one is the easiest to miss, because it means the verdict may never have been right in the first place. A test that could not detect the effect you cared about did not show that the channel failed. It showed that the test was too small. How long an incrementality test should run covers the sample side, and statistical significance the reading side.

Verdicts go stale in the other direction too. A channel you kept because it worked two years ago deserves the same expiry date as one you cut, and the same re-test when its conditions change.

Nobody is paid to reopen your old verdicts. The platform you cut would love to have you back, which is exactly why its own reporting is not how you should decide.

How to tell whether a verdict is worth reopening

Take the verdict you are least sure of and answer four questions from your own records.

1. What was measured? Platform-reported conversions, last-click revenue in GA4, or total orders against a holdout? Only the last is a test of whether the channel caused sales.
2. Could the test have seen the effect? Write down the smallest lift that would have changed your decision, and check whether the test's sample and length could have detected it. If they could not, the result is unknown, not no.
3. What has changed since? List the product, price, margin, creative, market and platform changes since the test. Two or more material changes are reason enough to look again.
4. Is the old data still there? GA4 event data retention can be set to 2 or 14 months on a standard property, and the setting governs explorations and funnel reports, while standard aggregated reports are unaffected (Google's help page). If you want the old test period at event level, export it before it ages out.

What to test when you reopen one

Rules for a re-test:

• Decide the stakes first. Write down the result that would keep the channel off and the result that would turn it back on, before the test starts. It is the same discipline as defending an attribution finding: name what would change your mind.
• Use a design with a control group. A holdout by region or by audience, with total orders as the outcome rather than the conversions the platform claims. Cut a channel the right way: run the holdout first works in reverse for switching one back on.
• Stay out of peak season. A re-test that runs into peak trading cannot be separated from the season, and test now or wait until the season ends explains the trade.
• Size it to the decision. Small, cheap and time-boxed is fine, as long as it can detect the smallest effect that would change your mind.

Write the verdict down with its conditions

Whatever the re-test shows, record it in a form the next person can check.

A verdict record, one line each:

• Verdict: keep off, turn on, or unknown.
• Period tested, with the dates.
• Conditions: the product, offer, price, margin and market it applies to.
• How it was measured: a holdout with total orders, platform conversions, or last click.
• Smallest effect the test could detect, next to the smallest effect that would have changed the decision.
• Reopen when: the two or three changes that would make the verdict worth testing again.

A verdict with its conditions written down tells you when it has expired. One without them just hardens into habit, and the cost of cutting the wrong channel keeps accruing while it does. The record also makes the next re-test cheaper, because the design and the stakes are already on paper.

Where a read fits

Then, if you want a second opinion before you spend on a re-test, a causal attribution read such as Causality Engine's reads whatever period you export from GA4, with no lookback limit, and shows what each channel caused next to what last-click gave it. Run it on an export of the old test period and on one of the months since, and you have two readings to set side by side. It is €99 once per upload, excluding VAT, with a full refund within 30 days, no questions asked.

Related answers

• The cost of cutting the wrong channel
• How long should an incrementality test run?
• Cut a channel the right way: run the holdout first
• Test now or wait until the season ends
• Your Meta creative test is not an A/B test