A real marketing test you can run with no data team
A genuine experiment does not need an analyst. A single-channel pause, sized right and written down first, is available to anyone with a calendar.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 3 min read
Run the numbers for your store: the free holdout test planner.
A genuine experiment needs a calendar and discipline, not an analyst. The simplest one available to a founder is a single-channel pause, and it works if you get three things right.
The three things
| Decision | Get it right by |
|---|---|
| Which channel | Pick one you genuinely suspect, with enough spend to matter |
| How long | At least two full purchase cycles, dark |
| The read window | Written down before you start |
Why the channel choice matters
Pausing your smallest channel produces no readable result, because its effect is smaller than the ordinary week-to-week movement in your orders. Pausing your largest may cost more than you want to risk.
The middle is usually right: a channel with real spend that you are unsure about. If you do not know which that is, an observational read will tell you, and that is a €99 one-time upload of a Google Analytics export, refundable within 30 days, no questions asked.
Why two purchase cycles
Because a channel's effect does not stop the day you pause it. People who saw the ad last week still buy this week. Pause for one week and you measure a week that still contains most of the channel's influence, which produces a comfortable result that means nothing.
Find your typical repeat interval in your store data and double it. The design considerations are in how to measure incremental lift.
Why the read window goes first
Because if you choose it afterwards you will choose the flattering one, without meaning to. Write the dates down before the pause starts and read exactly those dates.
What to hold still
Everything else. No promotion, no new product launch, no creative overhaul on other channels during the test. If that is impossible, which it often is for a small brand, a geographic split is the better design because it holds time constant instead. The geo testing guide explains it.
Reading the result honestly
Compare orders during the dark period against the equivalent period before it. Then say out loud what you cannot rule out: seasonality, a competitor's promotion, weather, anything. One pause on one channel in one period is one observation, and treating it as proof is the most common way founders draw a wrong conclusion from a right method.
Pair it with an observational read on the same window. Two weak instruments agreeing is meaningfully better than either alone.
What this buys you
The ability to say, in a conversation with an agency or a platform rep, that you tested it. That sentence changes the conversation more than any dashboard, and it is available to anyone willing to hold a channel dark for a fortnight. The related dynamic is in settling attribution debates with your agency.
Related answers
Frequently asked questions
Can I run a marketing experiment without an analyst?
Yes. A single-channel pause needs a calendar and discipline rather than statistics. Choose a channel with enough spend to matter, pause for at least two full purchase cycles, and write the read window down before you start.Which channel should I pause to test?
Not the smallest, because its effect is below the noise in your orders, and not necessarily the largest if the revenue risk is too high. A middling channel you genuinely have doubts about is usually the right choice.What can I conclude from one pause test?
That the result is consistent with one story about one channel in one period. It is one observation, not proof. Say out loud what you cannot rule out, and pair it with an observational read on the same window.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AnalyticsAnalytics is the systematic computational analysis of data. It reveals customer behavior and measures campaign performance.
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- DashboardA dashboard is a visual display of key information required to achieve specific objectives. It consolidates data onto a single screen for quick review.
- Google AnalyticsGoogle Analytics is a web analytics service that tracks and reports website traffic.