Price tests: read margin per visitor, not conversion
A higher price can lower conversion rate and still earn more, so conversion rate is not the verdict. Judge a price test on margin per visitor after ad spend, and run it as a switchback or a product split instead of a before-and-after.
By Joris van Huët, Founder & CEOPublished 5 min read
A price test is not decided by conversion rate. Raise a price and fewer visitors may buy, while each buyer leaves more margin. The number that decides it is margin per visitor, and it is the one most price tests never compute.
Price comes up twice on ecommerce YouTube this week. A Depop dropshipping guide tells sellers to test their price points (the video), and a talk on premium brands asks why some brands can charge far more than their competitors without living on discounts (the video). Both are right that price is a lever. The measurement question is how you would know you pulled it the right way.
The number that decides it
For each price, pull these from your own store, over the same number of days:
- Visitors to the product page.
- Orders and revenue.
- Cost of goods, shipping, payment fees and refunds for those orders.
- Ad spend, if you paid for the traffic.
Then two lines of arithmetic. Contribution margin is revenue minus cost of goods, shipping, fees and refunds. Margin per visitor is that contribution margin divided by visitors. If you paid for the traffic, subtract the ad spend first: the number that matters is margin after ad spend, per visitor.
A higher price that shows a lower conversion rate and a higher margin per visitor won. A lower price that sells more units and leaves less per visitor lost, however busy the warehouse looks.
Why before-and-after lies here
- Everything else moves too. Season, traffic mix, promotions and competitors change from week to week, and a before-and-after credits all of it to the price. See test now or wait until the season ends.
- Your ads react. If your campaigns optimize for purchases, fewer purchases at a higher price can change who the ads reach, so the test's audience shifts in the middle of the test.
- Returns can move. A price change can change who buys and what comes back, so count refunds, as in returns-adjusted ROAS.
Who grades the campaign
The ad platform grades you on cost per purchase and on return on ad spend. A price rise makes cost per purchase worse by design, because you sell fewer, more valuable orders on similar spend. Its report will call a better business a worse campaign. That is not the platform lying. It is the platform counting orders while you are trying to count margin. POAS vs ROAS makes the same argument at account level.
Three test designs that hold up
| Design | How it works | Good for |
|---|---|---|
| Switchback | Alternate two prices by week, for several cycles, changing nothing else | One product with steady traffic |
| Product split | Raise the price on a random half of comparable products, keep the rest as control | A range of similar products |
| Market split | Change the price in one country or currency, not in a comparable one | Stores selling in several markets |
The product split is a small randomized controlled trial. The market split is read as a difference in differences: the change in the test market minus the change in the comparison market over the same weeks. The switchback works because season and noise hit both prices across several cycles, instead of hitting one price in one block of weeks.
What to avoid: showing two prices to random visitors at the same moment. It is the cleanest design on paper, and customers who compare notes will not see it that way.
Reading a switchback
Line the weeks up by price, not by date. Add up visitors, orders, revenue and costs for all the weeks at each price, and compute margin per visitor from each total. Then look at the cycles one at a time. If one price wins most cycles, the result is steadier than if a single big week carries the total. A result that flips from cycle to cycle is not a result yet: run more cycles, or accept that the two prices earn about the same, which is also an answer worth having.
Before you start
- Fix the rule first: the metric (margin per visitor after ad spend), the length, and the result that makes you roll back.
- Size it. A price test needs enough orders in each arm to read. How long a test should run depends on your own weekly orders.
- Keep the page the same. New photos or copy during a price test make the result about two things at once.
- Watch the second order. A higher price can change who buys, and so who comes back. Read repeat purchases for the test cohort at a fixed horizon before you call it.
Where offers fit in
Price tests and offer tests answer the same question from different ends. A discount is a temporary price cut with a label on it, and a free-shipping threshold or a quantity break is a price change dressed as a perk. Each one deserves the same margin-per-visitor read. Free-plus-shipping offers and the quantity-break discount walk through two of them.
The premium-brand argument in the video above is, in measurement terms, a bet that fewer discounts and a higher price earn more margin per visitor over a year. That is testable. Run the switchback on one hero product, read margin per visitor after ad spend, and you have your own answer instead of someone else's brand story.
Related answers
Frequently asked questions
How do I know if a price increase worked?
Compare margin per visitor at each price, after cost of goods, shipping, fees, refunds and ad spend. A lower conversion rate with a higher margin per visitor is a win.Why is a before-and-after price test unreliable?
Everything else moves too: season, traffic mix, promotions and the way your ads deliver. A before-and-after credits all of it to the price. Alternate the prices over several weeks, or split comparable products, instead.Why did my cost per purchase go up after a price increase?
Because you sold fewer orders on similar spend. Cost per purchase counts orders, not margin, so a price rise makes it worse even when each order earns more.Should I show different prices to different visitors at the same time?
It gives the cleanest statistics, and it risks trust if customers compare notes. Switchback tests and product-level splits answer most price questions without it.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Keep reading
Terms in this article
- Ad SpendAd Spend is the total amount invested in advertising campaigns. It is measured against Return on Ad Spend (ROAS) to evaluate campaign effectiveness.
- ConversionConversion is a specific, desired action a user takes in response to a marketing message, such as a purchase or a sign-up.
- Conversion rateConversion Rate is the percentage of website visitors who complete a desired action out of the total number of visitors.
- Difference In DifferencesDifference In Differences is a quasi-experimental method that estimates the causal effect of an intervention. It compares outcome changes over time between a treatment group and a control group.
- IncrementalityIncrementality measures the true causal impact of a marketing campaign. It quantifies the additional conversions or revenue directly from that activity.
- Product PageProduct Page is a webpage dedicated to a single product. It includes images, descriptions, pricing, and purchase options.
- RevenueRevenue is the total income generated by the sale of goods or services related to a company's primary operations.