How to set up a holdout group from your customer list

Usually in seven steps. Build a segment of past buyers in Shopify and export it. Give each customer a random number in a spreadsheet and mark a slice as the holdout. Upload that slice to Meta as an exclusion list and keep it out of every ad set. Afterwards, export the buyers and compare.

This is the do-it-yourself holdout. It needs no lift-study minimums and no account rep, just a Shopify segment, a spreadsheet and Meta's customer lists. It suits ads aimed at people who already bought from you: a win-back push, retargeting to customers, or a launch to your list.

It also writes nothing back into Shopify. Both groups live in your spreadsheet, so a typo cannot overwrite a customer profile.

Step by step

1. Build a segment of the customers you could reach. From your Shopify admin, go to Customers, click Segments, then Create segment. For example, type orders_placed MATCHES (count >= 1, date >= -12m) for everyone who ordered in the past year, and click Run query. Shopify shows the number of matching customers above the editor; click Save segment. Path: Shopify admin > Customers > Segments > Create segment.
2. Export that segment. With the segment open, click Export and choose the option for customers matching your filters. Pick the CSV for Excel, Numbers, and other spreadsheet programs, then click Export customers. Shopify emails the file when you export more than 50 customers. Path: Shopify admin > Customers > Export.
3. Give every customer a random number. Open the file in Google Sheets and type =RAND() in a new column. Fill it down to the last customer. Google's help says RAND returns a random number between 0 inclusive and 1 exclusive. Copy the column and paste it back with Paste values only (Ctrl + Shift + v), so the draw stays fixed. Path: Google Sheets > new column > =RAND() > Paste values only.
4. Mark the holdout. Add a Group column with a rule such as =IF(C2<0.1,"holdout","test"), which sends roughly a tenth of customers to the holdout. Nobody picks names by hand; the random number does. How big that slice should be is its own question. Path: Google Sheets > Group column > =IF(...), then COUNTIF to count each group.
5. Upload the holdout to Meta as an exclusion-only list. Save the holdout rows as their own file. Meta notes that under the GDPR, advertisers act as the data controller for any list they upload, so check your legal basis first. In Ads Manager, open Audiences, select Create audience, then Custom audience, then Customer list. At Add list, choose Only exclusions, so nobody on your team can target these people by mistake. Path: Ads Manager > Audiences > Create audience > Custom audience > Customer list > Only exclusions.
6. Exclude it from every ad set that could reach them. In a Sales ad set, go to Audience and select Show more controls. Add the list under Custom audience exclusion. In an Awareness, Traffic or Engagement ad set, the same job sits under Controls, then Exclude these custom audiences. Check every live ad set, including ones built long before the test. Path: Ads Manager > ad set > Audience > Show more controls > Custom audience exclusion.
7. Run, then count who bought. Leave budgets, audiences and offers alone for the planned window. Afterwards, build a segment of everyone who ordered in the window and export it, then mark those customers in your sheet with COUNTIF. Compare the share who bought in each group. Path: Shopify admin > Customers > Segments > Create segment, then Customers > Export.

For the segment in step 7, orders_placed MATCHES (count >= 1, date BETWEEN 2026-10-06 AND 2026-11-16) follows the date format in Shopify's own examples. Score everyone you assigned. A holdout customer Meta never matched stays in the holdout, and a test customer who never saw an ad stays in the test.

A worked example

For illustration, with round invented numbers. Say your past-year segment holds 20,000 customers, and the random column sends 2,000 of them to the holdout. Say you plan a four-week Meta win-back campaign.

Leave room for slow deciders. In one store's export, journeys with 2 to 3 touches took 12.5 days to buy (Journeys sheet). So you count orders for six weeks: the four campaign weeks plus two.

Suppose each extra buyer places one order of €60. For illustration, the campaign then added €10,800 of sales. Suppose it spent €6,000. For illustration, its incremental ROAS is €10,800 / €6,000 = 1.8x.

At a 40% margin, break-even ROAS is 1 / 0.40 = 2.5x, the arithmetic on the Break-even sheet of one store's export. So at that margin, the campaign loses money on the buyers it truly added.

Now the twist. If the dashboard credits the campaign with 700 purchases, its ROAS reads 7.0x: 700 × €60 / €6,000. Same campaign, two answers. Only the holdout priced what the ads caused.

What to check when the report looks wrong

• The holdout bought more than the test group. With small groups, chance can swing the gap either way. Before you call the ads harmful, check both group sizes, then rerun with a bigger holdout or a longer window.
• Holdout customers still saw ads. One ad set without the exclusion is enough to leak. Check every live campaign, including always-on ones nobody touches.
• The match score is low. In Audiences, Meta shows a Match score out of ten: matched rows divided by uploaded rows. Meta cannot exclude holdout customers it fails to match, so a low score means a leakier holdout. Meta says adding emails, phone numbers and device IDs can lift it.
• Both groups bought more than usual. A sale or a newsletter hit everyone. That is fine if it hit both groups alike, because the gap still stands. If an email went to one group only, you are now testing the email too.
• The audience will not deliver. Meta recommends that a custom audience includes at least 1,000 customers. A small test group may barely spend, which leaves the test little to measure.
• Customers went missing between exports. Someone who changed their email will not match across the two files. Expect a few strays, and check that both groups lost about the same share.

What to do this week

1. Size your list. In Shopify, build the past-year buyers segment and read the count above the editor. Pass: the test group, after the holdout leaves, still clears Meta's recommended 1,000 customers. Fail: it doesn't, so widen the date range or start with email.
2. Audit your ad sets for the exclusion control. In Ads Manager, open each live ad set that could reach past buyers. Look under Audience for Show more controls. Pass: every one offers a custom audience exclusion. Fail: one doesn't, so plan to pause it during the test.
3. Do a dry run with ten customers. Export the segment and add the random number and Group columns. Then upload the holdout rows as an Only exclusions list. Pass: the audience shows Ready under Status in Audiences. Fail: the upload errors, so fix the file before the real draw.

Check the homework. Your GA4 Attribution paths export already holds the evidence. Causality Engine reads that one file and shows what each channel caused next to what last-click gave it, in 1 to 2 minutes, for €99 once (excluding VAT), refundable within 30 days. Check the homework

Sources, 1 October 2026: Creating customer segments (Shopify Help Center); Shopify-based segment filters (Shopify Help Center); Importing and exporting customers (Shopify Help Center); RAND (Google Docs Editors Help); Keyboard shortcuts for Google Sheets (Google Docs Editors Help); COUNTIF (Google Docs Editors Help); Create a customer list custom audience (Meta Business Help Center); How to use custom or lookalike audiences (Meta Business Help Center).

Related answers

• All Incrementality questions
• What is a holdout group in marketing?
• What is a holdout group for a home goods store?
• Incrementality testing, explained
• Control group, in the glossary