UGC video and ROI: how to test creator content

Don't let the ad platform choose who sees each version. In a platform A/B test of a creator video against a brand video, the delivery system can show each ad to a different mix of people, so the winner may only be the ad that reached likelier buyers. Across 3,204 Lift tests and 181,890 A/B tests on Meta, Lift tests showed no meaningful audience imbalance and A/B tests showed clear imbalance (Burtch and colleagues, 2025). Assign the audience yourself, or use a lift test with a no-ad control, and put creator fees and usage rights in the cost.

Why doesn't a platform A/B test settle creator versus brand creative?

A platform A/B test compares users exposed to alternative ad configurations, with no control group, and ad delivery algorithms may target different variants to different audience segments (Burtch and colleagues, 2025). Braun and Schwartz, in the American Marketing Association's summary of their Journal of Marketing study, put the consequence plainly: the winning ad may have performed better simply because the algorithm showed it to users more prone to respond. Their advice splits by goal. To predict which creative performs best on the same platform with the same settings, carry on using the platform's A/B tools. To learn how creative works beyond that platform, don't rely on them for causal evidence (AMA, 2025). They also note that a platform's primary goal is to maximise ad performance, not to provide experimental results for marketers, so it has little incentive to help you untangle creative from targeting. Your Meta creative test is not an A/B test covers the mechanics.

Creator content adds a second problem: creator versus brand is not one variable. The creator's voice, the format, the length and the hook change together, so even a clean test compares bundles. That answers whether the bundle earns more. It says nothing about which element did it.

How do you test it without the platform choosing the audience?

Three designs, by the question each answers:

1. On your own site: does the content persuade people who are already there? Randomly show each visitor the creator video or the brand video on the product page, and compare add-to-cart rate and orders per visitor. You assigned the groups, so no algorithm sorted them. It tests the content, not the ad's delivery or cost.
2. In email: the same question, off the platform. Send each version to a random half of a list and compare clicks and orders per recipient.
3. On the platform: does each creative cause orders? Give each creative its own campaign with the same budget, objective and settings, and measure each against a no-ad control group in a lift test, in one study with two test cells if your tool offers that. A lift test compares users eligible to see the ads with a no-ad control group, and it showed no meaningful audience imbalance in that study (Burtch and colleagues, 2025). Compare incremental orders per euro spent, and keep the two audiences from overlapping where you can.

The platform lift test needs orders. Twenty-five large field experiments with major US retailers and brokerages found the median confidence interval on return on investment was over 100 percentage points wide (Lewis and Rao, 2015). Those were advertising experiments, but the lesson carries: with few orders, the gap between two creatives sits inside a wide interval. Size the test first with the holdout test planner. If you won't reach the sample, run the site or email test, which needs only your own traffic.

How do you count the ROI of a creator video?

Cost is more than ad spend. Add the creator's fee, the product you sent, editing and the usage rights. On TikTok, Spark Ads can run organic posts made by other creators with their authorization, and the duration of the authorization code is set per campaign (TikTok Business Help Center, updated June 2026), so note the expiry and what renewing costs. Return is incremental orders times contribution margin, taken from a lift test, not revenue the platform credits.

For illustration: a creator clip costing 600 plus ad spend of 2,000 costs 2,600, and a lift test that finds 30 extra orders at a contribution margin of 40 each returns 1,200, which is 0.46 of the cost. That is a loss, however the platform's ROAS looks.

What the result means:

• Pass: incremental contribution margin covers the total cost with the margin of safety you set, and the gap between the two creatives is larger than the noise at your order count.
• Fail: the creator version's incremental margin is below its total cost, even if its platform ROAS is higher.
• Undecided: too few orders to separate the two. Extend the test, or fall back on the site or email test.

Sources, 30 September 2026: Characterizing and Minimizing Divergent Delivery in Meta Advertising Experiments (Burtch, Moakler, Gordon, Zhang and Hill, arXiv, 2025); Can You Trust Your Ad Data? A New Study Exposes a Hidden Flaw in A/B Testing on Digital Ad Platforms (Braun and Schwartz, American Marketing Association, 7 January 2025); The Unfavorable Economics of Measuring the Returns to Advertising (Quarterly Journal of Economics, 2015); About Spark Ads (TikTok Business Help Center, updated June 2026). The arXiv paper is a preprint, not peer-reviewed; the AMA page is the study authors' own summary. No source read here compares creator-made with brand-made creative on incremental orders, so that comparison is yours to run.

Related answers

• Your Meta creative test is not an A/B test
• Vary one element, learn something reusable
• A/B testing: sample size first
• Holdout test, defined
• Incrementality testing