Skip to content

Average Treatment Effect on the Treated (ATT)

The average treatment effect on the treated (ATT) is the average causal effect of a treatment among the units that actually received it. It is the mean, over those units, of the outcome with the treatment minus the outcome without it.

Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free

By , Founder & CEOUpdated 8 min read

What is Average Treatment Effect on the Treated (ATT)?

Each unit has two potential outcomes: what happens with the treatment and what happens without it. The effect on one unit is the difference, and it cannot be observed, because only one of the two outcomes occurs. Aggregate effects are means of these unit-level differences, and they differ in whose differences are averaged. The ATT averages over the units in the treatment group and leaves out the control group. The average treatment effect (ATE) averages over everyone, and the average treatment effect on the untreated (ATU) averages over the units that did not receive the treatment. The ATE is a weighted average of the ATT and the ATU, weighted by the shares of treated and untreated units.

The ATT and the ATE differ when the treated respond differently from everyone else. Nick Huntington-Klein's textbook The Effect observes that people often choose for themselves whether to get treated, so treated and untreated people differ in many ways and the effect may differ too. Scott Cunningham's online textbook Causal Inference: The Remix lists a website that targets consumers with ads they think will benefit them among the processes that sort people into treatment by expected gain. Which average to estimate depends on the decision. The Effect suggests the ATE for a treatment applied to everyone or at random, the ATT for a policy that people opt into and that will continue, and the effect on the untreated as the likelier fit for an expansion to people not yet reached.

In a large randomized test where everyone assigned to the treatment receives it, the treated and untreated groups have the same distribution of effects, so the difference in group means estimates the ATT and the ATE alike. Without randomization the ATT needs less than the ATE. Cunningham's weak unconfoundedness asks for independence of treatment from only the outcome without treatment, given covariates, and weak common support asks for comparable control units only at the covariate values found among the treated.

For illustration, suppose four customers, A to D, would spend 12, 8, 4 and 2 extra euros in the next month if they saw a retargeting ad, compared with not seeing one. Suppose the ad was shown only to A and B, the two the platform judged most likely to buy. Then ATT = (12 + 8) / 2 = 10 euros, ATU = (4 + 2) / 2 = 3 euros, and ATE = (12 + 8 + 4 + 2) / 4 = 6.5 euros. So applying the ATT of 10 euros to an audience like C and D would overstate the effect, which is 3 euros for them.

Why Average Treatment Effect on the Treated (ATT) matters for ecommerce

When a campaign reaches only some customers, the ATT is the effect on the customers it reached. Multiplying it by the number of customers reached gives the extra spend the campaign caused among them, a figure that can be set against its cost. The risk is selection. People rarely choose at random, and customers an ad platform picks are picked for a reason, so a gap between exposed and unexposed customers is the ATT plus selection bias. Scott Cunningham's online textbook Causal Inference: The Remix recounts a field experiment by Blake, Nosko and Tadelis that shows how large that bias can be: eBay paid for ads on searches that already contained its brand name, and the book reads virtually all of the estimated return in the non-randomized comparison as selection, because the users who clicked were already far more likely to spend at eBay however they arrived. A 2019 Marketing Science paper on Facebook ad experiments likewise found that observational methods often failed to match randomized results, even with extensive controls.

How to use Average Treatment Effect on the Treated (ATT)

  1. Name the treated group and the question. Treated customers are the ones who actually received the email, saw the ad or joined the program, and the ATT speaks to them. If the plan is to widen the audience, the effect on the untreated is the closer target.
  2. Build a comparison group before the treatment starts: a random holdout from the same audience, kept out of the send or the campaign. Compare people as assigned. If some assigned customers are never exposed, that difference is the effect of assignment, called the intent-to-treat effect, and the effect on those exposed needs an adjustment such as an instrumental variable, which Cunningham notes estimates an effect for compliers only. He also notes that comparing exposed with unexposed individuals is biased even in a randomized trial when people do not comply.
  3. Compute the mean outcome per customer in each group over the same window, such as orders or revenue per customer from Shopify order data, and subtract the comparison mean from the treated mean. Multiply by the number of treated customers for the total extra spend among them.
  4. Check the comparison. Compare the groups on past orders and spend before the start. For a regional campaign without randomization, a difference-in-differences comparison looks at changes over time in treated and comparison regions, and it relies on an assumption that cannot be tested, so pre-period trends give only indirect support.
  5. Pass: assignment was random, or the groups moved together before the start, and the interval around the difference is narrow enough to act on. Fail: customers were sorted into treatment by their own clicks or by a targeting algorithm, or the groups differed before the start. Then report the gap as a raw difference, not as the ATT.

Formula

ATT = E[Y(1) - Y(0) | D = 1]
ATE = p x ATT + (1 - p) x ATU

Where:
- ATT: the average treatment effect on the treated
- ATU: the average treatment effect on the untreated
- ATE: the average treatment effect over everyone
- p: the share of units that were treated
- Y(1): the outcome if the unit is treated
- Y(0): the outcome if the unit is not treated
- D = 1: the unit actually received the treatment

The ATT averages over treated units only, and the outcome without treatment is never observed for them, so it has to be estimated from a comparison group.

Common mistakes

  1. Comparing clickers with non-clickers, or loyalty members with non-members, and calling the gap the ATT. The raw gap is the ATT plus selection bias, and in the eBay experiment Cunningham discusses, virtually all of the estimated return was selection. The cost is budget moved toward customers who would have bought anyway.
  2. Applying the ATT to a new audience. It describes the customers who were treated, and the effect on customers not yet reached can differ, as the untreated customers' effect does in the illustration above. The cost is a forecast for the expansion that does not hold.
  3. Treating the ATT as the effect on every treated customer. It is an average. Individual effects differ and are never observed, so some treated customers may have been unaffected. The cost is a claim about each customer that the data cannot support.
  4. Ignoring spillovers. If treated customers pass a code or a recommendation to comparison customers, or share a household with them, the comparison group no longer shows what would have happened without the treatment. Cunningham calls the assumption that one person's treatment status cannot affect another person's outcome SUTVA. The cost is an estimate that no longer matches the effect of treating versus not treating.
  5. Reading a difference-in-differences estimate as an effect for all customers. The design speaks only to the treated group after treatment starts, and its parallel-trends assumption can be supported with pre-period checks but not tested. The cost is a forecast for customers the design never observed.

Frequently asked questions

  • What does ATT mean in simple terms?
    The ATT is the average effect of a treatment on the people who actually got it, such as the customers who received an email. It compares what happened to them with what would have happened to the same customers without the email, which is never observed and has to be estimated from a comparison group.
  • What is the difference between ATT and ATE?
    The ATE averages the treatment effect over everyone in the population, and the ATT averages it over only the units that were treated. They match when the treated respond like everyone else, as in a large randomized test, and differ when the people who get treated are different from those who do not.
  • Should I measure the ATT or the ATE for a marketing campaign?
    It depends on the decision. The ATT fits a campaign that will keep reaching the same kind of customers. The ATE fits a treatment applied to everyone or at random, and the effect on the untreated is closer to the question when expanding to customers not yet reached. Huntington-Klein's textbook The Effect gives this rule of thumb.
  • Can the ATT be estimated without a randomized test?
    Yes, under assumptions about how treatment was assigned. The ATT needs weaker conditions than the ATE: independence of treatment from the outcome without treatment given covariates, and comparable control units for the treated units' covariates. Difference-in-differences also targets the treated group. The independence and parallel-trends assumptions cannot be tested directly, only supported indirectly, for example with checks on the period before treatment.

Further reading

Your platforms guess.
We run the math.

Upload a GA4 export and see what each channel caused, next to last-click, in 1–2 minutes. The read is yours to keep.

Get your first finding free

The free finding runs in your browser. Your file never leaves it. The full read is €99, refundable within 30 days, no questions asked. Prices exclude VAT.