Skip to content

Average Treatment Effect (ATE)

The average treatment effect (ATE) is the average causal effect of a treatment over a whole population: the mean of each unit's outcome with treatment minus its outcome without. It describes the change to expect on average if the treatment were applied to everyone.

Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free

By , Founder & CEOUpdated 7 min read

What is Average Treatment Effect (ATE)?

Every unit has two potential outcomes, one with the treatment and one without, and the effect on a unit is their difference. Only one of the two is ever observed. This is the fundamental problem of causal inference, and no matter how large the dataset gets, half of the information needed for a unit's effect stays missing. Aggregate effects are means of the unit-level effects. The ATE is the most general of them: the mean over the whole population. It is a target, a causal parameter that a study tries to estimate.

Averaging does not solve the missing-data problem, so the ATE has to be estimated from a comparison. In a randomized experiment, assignment is independent of potential outcomes, the two groups have the same potential outcomes on average, and the difference in mean outcomes between treatment and control estimates the ATE. With observational data the ATE needs strong unconfoundedness, meaning independence of treatment from both potential outcomes given the covariates, and overlap: comparable treated and untreated units across the whole range of the covariates. Matching, inverse-probability weighting, regression adjustment and augmented inverse-propensity weighting apply those assumptions. They do not replace them.

An average can hide differences. A treatment that helps some units and harms others can have an ATE of zero, and the documentation of grf, an R package for generalized random forests, notes that the ATE can be zero even when effects differ across the population. The ATE also belongs to the population studied: Nick Huntington-Klein's textbook The Effect notes that an average for one group is only a wrong answer if it is taken to apply to everyone. And it assumes that one unit's treatment does not change another unit's outcome, the no-interference part of SUTVA.

For illustration, suppose a store sends an email to 4,000 subscribers chosen at random and holds out 1,000 others. Suppose mean spend in the following week is 5.50 euros per subscriber with the email and 5.

00 euros without it. Then the estimated ATE = 5.50 - 5.

00 = 0.50 euros per subscriber. A different random split would give a different estimate, so the figure needs an interval around it, and it describes subscribers, not all customers.

Why Average Treatment Effect (ATE) matters for ecommerce

A budget decision is a question about an average effect: how much extra revenue per customer does this channel or campaign cause? A holdout or randomized split estimates the ATE directly. Comparing exposed with unexposed customers does not, because exposure is tied to customers' own choices and to ad targeting. In the eBay paid-search experiment described in Scott Cunningham's online textbook Causal Inference: The Remix, the non-randomized estimate of the return was almost entirely selection bias, and the randomized estimate was close to zero. A 2019 Marketing Science paper on Facebook ad experiments states that unobservable factors make ad exposure endogenous, and it found that observational methods often failed to match the randomized results, even after conditioning on extensive demographic and behavioral variables. The ATE answers one specific question, what happens if everyone in the studied population is treated, and a campaign that reaches only some customers may need the ATT instead.

How to use Average Treatment Effect (ATE)

  1. Define the treatment, the outcome and the population: for example sending a discount email, orders per customer in a fixed window, and all newsletter subscribers. Define the treatment as what the store controls, the send rather than the open, because opening is the customer's choice. The ATE will describe that population only.
  2. Randomize before the treatment starts. Give each customer a random number in a spreadsheet, assign treatment or holdout by it, and record the assignment. Randomization makes assignment independent of what customers would have done.
  3. When the window closes, compute the mean outcome per customer in each group from the order data and subtract the holdout mean from the treatment mean. That is the estimated ATE per customer. Multiply by the number of customers in the population for a total.
  4. Put an interval around it. The difference between two group means has sampling error, and a different random split would give a different number. Estimation software such as grf returns a standard error with its ATE estimate.
  5. Check the split. Compare the groups on past orders and spend before the start. Large gaps point to a problem in how the groups were formed.
  6. Pass: the groups look alike before the start, and the interval is narrow enough to separate an effect that would change the decision from none. Fail: the groups differ before the start, customers chose their own group, or the interval is too wide. Then report the difference as an estimate with its limits, not as the ATE.

Formula

ATE = E[Y(1) - Y(0)] = E[Y(1)] - E[Y(0)]

Where:
- ATE: the average treatment effect
- Y(1): the outcome if the unit is treated
- Y(0): the outcome if the unit is not treated
- E[ ]: the mean over the whole population

In a randomized test, the mean outcome of the treatment group minus the mean outcome of the control group estimates the ATE.

Common mistakes

  1. Comparing customers who chose the treatment with those who did not and calling the gap the ATE. The gap carries selection bias: in the eBay experiment the non-randomized comparison implied a large return and the randomized one did not. The cost is spend justified by customers who would have bought anyway.
  2. Reading the ATE as the effect on every customer. It is a mean of unit effects, so opposite effects can cancel, and the grf reference notes the ATE can be zero while effects differ across the population. The cost is dropping a treatment that works for one segment, or rolling out one that harms another.
  3. Generalizing beyond the population studied. An experiment on email subscribers estimates the effect for subscribers, and an average for one group is a wrong answer only if it is taken to apply to everyone. The cost is a rollout forecast built on the wrong customers.
  4. Letting the groups contaminate each other. If holdout customers hear about a code from treated ones, or share a household with them, the holdout no longer shows the outcome without the treatment, and the no-interference assumption fails. The cost is an estimate that does not match the effect of treating versus not treating.
  5. Reading a regression coefficient on a treatment flag as the ATE. The grf guide lists what that reading relies on: treatment unconfounded given the controls, controls that act linearly, and a constant effect. The first is an identifying assumption to be argued from domain knowledge. The cost is a coefficient that mixes the effect with selection.

Frequently asked questions

  • What is the average treatment effect in simple terms?
    It is the average change in an outcome that a treatment causes across a whole population, such as the extra spend per customer from an email. It compares what each customer would do with the email against what they would do without it, then averages the difference. Only one of the two is ever observed, so it has to be estimated.
  • How do you calculate the ATE from a holdout test?
    Assign customers to treatment or holdout at random, measure the mean outcome per customer in each group over the same window, and subtract the holdout mean from the treatment mean. Randomization makes the groups comparable on average, so the difference estimates the ATE, with sampling error that calls for an interval.
  • What is the difference between ATE and ATT?
    The ATE averages the effect over everyone in the population, while the ATT averages it over only the units that actually received the treatment. They match in a large randomized test where everyone assigned is treated, and they differ when the treated are chosen by their own behavior or by targeting.
  • Can the ATE be estimated without a randomized experiment?
    Yes, but only under assumptions: unconfoundedness, meaning treatment is independent of potential outcomes given the covariates, and overlap across the covariates. Methods such as matching and weighting apply those assumptions but do not confirm them. A 2019 comparison of Facebook ad experiments found observational methods often failed to match randomized results.

Further reading

Your platforms guess.
We run the math.

Upload a GA4 export and see what each channel caused, next to last-click, in 1–2 minutes. The read is yours to keep.

Get your first finding free

The free finding runs in your browser. Your file never leaves it. The full read is €99, refundable within 30 days, no questions asked. Prices exclude VAT.