Uplift Modeling
Uplift modeling is a predictive technique that estimates how much a marketing action changes an individual customer's behavior, such as the chance of buying, compared with no action. It is used to decide whom to contact and whom to leave alone.
By Joris van Huët, Founder & CEOUpdated 5 min read
Run the numbers for your store: the free holdout test planner.
What is Uplift Modeling?
Uplift modeling, also called incremental or net modeling, predicts the difference an action makes: for customers with given features, how much more or less likely they are to buy if contacted than if left alone. A conventional response model is trained on contacted customers only and ranks them by likelihood to respond; an uplift model uses contacted and control customers together and focuses on the incremental response.
Four types of customer are usually described. Persuadables respond only because they were targeted. Sure things would respond either way, lost causes respond under neither condition, and do-not-disturbs (sleeping dogs) are less likely to respond because they were targeted. Only persuadables give incremental responses, and a response model often targets sure things because it cannot tell them apart. You never see the same customer both contacted and not contacted, so no one's type is observed. A model estimates the net effect of the action for customers with similar features, not each person's type.
The data come from an experiment in which customers are randomly assigned to receive the action or be held out. A campaign's uplift is the difference in response rate between the two groups; uplift modeling estimates that difference as a function of customer features. Gutierrez and Gérardy describe three families of method: two models (one fitted to contacted customers, one to controls, with uplift as the difference of their predictions), a class transformation, and modeling uplift directly.
For illustration, suppose a store emails a discount to 10,000 customers chosen at random and holds back another 10,000. Suppose 310 emailed and 250 held-back customers buy: the response rates are 3.1% and 2.
5%, an average uplift of 0.6 percentage points. Suppose 2,000 customers per group are first-time buyers, and 120 of those emailed and 60 of those held back buy: their uplift is 6.
0% - 3.0% = 3.0 percentage points, while the other 8,000 per group show zero, with 190 buyers each.
The average hides that the email moved only first-time buyers.
Why Uplift Modeling matters for ecommerce
A response model ranks customers by how likely they are to buy, and Wikipedia notes it often ends up targeting sure things, who would have bought anyway. Without a control group, you cannot tell which orders from contacted customers the message caused, and a discount sent to someone who was going to buy anyway is margin given away. Uplift modeling is the step from who is likely to buy to who buys because of the message. It can also flag customers a message pushes away; Wikipedia describes retention campaigns that trigger cancellations. It applies where you decide who is contacted and can hold some customers out at random, such as an email or SMS list.
How to use Uplift Modeling
- Pick one action you control per customer, such as a discount email, and one outcome you can see in Shopify, such as an order within a window fixed before the send.
- Assign a representative part of your customer base at random to receive it or be held out, for example with a random number column in a spreadsheet, and keep the list.
- After the window, join the list to Shopify orders and compare response rates. The gap is the average uplift. Pass: a gap clearly larger than chance alone would produce. Fail: a gap near zero. That means no average effect, not that no group responds; step 4 looks for groups.
- To look for who responds, split customers at random into a build half and a check half. Estimate uplift by features known before the send, such as past orders, on the build half, and confirm the gaps on the check half.
- If you fit a model, score it on the check half, for example with an uplift curve. Pass: customers ranked highest show more uplift than customers ranked lowest. Fail: the two groups show about the same uplift. Keep a random holdout when you roll out.
Formula
uplift(x) = P(Y = 1 | T = 1, X = x) - P(Y = 1 | T = 0, X = x) Where: - Y: 1 if a purchase happens, 0 if not - T: 1 if the message was sent, 0 if held out - X = x: customer features, such as number of past orders
With random assignment, this difference is the average effect of the message on customers with features x. The average uplift of a whole campaign is the same difference without conditioning on x.
Common mistakes
- Training a response model on contacted customers only and calling it uplift. It ranks by likelihood to buy, so it often picks sure things.
- Skipping the random control. Without it, the gap between contacted and uncontacted customers is the effect only if assignment is independent of the outcomes given the features.
- Scoring a model by purchase-prediction accuracy. No individual's uplift is observed, so uplift models need their own measures, such as the area under the uplift curve.
- Treating the four types as facts. They describe customers but are never observed; a model estimates a net effect for customers with similar features.
- Trusting a two-model difference unchecked. A weak uplift signal can be lost because both models focus on predicting the original response, so check against the control group on held-out data.
Frequently asked questions
What is the difference between uplift modeling and a response model?
A response model is trained on contacted customers and predicts who will respond. An uplift model also uses control customers and predicts who will respond because of the action.Does uplift modeling need a randomized control group?
In practice, yes. Wikipedia says uplift modeling uses a randomized control. Without random assignment, the estimate is the effect only if assignment is independent of the outcomes given the features, an assumption you have to defend.What is the difference between lift and uplift?
A campaign's uplift or lift is the difference in response rate between a treated group and a randomized control group. Wikipedia notes that many marketers call that difference lift, so uplift modeling can be read as improving lift through predictive modeling.Can uplift modeling tell me which customers are persuadable?
Not with certainty. You never see the same customer both contacted and not contacted, so each customer's type cannot be observed. A model estimates the net effect of the action for customers with similar features, not each person's type.
Further reading
- Uplift modelling · Wikipedia
- Causal Inference and Uplift Modelling: A Review of the Literature · Proceedings of Machine Learning Research (Gutierrez and Gérardy)
- Using control groups to target on predicted lift: Building and assessing uplift model · Direct Marketing Analytics Journal (Radcliffe, via University of Edinburgh)
- Types of customers · scikit-uplift documentation