Why real-time attribution alerts mislead
A causal estimate is not a live metric. Alerting on it in real time reports noise as news, and the team that acts on those alerts is acting on nothing.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 3 min read
A causal estimate is not a live metric, and alerting on it in real time reports noise as news.
What a live number and a causal estimate are
| A live operational metric | A causal estimate | |
|---|---|---|
| What it is | A count of something that happened | An inference about something that did not |
| How fast it settles | Immediately | Over a window long enough to separate signal |
| What a change means | The thing changed | Possibly the thing changed, possibly the window moved |
| Worth an alert? | Often | Almost never |
Spend is a live metric. Orders are a live metric. Site errors are a live metric. Alert on all of them. An estimate of what your channels caused is a different object, produced by comparing periods, and it moves as the comparison window slides regardless of whether anything in your marketing changed.
The specific failure
Set an alert on a causal estimate and it fires. It fires because the estimate has an interval around it, and any point inside that interval is consistent with the data. Crossing a threshold inside your own uncertainty is not an event.
The team then investigates, finds nothing, and learns that the alert means nothing.
What is worth alerting on
The ingestion or export job failed. Delivery is late. Coverage, the share of your orders the read explains, has dropped below a floor you set, which usually means something broke in collection.
Each of those describes a condition a human must fix, which is what an alert is for.
What "budget optimisation alerts" usually means
Two different products travel under that name. One watches spend and pacing, which is a live metric and a reasonable thing to automate. The other claims to watch attributed performance and tell you to move budget, which is the version this article is about.
If you are evaluating the second kind, ask what the underlying estimate's interval is and whether the alert accounts for it. An alert that fires on movement within the interval is firing on nothing, and the vendor should be able to say how it distinguishes the two.
What we do instead
The alternative to alerting is a cadence, and the case for it is in a weekly budget review instead of alerts.
The principle
Speed is a virtue in operational monitoring and a vice in causal inference. A method that gives you an answer faster than the underlying comparison can resolve is not being fast, it is being confident early.
Related answers
Frequently asked questions
Does Causality Engine have real-time alerts?
A causal estimate needs a window long enough for the comparison to resolve, so alerting on it in real time reports movement within its own uncertainty as though it were an event.What are budget optimisation alerts really watching?
Usually one of two things. Spend and pacing, which are live metrics and reasonable to automate, or attributed performance, which is an inference. If a vendor offers the second, ask how the alert accounts for the estimate's confidence interval.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Causal InferenceCausal inference is the set of methods for estimating how much an outcome changes because of a cause, rather than alongside it. It works by comparing what happened with an estimate of what would have happened without the cause.