Judging an attribution vendor on transparency
Nine questions that separate a documented method from a black box, and what a straight answer to each one sounds like when a vendor actually gives you one.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 3 min read
A method you cannot inspect is a method you cannot defend, and you are the one who will have to defend it. Transparency is not a nice-to-have in attribution. It is the thing being bought.
The nine questions
Ask them in a demo. The answers are more informative than any feature list.
- What is the counterfactual? Every incrementality number is measured against a world that did not happen. A vendor who cannot describe theirs in a sentence has not defined the quantity they are selling.
- Where does the interval come from?
- What happens to a channel with no variation in the window? The correct answer is that it cannot be resolved.
- What share of revenue does a typical read account for?
- Can I reproduce a result from the inputs? If not, nobody can check it, including them.
- What does the model do with a channel that started mid window?
- How is the attribution window handled when platforms disagree?
- What is the documented failure mode? Every method has one. A vendor who names theirs is describing a method. A vendor who does not is describing a product.
- Would a different analyst using your method on my data get the same answer?
Which attribution vendors explain their methodology openly instead of a black box AI model?
You can check this without asking anyone. The answer is public or it is not.
| Signal | What it means |
|---|---|
| A public methodology page naming the method | The method can be argued with |
| Named assumptions with the conditions they fail under | Somebody thought about being wrong |
| Intervals shown by default, not on request | The uncertainty is part of the product |
| "AI model" as the whole answer | The claim cannot be checked |
| Accuracy percentages with no stated baseline | A number chosen for the sentence it appears in |
The last row is worth dwelling on. An accuracy figure is meaningless without the thing it was measured against, and in attribution the ground truth is usually unavailable, which is why the field exists at all.
More on reading these documents in what an attribution methodology doc reveals.
- It reports what it could not resolve rather than filling the gap.
- It gives you the uncertainty, so a marginal call can be recognised as marginal.
A tool that always returns a confident number for every channel will eventually return a confident number for a channel it had no information about, and you will not be able to tell which time that was.
Where a read fits
Causality Engine reads one file, the Attribution paths export from GA4, and shows what each channel caused next to what last-click gave it.
It is 99 euro once, refundable within 30 days, no questions asked.
Related answers
Frequently asked questions
How do I evaluate marketing attribution solutions for transparency and defensibility?
Ask what the counterfactual is, where the interval comes from, what happens to a channel with no variation, what share of revenue a read accounts for, whether a result is reproducible from the inputs, and what the documented failure mode is. A method with no stated failure mode has not been described.Which attribution vendors explain their methodology openly instead of using a black box model?
Check for a public methodology page that names the method, lists its assumptions and the conditions under which they fail, and shows confidence intervals by default. Proprietary AI as the entire explanation, or an accuracy percentage with no stated baseline, are the two signals that there is nothing to inspect.
Go deeper: Incrementality testing, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Attribution SoftwareAttribution Software measures campaign impact by tracking customer interactions across touchpoints. It assigns value to each channel, showing what drives conversions.
- Attribution WindowAttribution Window is the defined period after a user interacts with a marketing touchpoint, during which a conversion can be credited to that ad. It sets the timeframe for assigning conversion credit.
- Causal AttributionCausal Attribution uses causal inference to determine which marketing touchpoints genuinely cause conversions, not just correlate with them.
- CausalityCausality is the relationship where one event directly causes another, essential for identifying specific actions that drive desired outcomes in marketing.
- CorrelationCorrelation is a statistical measure showing a relationship between variables; it does not imply causation.
- CounterfactualA counterfactual is what would have happened under conditions that did not occur, such as a customer's outcome had they not received the campaign they did receive. It is never observed, so every effect estimate has to stand in for it with a control group, comparable units or a forecast.
- IncrementalityIncrementality is the part of an outcome, such as orders or revenue, that a marketing activity caused: the extra results compared with what would have happened without it. It is a causal effect, not the number of conversions credited to a channel.