Is the Future of Attribution Conversational or Causal?
A chat box is an interface, and the number it reads out has to be an estimate of what each channel caused. What published benchmarks show about language models and causation, and checks for your own export.
By Joris van Huët, Founder & CEOUpdated 4 min read
Both, for different jobs: conversation is how you ask, and the answer still has to be an estimate of what each channel caused. Neither asked a model what your channels caused.
What do the published benchmarks measure?
Evidence (two peer-reviewed papers, both first posted in 2023):
- Jin and co-authors (ICLR 2024): on a dataset of more than 200K samples, each a set of correlational statements with a causal question, seventeen LLMs "achieve almost close to random performance on the task".
Both test causal reasoning about described variables. Neither tests an orders export or a channel report.
What has been tested on attribution itself?
Evidence (conference paper, August 2026, Nanjing University and Alibaba authors): a short paper accepted at CIKM 2026 asks whether LLMs can identify "meaningful touchpoints" in conversion attribution. It reports that "LLMs effectively uncover a substantial portion of implicitly-related touchpoints" and that "significant room for improvement remains in their selection performance" (Wu and co-authors, arXiv). That is touchpoint selection for recommendation and advertising, not an estimate of what a channel caused.
Evidence (preprint, July 2026): the CausalDS benchmark treats "recognizing when a question admits no warranted answer and abstaining" as "a first-class scored outcome". It is a benchmark design, with no result in its abstract.
No source cited here measures a language model answering "what did each channel cause?" on a store's own export.
What must a causal answer contain?
Opinion: what each channel caused means orders that would not have happened without it, so the answer has to say where that comparison came from: a holdout, a geo test or stated assumptions. It also has to say when the export cannot answer.
Evidence (2026 papers): two anchor their estimates on experiments. A June 2026 paper by TikTok authors, accepted at ADKDD 2026, describes TikTok's own system, deployed across multiple global TikTok markets, and its correction "uses incrementality experiments as causal anchors". An April 2026 NBER working paper by Kellogg and Meta authors, on Meta ad experiments, says randomized trials "provide the most credible estimates of advertising incrementality but are difficult to scale".
What can you check today?
Three checks for any tool or assistant:
- Ask what it cannot know. Ask what a channel caused that has no spend or sessions in the export, or what a region caused that has no orders. Pass: it reports no data, or zero with a reason. Fail: it returns a share of revenue.
- Screen it with a pause you already have. Find a channel that was off for a known stretch, such as a paused account or a holiday shutdown, and skip pauses that coincided with a sale, an outage or a stock-out. Ask what the channel caused in the weeks before, during and after, then set that beside Shopify's total orders for the same weeks. Pass: the credit falls when the channel goes off, allowing for the lag between a click and an order, and total orders move the way the credit implies. Fail: the credit holds steady while spend is zero.
- Ask what it assumed. Pass: it names a design or an assumption behind the split. Fail: a confident split with neither.
A pause is a rough screen, not a test of cause: seasonality and other channels move at the same time. Confirm the channels that matter with a holdout. A chat interface is a good way to run all three, because it can draft the queries and explain the results.
Where does a causal read fit?
Sources, 30 September 2026: Causal Reasoning and Large Language Models (Kiciman et al., TMLR, 2024; arXiv version); Can Large Language Models Infer Causation from Correlation? (Jin et al., ICLR, 2024); Can Large Language Models Identify Meaningful Touchpoints in Conversion Attribution? (Wu et al., arXiv, 2026); CausalDS (Leban and co-author, arXiv, 2026); Attributed, But Not Incremental (arXiv, 2026); Predicted Incrementality by Experimentation (PIE) for Ad Measurement (Gordon et al., NBER, 2026).
Related answers
Frequently asked questions
Can an LLM do causal inference on my marketing data?
It depends on the task and the model.Will chat replace attribution dashboards?
That is a question about interfaces, and no source cited here measures adoption. Whatever the interface, the number behind it should come from a method you can check, such as a holdout.What does a causal attribution answer look like?
It says what each channel caused, meaning orders that would not have happened without it, and where that comparison came from: a holdout, a geo test or stated assumptions. It also says when the export cannot answer.
Go deeper: Incrementality testing, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- Causal AttributionCausal Attribution uses causal inference to determine which marketing touchpoints genuinely cause conversions, not just correlate with them.
- Causal ModelA Causal Model is a mathematical representation describing the causal relationships between variables, used to reason about and estimate intervention effects.
- CorrelationCorrelation is a statistical measure showing a relationship between variables; it does not imply causation.
- CounterfactualA counterfactual is what would have happened under conditions that did not occur, such as a customer's outcome had they not received the campaign they did receive. It is never observed, so every effect estimate has to stand in for it with a control group, comparable units or a forecast.
- ExperimentationExperimentation in marketing conducts controlled tests to determine the causal impact of specific actions. This includes A/B testing and other controlled experiments to establish causality.
- ExperimentsExperiments are scientific procedures that test hypotheses or demonstrate facts. In marketing, experiments like A/B tests determine the causal effect of campaign changes, enabling data-driven decisions.
- IncrementalityIncrementality is the part of an outcome, such as orders or revenue, that a marketing activity caused: the extra results compared with what would have happened without it. It is a causal effect, not the number of conversions credited to a channel.