GA4 or an LLM for Attribution: Which One Can You Audit?

Neither is fully open, and they are closed in different ways. Google's help pages name three attribution models for GA4's reports and let you switch between them, but say data-driven conversions can be reattributed for up to 7 days after the conversion. The test for both is the same: can you re-derive the number from rows you hold?

What does GA4 document about its attribution?

Google's attribution help page (official documentation, not independently audited) lists three models in GA4's Attribution reports: data-driven, paid and organic last click, and Google paid channels last click. Last click is a rule: it ignores direct traffic and gives all the value to the last non-direct channel, so you can rebuild it from paths. The data-driven model "uses machine learning algorithms to evaluate both converting and non-converting paths". The page describes the method in words, with a high-level illustration, but gives no formula you can rerun on your own paths. It adds that, depending on data availability, the model may use aggregate data from your data sharing settings. Google's data-driven model is trained and reported by Google. No source cited here tested how closely its credit matches a holdout.

What does a chat answer leave you with?

Chen, Zaharia and Zou (arXiv preprint, 2023) evaluated the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on seven tasks and concluded that "the behavior of the 'same' LLM service can change substantially in a relatively short amount of time".

Providers also retire models. One provider's documentation (Anthropic's model deprecations page, official documentation) says it "regularly retires older ones" and that "Requests to retired models will fail". So record the model, the prompt, the file and the date with every LLM answer you rely on.

How do you audit either one?

Re-derive one number, then reconcile the total:

1. Pick a closed period. Use dates older than a week, so GA4 is no longer reattributing them.
2. Compare the models. In GA4, open Advertising, then Attribution, then Attribution models. Note one channel's key events under data-driven and under paid and organic last click. Google says the report lets you "compare key event metrics for different attribution models side-by-side". The gap is how much of that number is the model's doing.
3. Rebuild last click. Open Key event attribution paths and export the table. In a spreadsheet, credit each path to its last non-Direct channel (a path made only of Direct visits stays with Direct), add up by channel, and compare with the last-click figure from the model comparison.
4. Reconcile the total. Add up purchases across channels and compare with Shopify orders for the same dates and time zone.
5. Put the LLM through the same test. Ask for the query or formula behind its number, run it yourself, then ask again in a fresh chat. Record the model, the date and the prompt.

Pass: you reproduce the number, or you can name why it differs (refunds, time zone, paths longer than the report's twenty-touchpoint limit). Fail: a number with no route back to rows. The data-driven figure will not rebuild, because Google describes it as a machine-learned model, not a rule. For that one, compare it across models and settle any budget decision with a holdout. An LLM is still useful here for explaining a finished table and drafting the query you then run yourself.

Sources, 30 September 2026: Get started with attribution, Key event attribution models report, Key events attribution paths report and Select attribution settings (Google Analytics Help, official documentation); How is ChatGPT's behavior changing over time? (Chen, Zaharia and Zou, arXiv preprint, 2023); Model deprecations (Anthropic, official documentation).

Related answers

• Audit trails for LLM attribution answers
• Same question, different answer: LLM reproducibility
• Data-driven attribution
• Why attribution dashboards disagree
• Causal attribution for ecommerce brands