The Future of AI in Marketing Analytics: Assistants, Not Analysts

Assistants draft and explain; a person, or a method you can check, owns any number that moves budget. That is an opinion about roles, not a forecast: nothing here predicts adoption or which tools win. The evidence is narrower.

What do assistants do well?

Defined requests, with an error rate that rises as the work gets harder. Evidence (preprint published by Tencent authors, not independently audited; February 2026, revised June 2026): AD-Bench is built from "real user marketing-analysis requests on a production advertising platform", 225 of them in its final set. An LLM judge grades the answers, and that judge is also one of the models tested, a limitation the authors list. They report that the best model "attains Pass@1 = 76.9% and Pass@3 = 80.4% with 82.7% trajectory coverage overall, yet drops sharply on L3 to Pass@1 = 61.4% and Pass@3 = 65.1%" (Hu and co-authors, arXiv). Pass@1 is the share answered correctly on the first attempt. L3, 83 of the 225, asks the agent to "synthesize multiple conditions and make a judgment".

Evidence (NBER working paper, April 2023, revised November 2023): researchers studying the staggered introduction of a generative AI-based conversational assistant, with data from 5,179 customer support agents, report that "access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers" (Brynjolfsson, Li and Raymond). That is customer support, not analytics.

Where do they need an owner?

Evidence (preprint, June 2026): on an immigration and social-policy analysis, 20 independent runs of each of two coding agents produced effect estimates that "remain broadly aligned with the human consensus".

Evidence (preprint, April 2026): on 11 real-world datasets, "in 6 of the datasets an affirmative conclusion is not well-supported, even though a single ADS run may support one", where ADS means agentic data science. The authors add that "ADS self-reported confidence is poorly calibrated to the empirical stability of its conclusions" (Rewolinski and co-authors, arXiv).

A single report can look sound while a second run disagrees, which is why the check below runs each question twice.

None of these three studies used marketing exports, and each used the models available at its test date.

Who should own what?

Opinion, a split that fits that evidence:

• Assistant: drafting queries, explaining a finished result, summarizing a report, first-pass cleaning you can compare with the original.
• A person, or a method you can check: the question and its decision rule, the assumptions, the verdict, and any number that moves budget.
• Nobody, until checked: a number with no query or calculation you can read.

How do you find out which questions your assistant can own?

No source cited here measures an assistant on your own data. OpenAI's guide recommends "Building tests and evaluation suites that measure prompt behavior"; this is a small suite for your own questions.

1. Write twenty questions you can answer yourself from Shopify and GA4: orders last month, revenue by country, refund rate by month, new versus returning buyers, revenue by default channel group. Keep the answers sealed.
2. Ask each in a fresh chat and require the query or calculation behind the answer.
3. Score it. Pass: the number matches and the query reads correctly. Fail: a number differs, or there is nothing to read. Note which types of question fail.
4. Perturb the passes. Ask each again in different words, and once about a period with no data. Pass: the same number, and "no data" instead of a guess. The Rewolinski preprint proposes perturbation checks of this kind.
5. Delegate only the types that passed, and re-run the check after any model or prompt change.

Where does chat fit in a causal read?

Sources, 30 September 2026: AD-Bench (Hu et al., arXiv, 2026); AI Coding Agents in Social Science (Alizadeh et al., arXiv, 2026); Sanity Checks for Agentic Data Science (Rewolinski et al., arXiv, 2026); The Agentic Garden of Forking Paths (Miao et al., arXiv, 2026); Generative AI at Work (Brynjolfsson, Li and Raymond, NBER, 2023); OpenAI text generation guide (OpenAI, 2026).

Related answers

• Where LLMs help in marketing
• Data analyst or LLM: who owns attribution?
• Natural language to SQL for marketing
• Causal attribution
• Attribution audit