How DTC teams lose the plot on automation
Four patterns that recur when ecommerce teams hand catalogue and ad operations to an agent, and what each of them costs in measurement terms.
Three platforms, three answers, one budget to defend. Your GA4 export already holds the answer, and the first finding is free and never leaves your browser. Get your first finding free
By Joris van Huët, Founder & CEOUpdated 2 min read
The failure modes are consistent enough to name. Four of them, and none is about the agent being bad at its job.
One: judging the agent on throughput
Products touched per hour is the metric that presents itself, and it measures the wrong thing. An agent that changes more per hour while making each change less measurable is moving backwards.
The metric that matters is decisions supported per quarter, which is slower and less satisfying to watch.
Two: letting the agent choose its own scope
Asked to improve conversion, an agent will sensibly touch everything that plausibly affects conversion. That is good problem solving and terrible experimental design.
Scope is a decision the operator makes, not one to delegate. One lever, a named holdout, a fixed window.
Three: trusting platform reporting to grade the work
The ad platforms grade their own homework, and their numbers already overcount for reasons that predate agents entirely. Handing an agent a platform ROAS target means optimising against a metric with a known bias. The mechanics are in platform attribution overcounting.
Four: no rollback path
An agent that rewrote 250 listings without storing the previous versions has made a one-way change. If the read comes back negative there is nothing to return to, and the catalogue has to be rebuilt rather than reverted.
Storing the prior state costs almost nothing at the time and is impossible to arrange afterwards.
What good looks like
| Instead of | Do |
|---|---|
| Products changed per hour | Decisions the quarter's changes supported |
| "Improve conversion" | One lever, named holdout, fixed window |
| Platform ROAS as the target | A causal estimate |
| Fire and forget | Store prior state before the batch |
A causal read on a Google Analytics export returns what each channel caused next to last-click, a data-health score rather than a confidence interval, and a next step for every channel, for 99 euro once, refundable within 30 days, no questions asked.
Related answers
Frequently asked questions
What is the wrong way to measure an AI agent's value?
Throughput. Products touched per hour measures activity, and an agent that changes more per hour while making each change less measurable is going backwards. Decisions supported per quarter is the metric that matters.Should an agent decide its own scope?
No. Asked to improve conversion it will touch everything that plausibly affects conversion, which is good problem solving and poor experimental design. Scope stays with the operator: one lever, a named holdout, a fixed window.Why not give an agent a platform ROAS target?
Because platform reporting already overcounts for reasons that predate agents. Optimising against a metric with a known bias means the agent gets better at producing the bias.
Go deeper: Causal attribution, explained.
Sixty-second versions of these ideas: Causality Engine on YouTube Shorts.
Have an audience of ecommerce operators? Earn 20% recurring commission by recommending the €99 read.
Keep reading
Terms in this article
- AnalyticsAnalytics is the systematic computational analysis of data. It reveals customer behavior and measures campaign performance.
- AttributionAttribution identifies user actions that contribute to a desired outcome and assigns value to each. It reveals which marketing touchpoints drive conversions.
- ClickClick is the action a user takes to interact with a digital advertisement, redirecting them to a website or landing page. Clicks are a fundamental metric for measuring ad engagement and a primary input for click-based attribution models.
- Confidence IntervalConfidence Interval is a statistical range of values that likely contains the true value of a metric. In marketing analytics, it quantifies uncertainty around estimates, indicating the precision of an outcome or causal effect.
- ConversionConversion is a specific, desired action a user takes in response to a marketing message, such as a purchase or a sign-up.
- Google AnalyticsGoogle Analytics is a web analytics service that tracks and reports website traffic.