Opinion: Analytics / Measurement

Marketing Attribution Is Astrology for CMOs.

Dashboards give false confidence. Incrementality testing and holdout groups are the only honest measurement left.

Attribution Analytics Opinion

I don’t say this lightly, because I’ve built plenty of attribution dashboards myself and presented plenty of them with a straight face. But I’ve come around to a fairly harsh view: most multi-touch attribution reporting functions less like measurement and more like astrology. It takes a genuinely chaotic, hard-to-observe system, the actual reasons a human decided to buy something, and produces a clean, confident-looking number anyway, because a confident-looking number is what the meeting needs.

Nobody’s lying on purpose. The model is doing exactly what it was built to do. It’s just that what it was built to do was never actually possible to do accurately, and we’ve collectively agreed not to say that part out loud in the boardroom.

Why This Was Always Shaky

Attribution models assign credit. They don’t measure causation.

Multi-touch attribution takes the touchpoints it can see, an ad click, an email open, a landing page visit, and distributes conversion credit across them using a rule someone chose, linear, time-decay, position-based, whichever fits the story you want to tell that quarter. What it fundamentally cannot do is answer the actual question that matters: would this person have converted anyway, without that specific touchpoint? That’s a causal question. Attribution models answer a correlational one and let the confident dashboard formatting imply otherwise.

Add AI-driven discovery into the mix, where a meaningful chunk of influence now happens in completely untracked private conversations, and the model isn’t just imprecise anymore. It’s often working from a dataset with entire categories of real influence missing from it, while still outputting numbers with the same confident level of precision.

The Honest Alternative

Incrementality testing tells you something true, even when it’s inconvenient

Holdout groups and geo-based incrementality tests answer the actual question attribution can’t: does this specific spend cause more conversions than not spending it at all? It’s a much less flattering process. Sometimes it tells you a channel you love isn’t actually adding much beyond what would’ve happened organically. That’s an uncomfortable meeting. It’s also the only version of that meeting based on something real instead of a model quietly built to always have a tidy answer available.

I’d rather walk into a budget conversation with a slightly awkward, genuinely true incrementality result than a beautifully formatted attribution report that’s confidently describing a story nobody can actually verify happened.

What To Actually Do

Shift budget and belief toward testing, not modeling

  • Run genuine holdout or geo-lift tests on your top two or three channels at least once or twice a year, not as a one-off experiment but as a recurring practice.
  • Treat attribution dashboards as a rough directional signal, not a source of truth for budget decisions.
  • Get comfortable presenting incrementality results even when they contradict a channel’s reported attribution performance. That contradiction is often where the real insight is.
  • Stop asking “which channel gets the credit” and start asking “what happens if we stop spending here for a month.” The second question is harder and much more honest.
Quick Answers

A few direct questions, answered directly

Why is multi-touch attribution unreliable?
Because it assigns conversion credit based on chosen rules rather than proven causation, and increasingly misses untracked influence such as AI-driven discovery, producing confident-looking but often inaccurate results.

What is incrementality testing in marketing?
A measurement method, often using holdout groups or geographic testing, that determines whether a specific marketing spend actually causes additional conversions beyond what would have happened anyway.

Should marketers stop using attribution models entirely?
Not entirely, they still offer useful directional signal. The recommendation is to treat them as one input alongside incrementality testing, rather than as the primary source of truth for budget decisions.

Want to know what your marketing spend is actually causing, not just touching?

Let’s build a measurement approach you can actually trust.

See Advisory Options