AI Has an Attribution Problem. Marketing Has Seen This Movie Before.

I've spent a good part of my career working with marketing measurement, and there's one frustrating question that never really goes away:

Did marketing actually cause the result?

I've worked with attribution, marketing ROI and marketing mix modelling. The methods have become more sophisticated, but the fundamental problem remains.

A customer sees an ad and later buys.

Did the ad cause the purchase?

Maybe.

Or maybe they were already planning to buy. Maybe they saw three other campaigns. Maybe a salesperson spoke to them. Maybe there was a promotion. Maybe demand simply went up.

Recently, while looking at how companies are measuring ROI from AI, I realised we're dealing with a surprisingly similar problem.

An AI tool is introduced. Productivity improves. And we give AI the credit.

But should we?

This feels a lot like last-click attribution

For a long time, marketing had a wonderfully convenient way of assigning credit.

A customer clicked a search ad and then converted? Search gets the conversion.

Simple.

The problem, of course, was that the customer might have encountered the brand through several other channels before that final click.

Last-click told us what happened immediately before the conversion. It didn't necessarily tell us what caused it.

I think we're at risk of doing something similar with AI.

Imagine a company introduces an AI sales assistant in January. Six months later, sales productivity is up 15%.

It's very tempting to put that on a slide.

But what else happened during those six months?

Maybe the sales process changed. Maybe incentives changed. Maybe demand improved. Maybe territories were reorganised. Maybe another technology was introduced.

What would have happened without AI?

This is where another familiar measurement concept becomes useful: the counterfactual.

Essentially:

What would have happened if we hadn't introduced the AI initiative?

We can't observe that alternate reality directly, which is what makes measurement interesting, and occasionally frustrating.

In marketing, we've tried to get closer to the answer using experiments, control groups, incrementality testing, attribution models and marketing mix modelling.

Different methods, different limitations. But they're all trying, in one way or another, to separate what happened from what happened because of the intervention.

I think AI ROI needs the same thinking.

Suppose an AI customer-service tool is introduced and average handling time falls from 12 minutes to 9.

At first glance, AI produced a 25% improvement.

But imagine handling time had already been declining and, without AI, would probably have reached 10 minutes anyway.

Should AI receive credit for three minutes or one?

That difference will eventually flow straight into the ROI calculation.

Hours saved aren't automatically euros saved

There's another AI ROI calculation I've been thinking about.

Suppose employees report that an AI tool saves 10,000 hours annually. Their average employment cost is €50/hour.

It looks reasonable.

But where exactly is that €500,000?

If the company avoided additional hiring because of the capacity created, there may be measurable cost avoidance. If employees used those hours to generate additional output or revenue, there's potentially measurable economic value. If overtime decreased, there's a direct saving.

But if people simply completed their work faster, the organisation has certainly created capacity, and that may be valuable, but it hasn't necessarily created €500,000 of financial return.

Marketing has a similar chain.

Leads aren't pipeline. Pipeline isn't revenue. Revenue influenced isn't necessarily revenue caused.

There are steps between activity and financial value. AI needs those steps too.

I'm not suggesting we need an experiment for everything

That would be unrealistic.

Sometimes we'll have a control group or a good before-and-after comparison. Sometimes we'll have strong system data. And sometimes the best evidence available will genuinely be employees telling us how much time they're saving.

That's still useful information.

But I don't think we should treat all those estimates as equally reliable.

If you only look at ROI, Initiative B wins.

But if you had another €1 million to invest, would you automatically put it there?

I wouldn't. I'd want to understand the evidence behind the number.

Perhaps Initiative B is genuinely the better investment. Or perhaps its first priority should be better validation.

That's why I think AI ROI needs something we don't normally put next to an ROI percentage: a measure of confidence.

Not to make the calculation more complicated for the sake of it. Just to acknowledge that 320% based on an estimate isn't quite the same thing as 320% supported by observed outcomes.

This gets much more important at portfolio level

With three AI pilots, none of this is particularly difficult to keep track of.

With 100 AI initiatives, it becomes a different problem.

Imagine seeing this on an executive dashboard:

My measurement brain immediately starts asking questions.

How much of that €19.4M is observed versus forecast? What baselines were used? How much of the improvement are we giving AI credit for? How much has actually translated into economic value? How confident are we in those estimates?

Without that context, a very precise-looking portfolio ROI can be built from some surprisingly uncertain numbers.

We've seen this movie before

Marketing spent years learning that activity, outcomes and incremental value aren't the same thing.

We went from impressions and clicks to attribution, experimentation, incrementality and modelling because businesses eventually wanted a better answer to:

What did marketing actually contribute?

Now AI is heading toward the same question.

We're already getting quite good at measuring adoption. We can count licences, active users, prompts and use cases. We're getting better at estimating productivity improvements.

But the next question is harder:

What value did AI actually create that wouldn't otherwise have existed?

I don't think there's going to be one perfect methodology that answers that question for every AI use case. Marketing never found one either.

But we can at least avoid repeating some of the mistakes we've already made.

Because “this happened after we introduced AI” and “this happened because we introduced AI” are two very different statements.

And the difference between them might eventually be the difference between reported AI ROI and real AI value.