Incrementality Is the Only Honest Metric

In Issue #2 I wrote that the serious CMO's job in 2026 is to author the marketing budget alongside the CFO, on numbers both of them trust.

There is a follow-up question I owe everyone who's been reading these letters in order.

Which numbers can be trusted at all?

This is the issue where I answer it. I'll save you the suspense. The answer is one.

Incrementality is the only metric in marketing measurement that asks the question every other metric refuses to ask. And the reason the industry has spent fifteen years avoiding it — including, until eighteen months ago, us — is that the answer is uncomfortable.

I want to be honest about how uncomfortable.

Let me start with the definition, because the word has been laundered enough in vendor marketing that operators have started using it loosely.

Incrementality is the answer to one question. If we had not run this spend, what would have happened?

It is not a touch. It is not an attributed conversion. It is not a "lift" reported by the platform itself. It is a counterfactual — a comparison between what happened in the world where you ran the campaign and what would have happened in the world where you didn't.

To measure incrementality honestly, you have to construct that second world. You hold out a region, or a cohort, or a time window, and you watch what happens without the spend. The gap between that and the world where the spend ran is the incremental impact. The point estimate is interesting. The interval around it is what makes the answer honest. (Rajeev wrote about why that interval matters in the first issue of The Incrementalist. I won't restate the methodology argument here. I'll just say it is the floor under everything I'm about to write.)

Most other metrics in marketing don't ask that question. ROAS doesn't. Attribution doesn't. Multi-touch doesn't. The reason is structural. They are all measures of presence — whether the channel touched the converter, or appeared near the conversion, or got credit for being upstream. Presence is not contribution.

A streetlight is present at every car accident. That doesn't make it the cause.

This is the line I keep coming back to in conversations with operators. Most marketing metrics are streetlight metrics. They measure what was nearby when the conversion happened. Incrementality is the only metric that asks whether the light caused the accident.

I have to be honest about the second part of this letter, because it implicates me.

For most of the last fifteen years, marketing measurement vendors — including the one I run — have built and sold products that did not centrally rely on incrementality. Attribution platforms. Multi-touch. MMM consulting projects. Each one credible inside its own frame. None of them, on its own, capable of telling a CMO whether the spend caused the outcome.

The reason this happened is not malice. It is commercial gravity.

Incrementality testing requires holdouts. Holdouts require deliberate, planned underspending in a region or a cohort. Deliberate underspending feels, to a marketing team under quarterly pressure, like leaving money on the table. The team's first instinct is to skip the test and ship the campaign.

The vendor's first instinct is the same. A product that asks customers to deliberately reduce spend in service of a test result is a harder sell than a product that reports rich-looking attribution numbers on every dollar spent. Especially when the test result might tell the customer to spend less overall, which makes their next renewal conversation with the same vendor more complicated.

So the industry quietly settled into a different equilibrium. Vendors sold products that included incrementality as a feature — a tab in the platform, a quarterly report, a service engagement. Customers ran one or two tests on a low-stakes channel, got a result that confirmed their bias, and never went back to the methodology. The word incrementality showed up in keynote decks. The discipline did not show up in the budgeting cycle.

I am not pointing at this pattern from outside it. We sold attribution as our primary product for years. We added incrementality as a feature. Our customers ran some tests. Most never ran another.

The change at Lifesight eighteen months ago was the decision to invert this. Incrementality stopped being a feature of our platform. It became the calibration anchor that every other number in the platform has to defend itself against. If a campaign's reported ROAS cannot be reconciled with a recent geo-lift result, the reported ROAS is the number we treat as suspect. Not the test.

This is, in the candid version of the story, what walked us away from attribution as a standalone product. We could not in honesty sell a number that we knew was not defended by an actual counterfactual.

Here is the part most marketing teams have not internalized about incrementality, and the part I think the next two years are going to force the category to confront.

The honest answer often shrinks the defensible budget.

Most channels, when tested rigorously, deliver less incremental impact than their attribution numbers suggest. Sometimes substantially less. The most aggressive overcrediting tends to happen on the channels closest to the conversion. Branded paid search is famously near-zero incremental in many businesses — the customer searched for your brand because of upstream work the search ad didn't cause. Retargeting often has the same problem, scaled. Affiliate sometimes overlaps with organic intent the brand would have captured anyway.

When a CMO runs the first incrementality test on a high-confidence channel and the result comes back materially below the attribution-flavored number — and it almost always does — the CMO has a choice.

Option one. They can accept the new number and update the budget. Which usually means cutting spend on a channel they have already publicly defended, in front of a CFO who is now paying close attention, with a campaign team who is going to ask uncomfortable questions about what they have been optimizing for.

Option two. They can dispute the test. Maybe the holdout was too short. Maybe the geo-mix wasn't representative. Maybe the measurement methodology was confounded by a seasonal shift. There is always a reason. A motivated CMO can find one in any test result.

Option three. They can quietly never run another test.

The default behavior across the industry, for fifteen years, has been some combination of options two and three. I'm not blaming individual operators for this. The incentive structure rewarded it. The CMO who ran one test and never ran another kept a budget that was easier to defend. The CMO who ran tests every quarter and updated honestly often shrunk their own budget — which, until very recently, was a career-limiting move.

That last sentence is the one I expect to age the fastest. The career incentive is changing, because the CFO is now in the room. As I wrote in Issue #2, the CFO's arrival is the credible counterparty CMOs have been quietly waiting for. The CMO who runs honest incrementality tests, accepts the smaller budget the test justifies, and defends that smaller number for real is now the marketer the CFO trusts most. The CMO who runs no tests and defends every dollar is the one the CFO now quietly distrusts.

That's a different career incentive than the one that shaped the last decade. It is the one I think shapes the next one.

There is a line I keep using inside Lifesight when we are training a new account team. It is, I think, the cleanest one-sentence statement of why this matters.

Most channels can survive a one-month holdout. Most marketing arguments cannot.

What I mean by it is this. If you tested every channel honestly, the channels themselves would mostly be fine. The genuinely incremental ones would prove out. The marginal ones would shrink. The fake ones would disappear. The marketing function as a whole would emerge smaller, sharper, and significantly more credible to the rest of the business.

What would not survive is the argument structure most marketing teams have been making for the last fifteen years. Attribution-flavored stories. Funnel narratives unsupported by causal evidence. Brand spend defended on faith. The argument structure depends on never running the test that would falsify it.

The honest answer is the one you don't want to hear. That is how you know it is honest.

Three things I'm watching in this shift.

One. Whether competitors finally make incrementality the default. Several vendors in this space added incrementality as a feature in the last twelve months. None of them, to my knowledge, have made it the calibration anchor for every other number in their platform. That is the structural commitment, and it is the one that costs vendors revenue in the short term and earns category leadership over five years. I expect at least two competitors to attempt this shift in the next twelve months. I expect most of them to back off when the commercial pain shows up.

Two. Whether CMOs adopt "kill criteria" before campaigns launch. The cleanest discipline I have watched our best customers adopt is the practice of stating, in writing, what test result would justify cutting a channel — before the campaign launches. Once the campaign is in market, the politics of cutting become difficult. Once the kill criteria are written down up front, the cut is a procedural follow-through, not an organizational conflict. The CMOs who institutionalize this look, eighteen months later, like a different kind of operator. I'd like to see this become a standard.

Three. Whether agents trained on calibrated state outperform agents trained on attribution. This is the experiment I most want a third party — not us — to run cleanly. Two AI agents, same brand, same task, same window. One reads attribution dashboards. One reads incrementality-calibrated state. Measure decision quality at the ninety-day mark. If there is a meaningful gap, the entire vendor category is about to bifurcate on this axis. If there is no gap, my thesis is in trouble and I would want to know.

I want to close with the part I owe the reader.

Incrementality testing is not free, and it is not always pristine. Holdouts can be contaminated by market-level spillover. Small experiments on low-signal brands can produce intervals so wide they are not actionable. Some channels — high-frequency brand TV in saturated markets, for example — are genuinely hard to test cleanly inside a fortnightly cadence.

These are real limits. The honest framing is that incrementality testing produces better-calibrated uncertainty than the alternatives, not perfect certainty. A vendor — including us — who claims that incrementality measurement gives the CMO a single trustworthy number is selling the same false confidence the industry has been selling for fifteen years, just with a different label on the bottle. I'd rather be honest about the limits and let the customer decide what level of imperfection they will accept.

What incrementality does, that attribution cannot, is bound the question. It tells you the range inside which the answer lives, and it lets the marketer make a budget decision with a calibrated sense of how wrong the answer might be. That is what an adult marketing function is supposed to look like. That is what the rest of the business expects from finance, from operations, from product. Marketing has been the last function in the company to be held to this standard. I think that is changing, and I think it is changing fast.

If you've run an incrementality test that produced a result you didn't expect — or one you expected but couldn't get your team to act on — I'd like to hear about it. The cleanest way to learn this category is to compare notes with operators who have run the experiment and lived with the answer.

Thanks for reading the third one.

Tobin Co-Founder & CEO, Lifesight June 2026