Your analytics tool ships with a default: credit for each sale goes to the last thing the buyer clicked. That one setting decides which channels win your reports, the budget follows, and it has run untouched since install day. This week's issue is about what three companies learned when they turned the ads off, and the two-week experiment that gets you your answer.

Catch is an AI admin that's as easy as a conversation. Just call Catch and talk, like you would any assistant. Scheduling, bookings, follow-ups: say it once, consider it done. No apps to learn, no forms to fill. Get started at catchagent.ai and speak to your admin savior today.

Start now for free

The budget meeting ran 40 minutes. The decision took only five.

Your head of marketing walked in with a clean slide. Call her Elena; she's a composite of half a dozen operators I've worked beside, and each of them has built this exact page. Paid search delivered leads at $38 apiece, with a third of them booking demos, while the podcast and the newsletter showed a growing audience and a handful of tracked conversions between them.

Her recommendation followed the numbers, as recommendations will do, and the spend moved to the channel that converts.

You'd have made the same call, and so would I. The alternative was defending a podcast on feel and vibe, in front of the person who signs the checks, against arithmetic that all checked out.

Six months later the search numbers still look great, but the pipeline behind them is thinner. Demos arrive colder, and deals stretch out. Prospects seem far less certain

The dashboard measured which channel touched each buyer last. The meeting treated that as proof of where the demand began. Those are two different questions, and the company bet its growth on an instrument built to answer the first and hardly acknowledge, let alone credit, the second.

Old pickup trucks often carried two fuel tanks, with a single gauge and a toggle switch deciding which side the driver saw. Wire the needle to the full side while the engine drinks from the empty one, and it holds steady right up until the truck coasts onto the shoulder.

Every reading along the way was true. The fault lived one layer down, in the wiring.

Marketing attribution, the system that assigns each sale to a channel, is that gauge. Its default wiring asks one question: Who touched the buyer last before the money arrived?

A buyer hears you on a podcast in March, asks a peer about you in May, reads four issues of the newsletter, and then, the week her pain peaks, types your product's name into Google and clicks the ad perched on top of the results. The last touch books the sale, but the channel that spent a year earning it logs a zero.

Let that wiring run for a while, and each line of the report points the same way. Ads convert; audience-building costs money. So the room cuts those line items, which shrinks the pool of buyers who type your name, which the paid channel then harvests at a slowly declining rate, while the needle holds steady the whole way down.

eBay once wondered about this, so they handed the question to economists. Thomas Blake, Chris Nosko and Steven Tadelis ran controlled experiments on the company's paid search, shutting classes of ads down and comparing sales against the platforms and regions where the spending continued. With the brand-keyword ads off, shoppers took the free link below where the paid one had been, and sales barely moved. Their paper in Econometrica concluded that conventional attribution had been handing out credit for purchases already on their way.

The experiments held a second finding. The non-brand ads did move some sales, for shoppers who were new to eBay or hardly ever visited. The regulars, the buyers most of the spend went to, purchased at the same rate either way, and their share of the cost dragged the average return negative.

Uber got there by accident. Kevin Frisch, who ran performance marketing at the company, tells the story publicly: while chasing a separate problem, his team switched off $100 million of Uber's $150 million in annual ad spend. Rider app installs held level. Much of that budget had been buying credit for installs arriving on their own, with a layer of outright fraud collecting the rest.

Airbnb wrote the lesson into policy. In 2020 the company cut $541 million from performance marketing, and CEO Brian Chesky later put it plainly: the pandemic showed Airbnb could take marketing to zero and keep 95% of the traffic it had the year before. By that winter, per Campaign's coverage, more than 90% of what arrived was direct or unpaid.

All three found the same thing. The gauge read full while much of the fuel flowed from a tank that no instrument was watching.

A true number can steer a company wrong a second way — through the topline itself.

Say site traffic reads down 8% for the year: annoying but survivable, the kind of dip a room blames on the market. Underneath, the split tells a harder story. Visits from buyers who mean business, 40% of the total, have fallen by half, while low-intent drop-ins from an old listicle and a free tool have swelled by a fifth. Blend those two moves and out comes the calm little dip.

(The arithmetic is illustrative; the pattern is one I've watched operators uncover months after it started.)

A flat number looks settled, so it draws a nod and the review moves on. Whatever is going wrong inside it gets another month to get worse. The topline buries exactly that kind of trouble, because it adds two groups together: buyers who pay you and drop-ins who pad the count. Split the two and look more closely before you believe the total, because that total tells tall tales.

Michael Harris and Bill Tayler, writing in Harvard Business Review, call the underlying habit surrogation: the mind swaps a strategy for the metric meant to measure it. The strategy was grow efficiently. The stand-in became cost per lead. A few board decks later, the company runs on that number, and arguing with it feels like arguing with growth itself.

“Why, one who questions cost per lead is questioning growth! Who is the crazy person that questions growth, and why does that person have an office?”

Their marquee example is Wells Fargo, where the bank swapped a relationship strategy for a cross-selling target, and employees chasing that target opened 3.5 million unauthorized accounts. The metric became the point, and the relationships it stood in for paid the price.

The same researchers offer a repair: measure a strategy with several metrics at once, because a single number practically invites the swap. On your dashboard, that means cost per lead shares the panel with pipeline velocity and win rate — three readings the room has to reconcile before any one of them gets treated as the goal.

Elena's slide belongs to the same family, minus the fraud of course. Each figure on it survived checking. The wrong question literally shipped inside the tooling: analytics defaults credit the last touch, so the channels easiest to measure look strongest in the reports. The budget line that creates demand, on the other hand, shows up in review after review looking like pure cost. A sharp operator can follow accurate numbers straight into a starved pipeline — and a starved pipeline feeds no one

Before you say it out loud, I know what you're thinking.

“But Jason, those three experiments were safe to run because the whole market already knew where to find the companies behind them. That’s not where we are.”

Damn straight.

eBay's shoppers had the road to the store memorized, which is why removing the paid signpost changed so little. I mean, Airbnb came out of 2020 with more than 90% of its traffic arriving unpaid.

A $2 million ARR business is in a different position. Most of your buyers haven't heard of you yet, so ads can be the main way new customers find you, and cutting them would choke the pipeline.

Fortunately, there's another way to look at it.

The three experiments prove the test works. Two weeks of your own time will tell you what it finds at your company. I promise you won't need a single economist.

The experiment the giants ran fits inside a growth-stage company, and it should only take about two weeks to run effectively.

Ask the “anyway question.” For each channel taking credit, say it out loud in the room where money moves: What would have happened anyway? The dashboard already answered the last-touch question; asking this one sends you after the dollars that changed the outcome.

Flip one switch for two weeks. Pause the winning channel in one region or segment if you can split it, wholesale if you can't, then watch demos, signups and revenue. After all, investment in that channel exists to move those outcomes. A pipeline that holds tells you the credit was inflated. A pipeline that sags tells you the money earns its place. Either answer beats more weeks of reading the needle.

(One quick side note: Your version of this test will be noisier than eBay's, because a small company has small samples and a slow sales cycle, so keep a control where you can. Turn the ads off in Texas and leave them on in the Carolinas, or pause one campaign and hold its twin, then compare the two halves over the same weeks. A noisy comparison built that way still tells you more than a spotless last-touch report.)

Split the topline before trusting it. Separate the visitors who buy from the visitors who browse, and read the two lines separately. A collapse can hide inside an average, but a split view leaves it exposed.

Give each steering number an owner. Any metric that moves money gets a name next to it, somebody who can say which question it answers and when it last faced the “anyway” test. If your week has no room to carry that, hand the wiring to a person whose job is to own it. That job exists, and filling it costs much less than a year of budgets built on borrowed credit.

Run the test once and you come back owning something rare in our world: a number that earned its place. You paused the channel, watched the pipeline, compared the halves, and the claim survived. A number with that history carries a weight no untested reading can match, because you tried like hell to break it and it still held.

Certainty comes preinstalled with the tooling. Trust costs two weeks and one uncomfortable pause (and zero economists at this scale), and it holds strong when the next budget decision leans on it. So ask the question the dashboard isn't wired to answer:

What would have happened anyway?

The number that survives it is the one worth steering by.

Get revenue agents, workflows, and automations across every stage of your motion. Access customer data in real time through Attio's web app, MCP, API, and SDK.

Then Ask Attio anything about your business and get instant answers.

It's the CRM that runs the work behind every win.

Try Attio Now

Thanks for reading. See you next Wednesday.
— Jason

P.S. If your best month ever still had you sweating payroll, June's The Profit That Can't Make Payroll is the companion read.