Your New Campaign Looks Worse Because Its Customers Have Had Less Time

A campaign launched yesterday has had one day to convert. A campaign launched three weeks ago has had three weeks. Put their purchase rates side by side without accounting for that difference and you can mistake the passage of time for a change in performance.

This problem becomes expensive when a team refreshes its dashboard daily but sells something people consider for days before buying. Yesterday's spend appears immediately, while part of the revenue arrives later; the freshest campaign can look weakest precisely because its customers are still deciding.

Consider a hypothetical luggage retailer selling through Shopify. The following numbers are synthetic, designed to explain reporting logic rather than describe an actual merchant. We'll track first eligible site visits and subsequent purchases, using one consistent visitor definition throughout.

Give every cohort the same clock

Group visitors by the date of their first eligible visit. For each group, measure purchases within one day, seven days, and fourteen days of that visit. Those are cohort-age windows, separate from the calendar date on which a purchase happens.

An older cohort contains 10,000 visitors. It records 100 purchasers within one day, 220 within seven days, and 250 within fourteen days. Its cumulative purchase rates are therefore 1%, 2.2%, and 2.5%.

A newer cohort also contains 10,000 visitors and has recorded 110 purchasers within one day. Its one-day purchase rate is 1.1%. Comparing that figure with the older cohort's fourteen-day rate of 2.5% makes the new campaign look dramatically worse, even though its one-day rate is slightly higher.

The fair early comparison is 110 versus 100 purchasers at the same age. That doesn't establish a winner: sampling variation and differences in visitor mix could explain the gap. It does remove a reporting error that would otherwise dominate the discussion.

And the seven-day column for yesterday's cohort should remain unavailable. Filling it with zero quietly tells everyone that seven days elapsed and nobody purchased, which is a different claim.

Use a forecast with an expiration date

The older cohort accumulated 100 of its 250 fourteen-day purchasers in the first day. Its observed one-day maturity fraction is 40%. If the newer cohort follows exactly the same pattern, its 110 first-day purchasers imply 110 divided by 40%, or 275 purchasers within fourteen days.

That's a 2.75% projected purchase rate across 10,000 visitors. Call it a projection wherever it appears, and replace it with observed outcomes when the window closes. Don't combine forecast purchases with actual purchases in a total that looks fully realized.

This shortcut rests on a strong assumption: the timing pattern stays stable. A new promotion could accelerate purchases, a different audience could take longer to decide, or delivery expectations could change the journey. A first-day improvement might simply pull purchases forward from later days.

For that reason, use several mature cohorts to understand how much the maturity fraction varies. A single older cohort gives you an example, not a dependable forecasting model. Keep major differences in product, device, channel, and promotional exposure in view, while avoiding so many segments that each becomes too small to interpret.

Separate customer behavior from reporting delays

A purchase can happen three days after a visit because the customer needed time. A purchase can also happen today and appear in the warehouse tomorrow because an integration runs overnight. Those delays require different fixes.

Record event time and ingestion time separately where your data allows it. Use a stable reporting cutoff, and tell the team how far behind the data may be. If your order feed is incomplete for yesterday, you shouldn't diagnose a new customer-behavior problem from yesterday's missing orders.

Define the counting rules before you compare cohorts. Decide how you handle repeat visits, multiple orders, cancellations, and customers who return on another device. If visitor identities cannot be connected reliably, acknowledge that the cohort view measures the population you can observe.

The fourteen-day window also limits the question. It tells you about purchases within fourteen days, while later purchases remain outside the measure. Choose a window that fits the decision and keep it consistent across comparisons.

Build a report that resists a bad meeting

Keep three views beside each other: mature outcomes for closed cohorts, equal-age outcomes for recent cohorts, and explicitly labeled forecasts. A single blended total makes it too easy to compare observations with estimates without noticing.

In the luggage example, the older cohort's 250 purchases are observed at fourteen days. The newer cohort's 110 are observed at one day. Its 275 are a conditional forecast. Each number answers a useful question, provided the label travels with it.

You can still act before a cohort matures. Broken checkout, invalid traffic, or a stockout may justify an immediate response because those problems have direct evidence. A disappointing immature purchase rate requires more care.

This week, take one campaign report and add days since first eligible visit beside each cohort. Recalculate the comparison at a shared age, leave future windows blank, and review whether the proposed budget cut still makes sense.