Two measurement announcements landed on the same day this week, and both came with a performance number attached. I think the right place for those numbers is the test plan, not the forecast.
The numbers on the table
TikTok’s announcement of its ad network for US targeting says early testing found 65% improved CPA, 63% improved ROAS and 14% higher budget utilization. The footnotes in that post cite TikTok’s own internal measurement and carry the line that results may vary. The same day, DoubleVerify announced attribution measurement for ChatGPT Ads and said, in a release on its investor site, that for Weight Watchers the attributed cost per acquisition on ChatGPT was 15.3% below the company’s benchmark during the measured period.
I am not faulting either company. A launch needs a proof point, and a proof point from a first customer or an internal test is the normal kind. My argument is about what a marketing team does with it.
Why I would not budget from them
Each figure answers a narrower question than the one a budget owner asks. TikTok’s numbers describe tests on its own network, measured by its own systems, and the post says so. The Weight Watchers figure describes one advertiser over one measured period against that advertiser’s own benchmark. Neither tells you what the channel would do for a different brand, a different offer or a different quarter.
There is also a gap between a result and a cause. A cost per acquisition below benchmark can come from the channel, from the creative, from the audience that happened to be reachable, or from the timing. Attribution models assign credit. They do not run the experiment. A team that wants to know whether a placement adds sales needs a comparison group that did not see the ads, which is a holdout, and no launch announcement can supply one for your account.
Sample size matters here too. One advertiser’s measured period is a single observation, and a figure labelled early testing does not say how many campaigns sit behind it or how they were chosen. The releases do not give that detail, so a reader has no way to judge how far the result travels. I would ask for it before leaning on the number in a planning meeting.
The strongest counter-argument
The best objection is that a team has to start somewhere, and launch-day evidence is the only evidence that exists. Waiting for independent studies means ceding a new channel to competitors who moved first. If the early numbers are real, ignoring them is its own mistake. I accept most of that. I would not tell anyone to skip a new channel because its first numbers come from the seller.
My answer is about sequence. The figures justify a test. They do not justify the size of the budget line after the test. Treat 65% or 15.3% as the result you hope to reproduce, and write down before launch what result on your own account would keep the spend, what would cut it, and how long the test runs.
What a good test plan includes
Hold out a share of the audience, or a set of regions, that sees no ads in the channel. Ask for reporting at the placement level, so a blended campaign number does not hide what the new inventory contributed. Name the measurement vendor in the plan. In TikTok’s case the post lists DoubleVerify and Integral Ad Science for post-bid measurement, and Lidiane Jones, CEO of Integral Ad Science, said in the post: “This gives marketers the full picture of their campaigns across viewability, invalid traffic, and brand safety”. Viewability, invalid traffic and brand safety are quality checks. They do not measure incremental sales, so they answer a different question from the one the CPA figure answers.
Our recent pieces on measurement disputes moving from the number to the definition and on AI chat ads inheriting the web’s old tracking fight make the same point from other angles: before comparing two numbers, find out what each one counted.
The call
New channels will keep arriving with early figures, because that is how launches work. A marketing leader’s job is to keep the figure and the forecast in separate documents. Run the test, set the threshold in advance, and let your own holdout decide the budget.
Source: TikTok for Business, “Now available for U.S. targeting: TikTok Ad Network”
