On May 28, 2026, we switched off Google App Campaigns across the whole United Kingdom and kept them off for 18 days, with no regions held back for comparison. To see what the pause cost, we built a synthetic UK: a model of what UK purchases would have looked like if the ads had kept running. Our best estimate: first purchases fell by about 13%.
In our US test, we paused Google App Campaigns in 28 states for six weeks and couldn’t detect a meaningful drop in first-purchase revenue. Same app and same channel in the UK: this time the data points to a real drop, though its exact size is uncertain. Here is how the UK test worked, what it showed, and how far we trust it.
How does a synthetic control work?
We took daily first purchases from seven comparable countries where the ads kept running and combined them into a weighted blend that tracked the UK as closely as possible before May 28. After the pause, the gap between the real UK and this synthetic UK is the estimated effect.
Two rules kept it honest:
- The weights were set on data from before the pause, then locked. The model never saw what happened after May 28 while it was being built, so the answer couldn’t leak into the question.
- We checked a second version, built from 12 months of history instead of 6. It gave a similar estimate, so the result isn’t an accident of the time window.
What did the test show?
After the pause, the real UK ran consistently below the synthetic one. Our best estimate is that first purchases fell by about 13%. The 95% confidence interval runs from a 27% drop to a 0.3% rise, so the drop is likely real, but its size is uncertain.
How sure can you be after 18 days of paused ads?
Not completely, and the numbers say so. The best estimate is a 13% drop, but the range the data allows runs from a 27% drop to essentially no change. So the fair reading is a likely negative effect of uncertain size.
Part of the reason the range is wide: daily purchases have a quirk, where a good day tends to be followed by another good day. Ignore that, and a standard test overstates how sure you can be. Corrected for it, p ≈ 0.057, just above the usual 0.05 bar.
Before the test, our power analysis said we’d need more than 30 days to reliably detect an effect this size. We ended the pause at 18, a joint decision with the client’s team: the read on direction was strong enough to act on. A shorter test gives a wider range, and that is what we got.
One more check pointed the same way. A placebo test: we ran the same method on each of the seven comparison countries, as if each had paused ads. The UK’s gap was more than twice the largest of them, and the biggest of all eight. With only seven peers, the best this test can ever score is 1 in 8 (p = 0.125), so read it as “the UK stood out,” not as proof on its own.
What could make this result wrong?
The model and the placebo test use the same seven countries and rest on the same assumption: nothing else changed in the UK around May 28 that would affect purchases. If that assumption is wrong, both would agree with each other and both be wrong together.
It would be wrong if, for example, a competitor ran a UK-only promotion in late May, app store pricing or policy changed in the UK, a UK holiday or news event shifted behavior, or a tracking change affected how UK purchases were logged. We checked that the comparison countries didn’t shift. We couldn’t rule out a UK-only change from the data alone. If you run a test like this, ask that question before you trust the answer.
We also left out two possible comparisons: one country the model couldn’t fit well even before the pause, and the global total, which includes the UK itself.
How did a drop in purchases turn into a budget cut?
The best estimate, about 13%, is what Google was adding in the UK: roughly one in eight first purchases, the ones that disappeared when the ads stopped. To decide what to do with the budget, we turned that into money.
The test only counted first purchases, but a subscription app keeps earning from renewals. So we gave each lost purchase its expected lifetime value, then set the total against what Google cost while the ads were off.
Google was most likely driving incremental purchases in the UK, but fewer than its own reports suggested. On our best estimate, the revenue it added came close to what it cost, without clearing it.
What did we decide?
We cut Google spend in the UK by more than half.
Google most likely added purchases, so switching it off would probably have meant losing them. But on our best estimate, what it added was close to what it cost, and the size of the effect was uncertain. So we cut rather than switched off: we kept the most productive part of the budget and freed the rest for channels with a stronger incremental return.
What did testing like this change for the business?
This test was one step in a bigger process. Our marketing mix model estimates what every channel adds, from the business’s own data rather than the platforms’ reports. Tests like this one check the model’s least certain, most expensive estimates, and each result goes back in to recalibrate it.
Acting on the model and the tests, the business spent noticeably less on ads and had its most profitable year. Each new customer was also worth more relative to what it cost to win them. Those are whole-business results with more than one cause, but they show that cutting spend where tests found little incremental effect didn’t hold the business back.
FAQ
Does pausing Google App Campaigns reduce app purchases?
It depends on the market. When we switched off Google App Campaigns across the UK for 18 days, our best estimate is that first purchases fell by about 13%, though the range the data allows runs from a 27% drop to no change. In a six-week test in the US, for the same app, we couldn’t detect a meaningful drop. Test each market rather than assuming one result carries over.
What is a synthetic control?
A synthetic control estimates what a market would have done without a change, by blending comparable markets where nothing changed. The blend is fitted to match the market before the change, then used as the benchmark afterward. It is useful when you can only change one market as a whole, such as pausing ads in a whole country.
When should you use a synthetic control instead of a geo holdout?
Use a geo holdout when you can randomize many regions inside one market, like US states. Use a synthetic control when the change applies to a whole market and you have several comparable markets where nothing changed.
Is a p-value of 0.057 a failed test?
No. It means the test can’t formally rule out chance at the 95% level. Here the direction was most likely negative: the UK stood out against all its peers, and the test ran shorter than the power analysis asked for. What one test can’t pin down is the exact size, and the confidence interval shows how much is still unknown. Sometimes knowing the direction is enough to make a decision. Here it was.
How many comparison markets does a synthetic control need?
More is better. With seven, the placebo test can never show a result stronger than 1 in 8, however large the real effect. A bigger pool of comparable markets gives the placebo test more power and the blend more to work with. But the markets also have to move like the one you’re testing. Many markets that don’t track it give you little to build a synthetic version from.
The team behind the test
Chinmay KulkarniData Scientist · designed the test and ran the analysis
Markus SeppamApp & Business Pro · ran the Google campaigns and the pause