Incrementality in Apple Search Ads: How to Tell If Your Spend Is Actually Creating New Installs (Not Just Borrowing Organic Ones)

If your Apple Search Ads ROAS looks “okay,” but growth still feels stubborn—or worse, it improves right after you pause ads and nobody trusts the dashboard anymore—you’re probably mixing two different effects: incremental installs caused by ads, and organic installs you would have gotten anyway.

Attribution in ASA (via AdServices) tells you what Apple believes happened after an ad click—resolved within ~24h—but it doesn’t inherently separate “new demand” from “borrowed demand.” The fix is to measure incrementality.

Below is a practical, indie-friendly way to do that with small holdouts and a few metrics.

Why incrementality matters (and why Apple ROAS can mislead)

Apple Search Ads is a CPT (cost-per-tap) auction. You’re paying for taps, but the business question is: Did those taps generate installs that wouldn’t have occurred organically (or from other channels)?

Without incrementality, it’s easy to end up optimizing the wrong thing:

You increase bids and see attributed ROAS improve, because you captured traffic that was already “searching you.”
You pause a campaign and attributed revenue doesn’t collapse, because organic demand filled the gap.
You feel unsure whether your spend is buying growth or just buying attribution.

Measuring incrementality doesn’t require fancy tooling—just a controlled comparison.

The simplest incrementality test: geo holdout (no creative, no science)

A geo holdout keeps everything the same except: you advertise in one set of countries/regions while withholding in a matched set.

Step 1: Pick two “similar” groups of countries

Choose countries where:

Your app already has some baseline installs (so comparisons are stable).
You can reasonably run the same campaign structure in both groups.

Example approach (illustrative):

Group A (test): UK, CA, AU
Group B (holdout): IE, NZ, maybe another smaller market you can support consistently

The exact countries don’t matter as much as the stability of behavior.

Step 2: Run holdout for long enough to see purchase behavior

ASA attribution resolves around ~24h for install attribution, but revenue depends on subscription/purchase lag. Plan for a window that matches your purchase cycle:

If your monetization is fast (e.g., trial converts quickly), you can measure payback within a few days.
If purchases happen later, you need a longer observation window.

Practical rule: run the experiment long enough that you can see installs and at least one meaningful revenue window (even if that revenue is partial early on).

Step 3: Keep the non-ad variables as constant as possible

During the holdout period:

Don’t change store conversion levers radically (screenshots/CPP) in only one group.
Don’t launch major promos that affect just one group.
Avoid simultaneous acquisition changes that only hit the test group (e.g., a big feature landing press spike).

Step 4: Compare “test vs holdout” growth, not raw totals

For each group, compute growth relative to a pre-test baseline.

Metrics to compare:

Installs per day (or per 1,000 sessions if you have that)
Install-to-purchase conversion (installs → revenue)
Revenue per tap/spend (but interpret it carefully)

Then estimate incremental installs roughly as:

Incremental installs ≈ (Installs_test − Installs_holdout_adjusted_for_baseline)

Because countries differ, you don’t want to just subtract. At minimum, normalize by baseline averages before you started the holdout.

A more targeted holdout: campaign-level pause by placement

If you don’t want to risk pausing spend across entire countries, you can isolate incrementality within a campaign by controlling where your ads show.

Option A: Placement holdout

For example:

Keep Search Results running.
Temporarily withhold spend in Today tab or Product Pages for a period.

This tells you whether those placements are primarily capturing demand that would have already existed, or if they’re genuinely creating incremental taps and installs.

Caveat: placements can attract different user intent. That’s why you compare against a baseline and focus on changes over time.

Option B: Keyword intent holdout (high-risk categories)

Choose a bucket that’s most likely “borrowed” (e.g., aggressive broad match on very generic terms, or non-brand keywords near your brand name).

Pause only that slice for a week (or a shorter interval if your funnel is quick) and compare:

Total installs in the test markets
Total revenue
Whether conversion rates shift (they often don’t, but check)

If the app keeps getting the same installs/revenue without that slice, your ROI might be overstated by attribution capture.

The metric set that makes incrementality actionable

When you run incrementality tests, don’t rely on a single chart.

Use this small set of metrics:

Attributed installs (ASA) — what Apple says you bought
Revenue from those installs — mapped via RevenueCat/App Store subscription events
Organic baseline trend — what happens in the holdout group
Conversion stability — installs → subscriptions/purchases doesn’t collapse (otherwise you changed the store or user quality)

Why conversion stability matters: if store conversion drops during the test, the holdout comparison becomes noisy.

How to avoid false conclusions (common pitfalls)

Pitfall 1: You measure too early

If you’re buying trials or long subscriptions, early revenue can make everything look “incremental” when it’s only timing.

Fix: evaluate against multiple horizons (e.g., short-term and medium-term), even if the first window is imperfect.

Pitfall 2: The holdout group is not comparable

Some countries behave differently. If your holdout markets were weaker or improving faster, your “incremental” estimate will be wrong.

Fix: choose stable markets and normalize by pre-test averages.

Pitfall 3: You changed bids during the test

Bids change tap volume, which changes everything. The holdout should isolate only the ad variable you intended to test.

Fix: freeze the rest of the auction levers as much as possible.

Pitfall 4: Other acquisition channels changed

If you also ran social/email/ASO efforts, the holdout comparison can’t cleanly isolate incrementality.

Fix: coordinate timing, or interpret results as “incrementality vs other channels,” not pure ASA-only causality.

What to do with the result

Once you know whether installs are incremental, you can adjust your Apple Search Ads strategy with more confidence.

If you find low incrementality:

You’re likely paying for demand that already exists.
Reallocate budget toward keyword intent that expands reach (or toward placements/terms that bring new users).
Be more careful with brand-adjacent and overly broad queries.

If you find strong incrementality:

Your spend is buying real growth.
Then it makes sense to scale bids/CPT where you have headroom, while still monitoring CPI and conversion rate.

Either way, you stop treating attributed ROAS as the whole story.

Where an advisory workflow helps (optional)

If you want this to be less manual, an advisor workflow can help you structure the test, pick what to hold out, and ensure the apples-to-apples comparisons match your funnel and revenue mapping. Tools like AdsBuddy (which reads your Apple Search Ads performance + revenue and recommends a prioritized set of changes) can help you decide what to test first—but you still approve and apply the changes yourself.

Takeaway

Incrementality is the missing layer between “ASA attribution looks good” and “ASA is actually creating new business.”

Start with a lightweight geo holdout (or placement/campaign holdout if geo is risky), compare test vs holdout growth against a baseline, and make decisions based on whether installs and revenue meaningfully rise relative to the no-ads group.

If you measure incrementality once and treat it like a recurring check—not a one-off—you’ll stop over-optimizing for attribution and build a budget that buys sustainable growth.