Every retail media network hands you a dashboard. Amazon attributes sales to Sponsored Products. Walmart Connect reports ROAS. Kroger, Target, Instacart — they all have reports. They all look good.
Skai found that 94% of retail media advertisers don't fully trust retailer-reported ROAS numbers, per the company's 2026 State of Retail Media report — a survey of 166 advertisers conducted with research firm Stratably. The same survey found that only 15% of marketers describe themselves as effective at measuring retail media performance, and found that 75% call incrementality their single biggest measurement challenge.
This isn't a fringe complaint. It's the dominant view of the people spending the money.
The gap between attribution and causation
Retail media networks report attributed sales — every purchase from a shopper who saw your ad gets credited to the campaign. Attribution is not causation. If a buyer was going to purchase your product anyway, the ad didn't generate incremental revenue. It just got credit for a sale that was already happening.
Independent measurement tells a different story. Incremental ROAS — what you actually earned because of the ad, versus what would have happened without it — typically runs 30 to 60% below last-click ROAS, per data from Digiday's measurement research. Not sometimes. Consistently.
The gap is even more dramatic at the methodology level. iROAS can vary by 6.5x and flip results in 83% of campaigns based on measurement methodology alone, according to research covered by Digital Commerce 360 from a collaborative study by Albertsons Media Collective, Ovative Group, and Northwestern University's Kellogg School of Management. Same campaign, same spend, wildly different answers depending on how you measure.
If your retail media ROAS shows 8x on the network dashboard and your actual incremental ROAS is 3x, you might still be profitable — or you might not be. You need to know which.
Why the methodology problem is getting worse
No two retailers calculate ROAS the same way. Amazon reports by SKU. A grocery retail network may report by category. CPG platforms often fold subsidized promotions into the sales denominator in ways that inflate the reported number. Per the Skai/Stratably survey, advertisers run an average of six retail media networks today — six different methodologies that can't be compared directly.
That same survey found advertisers expect to grow from six networks to eleven by end of 2026. More networks means more fragmentation, which makes a bad measurement problem dramatically worse. You can't optimize a portfolio across eleven networks when their attribution frameworks are all different and none of them is running a control group.
What a holdout test actually measures
Holdout tests — exposing ads to one matched group of buyers while withholding them from an equivalent control group — are the only method that isolates whether your ad caused incremental purchases. The control group keeps buying normally. The test group gets your campaign. You measure the difference.
Done properly, this removes the "would have bought anyway" purchases that inflate reported ROAS. The brands running these tests consistently see incremental ROAS below their network dashboards. Brands not running them are operating on the assumption that every attributed sale was genuinely caused by their campaign.
Per the Skai/Stratably data, brands actively measuring incrementality reported 54% reduction in wasted spend and 49% improvement in new customer acquisition. Those gains don't come from better creative or smarter targeting. They come from reallocating budget away from campaigns that looked good on the dashboard but weren't generating real lift.
Four things to check before the next planning cycle
Review the last three retail media campaigns you ran with strong ROAS reports. For each: was there an independent holdout test? If not, treat the reported ROAS as a ceiling, not a confirmed result.
Map the attribution methodology across your active networks. How does each one calculate attributed sales? What's the attribution window? If a shopper sees both a sponsored product and a DSP impression on the same network in the same session, is that one sale counted once or twice? Your account team should be able to answer these. If they can't, that's information.
Set a budget line for incrementality testing. Holding out 10–15% of your retail media spend as an unexposed control group costs real impressions. It also tells you whether the rest of the budget is generating return. According to WARC, global retail media spend will reach $200B in 2026, while 94% of advertisers are uncertain about the measurement per the Skai data — the industry is operating at significant risk.
Compare your networks against each other on methodology, not just reported ROAS. If one network consistently returns 7x and another 4x, the higher number might reflect a more generous attribution model rather than better performance. Find out which.
The ROAS number on your retail media dashboard is the number the network wants you to see. The incremental number is the one that tells you whether to keep spending.
Sources: Search Engine Land, Digital Commerce 360, Digiday, WARC, September 2026
