Beyond Last-Click: How We Prove Incremental ROAS in Retail Media

As the dust settles on retail media’s gold rush, success is shifting from staking first-mover claims to proving measurable business outcomes.

Retail media has always been unique in its ability to unify goals between retailers and advertisers – namely, driving topline sales. However, it’s not always clear when media actually moves the needle versus taking credit for purchases that would’ve happened anyway. 

A 2024 survey by the Association of National Advertisers found that 71% of advertisers now consider incrementality as the most important performance metric for retail media network (RMN) investments. Increasingly, they are replacing traditional metrics like return on ad spend (ROAS) with advanced measures of incremental, ad-driven ROAS.

Yet, despite this growing recognition, most advertisers still make budget decisions based on legacy attribution models. Let’s take a closer look at this shift and how RMN players can move towards meaningful performance indicators. 

What is retail media incrementality and why it matters

In digital advertising, incrementality is the measurement of the true impact of an ad campaign — that is, the additional value generated because the ads were shown, compared to what would have happened without them. Rather than simply counting conversions or revenue, incrementality isolates the effect of advertising from other factors like organic behavior, seasonality, and existing brand loyalty. 

The fundamental question incrementality testing answers is: "Would this user have taken the same action even if they hadn’t seen the ad?"

Without these insights, marketers risk optimizing for vanity metrics rather than true business growth. And as marketing budgets contract, according to Gartner’s 2025 CMO survey, advertisers are leaning on incrementality to get more efficient with their spend. Not all conversions are equally valuable, and even a high ROAS might not actually increase total sales.  

The problem with traditional retail media attribution

While the industry is aligned on the need for incrementality measurements, several challenges stand in the way. Marketers often lack the in-house resources and know-how to run their own sophisticated methods like controlled experiments. As many as 44% are concerned about the accuracy and reliability of incrementality results they do receive, according to Skai’s 2025 State of Retail Media report. 

On the flip side, retailers and marketplaces want to provide these insights to their suppliers, but existing incrementality testing methods come with significant barriers:

• Manual and resource-intensive: Conventional methods require extensive setup, manual campaign management, and dedicated teams to design and monitor experiments—making it impractical for many advertisers to run regularly.
• Revenue-limiting holdout groups: Traditional tests require large control groups that are completely excluded from ad exposure, leading to questionable comparisons and forcing advertisers to sacrifice significant revenue in exchange for measurement insights. 
• Incompatible with modern optimization: As retail media campaigns increasingly rely on machine learning algorithms and sophisticated targeting parameters, traditional holdout strategies lose their explanatory power and can interfere with algorithm performance.

The new gold standard: AI-native incrementality testing

As the industry’s only AI-native onsite ads platform, Moloco has built incrementality measurement directly into our ad decisioning. The solution uses a randomized controlled trial (RCT) framework powered by ghost bidding. This lets us quantify causal lift during live campaigns,while maintaining our advanced targeting and without disrupting your broader media strategy. The methodology is available for advertisers who meet minimum traffic thresholds necessary for statistically meaningful measurement.

How does ‘ghost bidding’ work?

The ghost bidding methodology, first developed by academic researchers and detailed in the Journal of Marketing Research, enables precise measurement by identifying when an ad would have won an auction, then strategically withholding it from a randomized subset of users.

Our ghost bidding methodology centers on two distinct treatment cohorts:

• Exposed group. Our ad decision service runs as usual; your ads are eligible to win and serve.
• Control group. The decision service still runs, but if your ad would have won, we withhold it for that user. Everything else at auction proceeds normally – other advertisers can still win and serve their ads.

Ghost bidding maintains the scientific rigor of randomized trials while providing a more practical, scalable testing model. And by comparing behavior between these two audience cohorts, we isolate the effect of your ads from seasonality, organic demand, and brand loyalty—delivering a clean read of causal impact during the test period.

Getting the conversion scope right

To avoid noise and bias in our incrementality measurements, we carefully define which users to include in our conversion population and analysis:

• From the Exposed group, we count only users who were actually served at least one of your ads during the experiment. This number varies based on budget and bidding strategy – if an advertiser bids aggressively, they might reach 60 out of 100 eligible users, versus just 30 users for a more conservative advertiser with limited spend.
• In the Control group, we include those who were eligible and could have been served at least one of your ads (based on the same decision logic), even though we withheld the ad by design. Naturally, brands with higher budgets and more aggressive bidding will have correspondingly larger exposed and control groups for comparison.

This approach eliminates noise from users who were never in contention for ad exposure. Rather than comparing all 100 exposed users (including 70 who never saw ads) against all 100 control users, we compare only the 30 who actually saw the ad against the 30 who could have seen ads. 

For in-scope users across both analysis groups, we also count all purchases during the experiment—not just click- or view-attributed conversions. This distinction is critical because the control group contains no impressions or clicks from your brand; counting only attributed conversions would dramatically understate true lift. 

This nuanced methodology ensures an apples-to-apples comparison across both groups and a trustworthy iROAS calculation.

Early results show up to 15x iROAS impact

To validate our ghost bidding methodology, we’ve conducted incrementality experiments with advertisers across multiple retail media platforms. These experiments show that incrementality can vary significantly by partner and campaign:

• iROAS performance ranged from 253% to 1,609% across advertisers—clear evidence that some programs are creating substantial value, while others have room to optimize.
• Incremental conversions per exposed user ranged from 4% to 29%—meaning up to 29 out of every 100 conversions among exposed users were truly incremental.

These results highlight that even campaigns with lower incrementality rates are still delivering substantial performance gains when it comes to additional ad-driven sales. The wide variation in performance also underscores the importance of continuous and granular testing and optimization across platforms.

Incrementality best practices for retail media

Based on our experience running these experiments, we’ve identified several best practices for effective incrementality measurement:

1. Statistical rigor: Ensure sufficient sample sizes and test duration to achieve statistical significance. Rushing to conclusions with limited data can lead to misleading results.
2. Clean population definition: Carefully define your exposed and control groups to ensure fair comparison. Include only users who were actually eligible for ad exposure in both arms.
3. Comprehensive conversion tracking: Count all relevant conversions during the test period, not just attributed ones. This prevents understating the impact in holdout groups.
4. Account for external factors: Consider promotions and other marketing activities that might influence results, and ensure your test timing doesn’t coincide with major external events that could skew baseline behavior.
5. Regular testing cadence: Incrementality isn't static—consumer behavior, competition, and platform dynamics all evolve. Regular testing helps optimize campaign performance over time.

The future of retail media measurement

As more platforms integrate testing capabilities directly into their ad serving infrastructure, incrementality measurement will become less expensive and more accessible for advertisers of all sizes.

Just as important will be future-proof incrementality solutions that don’t rely on cookies or personally identifiable information. Unlike traditional attribution methods, AI-native incrementality testing uses real-time session behavior and contextual intelligence rather than persistent user tracking, making it both privacy-compliant and more reliable for measuring ad effectiveness.

If you’re ready to quantify true ad impact and break through retail media growth ceilings, our AI-native incrementality framework can get you there—cleanly, causally, and with decisions you can trust.

Ready to compete on value instead of volume? Contact our team to discuss how Moloco can help transform advertiser conversations from costs to true ROI.

Special thanks to Sriram Ramesh, software engineer, Eunkyo Oh, product data scientist, Hyunwoo Kim, senior director of business development, and Jon Flugstad, head of business development, for their foundational work in bringing this methodology to production.