Blog post

Blog Article

Machine learning for commerce media: The strategic separation of Organic and Ads ML

By:
Henry Senger
No items found.

April 1, 2024

To truly understand and succeed in commerce media, it's important to recognize how big tech companies strategically use machine learning (ML). Commerce media, which refers to ads within e-commerce platforms, is deeply impacted by two main ML types: Organic and Ads. Each has its distinct use cases and different requirements. Platforms we speak to frequently ask why we suggest separating the ML models and why Big Tech companies like Google, Facebook, and Amazon do the same.

Let's dive into the distinctions between Organic and Ads ML and the benefits of separating them.

The use cases of Organic ML in ranking 

Organic ML generates predictions for products or items to determine their ranking. For instance, if a platform aims to increase clicks on organic items, a common approach would be to develop a model that predicts clickthrough rates, known as a pCTR model. The pCTRs then arrange the items in descending order of predicted clicks.

A standard method to assess the effectiveness of changes or improvements in Organic ML models is to conduct an A/B experiment. In this setup, users, cookies, or requests are split between a treatment group and a control group randomly. The performance metrics from each group should reflect the outcomes expected if the changes were implemented across all traffic. This method clearly indicates the impact of changes made to the Organic ML models.

Understanding the complexities of Ads ML beyond ranking

On the other hand, Ads ML predictions are one of several factors determining ad rankings. For example, ads are often ranked based on expected revenue per impression, calculated as the product of their cost-per-click (CPC) bid and pCTR.

To understand the different requirements of Ads ML, consider an e-commerce site ranking organic items by pCTR. Say there are two items, Item A with a pCTR of 0.6 and Item B with 0.1, but their trueCTRs are 0.3 and 0.2. This discrepancy might not significantly alter user experience in organic ranking, as the items' order remains the same. However, in ad ranking, which typically depends on expected revenue per impression, inaccuracies in CTR predictions can significantly impact ad placement and revenue.

Suppose Item A bids $1 and Item B $5, the initial expected revenue per impression would be $0.60 for A and $0.50 for B. But with trueCTRs of 0.3 and 0.2, the true potential revenue would be $0.30 for A and $1 for B. This misjudgment can lead to incorrectly prioritizing Item A over Item B in ad placements despite Item B having a significantly higher expected revenue. Moreover, if a second-price auction is used, inaccurate pCTRs can also affect the final price paid per ad click. For these reasons, it's not uncommon for a model that improves organic performance to hurt ad performance.

Additionally, Ads ML needs integration with other systems like bidding and budgeting, making experimentation more complex. Unlike with Organic ML, simply diverting traffic for an Ads ML experiment at CTR prediction system level isn't sufficient, as it can affect other operational systems. To accurately measure changes in an Ads ML model, these connected systems must account for experimental traffic diversions. At Moloco (and at big tech companies that run ads), advanced capabilities are necessary to separate budget and bidding feedback loops according to the traffic split in experiments.

The strategic separation of Organic and Ads ML for optimal performance

Given the distinct use cases and the need for different experimentation and measurement approaches, separating Organic and Ads ML is beneficial. This separation allows the Organic ML team to improve the organic experience without worrying about the impact on ads and vice versa. Changes beneficial for organic experiences can adversely affect ad performance and business, and the reverse is also true — this is why the distinction is important. Combined with the more complex experimentation required for Ads ML, separating Organic and Ads ML models is advisable and practiced by big tech companies. It has allowed modeling teams to concentrate on distinct requirements and use cases, improving their models in parallel without interference.

The optimal models for these two cases will likely differ significantly.

Using Organic ML as a foundation for ads on your site is a sound initial strategy. However, you'll soon discover significant advantages in separating Organic and Ads ML, a strategy adopted by tech giants. Understanding this early on is important for planning how much effort and resources you'll need for your ad system and ML models and selecting the best platform to partner with. At the very least, it can allow you to quickly understand why a great model improvement for your organic experience can be detrimental to your ads business and vice versa — a common issue many sites face.

Henry Senger

Senior Director of Engineering, Moloco

SEE MORE
Dark blue arrow to learn more about the subject
Editor’s choice
Why retail media supply-side platforms alone won’t save your ad businessWhy retail media supply-side platforms alone won’t save your ad business

Learn why relying on a retail media supply-side platform (SSP) won’t drive long-term growth for your ad business and how RMNs can unlock true value with machine learning and first-party data.

read more
White arrow to learn more about the subject
A conversation with commerce media expert Jason BaggA conversation with commerce media expert Jason Bagg

Commerce media expert Jason Bagg discusses his experience launching a scalable retail media platform on a tight deadline.

read more
White arrow to learn more about the subject
3 signs your onsite retail media ads are leaving money on the table3 signs your onsite retail media ads are leaving money on the table

Many retailers are overlooking a crucial element: maximizing the potential of their owned and operated sites. Learn about the three signs that your onsite retail media ads have untapped value.

read more
White arrow to learn more about the subject
Commerce Media: The ‘PromoteIQ Shutdown’ and what you need to knowCommerce Media: The ‘PromoteIQ Shutdown’ and what you need to know

The recent shutdown of Microsoft-backed PromoteIQ is a wake-up call for the retail media industry. Learn what you need to know to invest in flexible, high-performing, personalized onsite ad technologies to scale your ad business effectively.

read more
White arrow to learn more about the subject

Want to learn more?

GET STARTED
White arrow to learn more about the subject

Subscribe to the Moloco newsletter

arrow top