Source-linked AI summary
Stageboost: Recommending Signals Based on Counterfactual Estimation
Darpan Singhal, Matan Mandelbrod, Tal Franji, Manasa Kolla, Vipul Gaba, Yuri Brovman
TL;DR
The paper addresses deterministic signal assignment on eBay’s View-Item page by modeling which qualified signals best fit each item-user context. Stageboost uses two-stage XGBoost to personalize recommendations and estimate signal uplift, producing a 0.08% overall GMB increase and a 0.58% increase in Parts and Accessories GMB in online experimentation.
Problem
The task is to select and rank qualified signals for each item-user context and placement to maximize expected conversion probability, while prior CLE assignments lacked personalization for item characteristics, user preferences, and context.
Method
Stageboost uses two gradient-boosting stages: one predicts baseline conversion probability for an item-user pair, and the other estimates each signal’s incremental conversion uplift.
Results
0.08% increase in overall Gross Merchandise Bought (GMB) and 0.58% increase in Parts and Accessories GMB were observed in online experimentation.
Takeaways & Limitations
Personalized signal recommendations adapt to item characteristics and user context, while Lean StageBoost offers a production-suitable trade-off between effectiveness and inference latency.
Takeaways & Limitations
The preceding CLE approach was limited by deterministic assignments and data sparsity for rare qualification tuples, motivating Stageboost’s feature-based modeling.
Abstract
from arXiv · showhide
Signals are short textual or visual snippets displayed on the eBay View-Item (VI) page, providing additional, contextual information for users about the viewed item. The aim of displaying these signals is to facilitate intelligent purchase and to incentivize engagement. In this paper, we present a 2 stage xgboost based model that optimally populates the VI page with signals. This approach has shown a 0.08% lift in overall GMB (Gross Merchandise Bought) and 0.58% increase in Parts and Accessories GMB, primarily due to increase in conversion of high average price items in online experimentation.
1 Introduction
eBay’s View Item page uses contextual signals to guide purchases and engagement, but prior deterministic assignment methods did not adapt to user or item characteristics. Stageboost addresses this limitation with two-stage modeling of baseline conversion and signal-specific uplift.
- Motivation: The View Item page receives hundreds of millions of daily views and displays contextual signals across predefined placements.Signals supplement item details such as images, price, descriptions, seller information, and shipping options.
- Motivation: Urgency and Conversational placements show different signal types, including scarcity cues, item benefits, and trustworthiness information.Urgency displays up to one signal, while Conversational displays up to two signals simultaneously.
- Prior work: The prior CLE model assigned signals deterministically for each qualifying signal set and did not incorporate user preferences or detailed item characteristics.CLE assigned up to one Urgency signal and two Conversational signals.
- Stageboost: Stageboost is a two-stage XGBoost approach that adds personalized signal assignment while explicitly modeling conversion uplift.M1 predicts baseline conversion for an item-user pair, and M2 estimates each signal’s incremental conversion uplift.
- Prior work: CLE achieved a 0.28% increase in GMB but faced personalization and data-sparsity limitations for rare qualification tuples.These limitations motivated modeling how item and user features relate to signal effectiveness.
- Stageboost: Stageboost uses a unified single-model formulation with differential feature weighting tailored to eBay’s latency and scale requirements.The approach builds on uplift-modeling ideas while targeting production signal assignment.
2 Problem Statement
The problem is to choose and rank qualified signals for each View Item placement using item-user context. The selected subset must fit the available display slots and maximize expected conversion probability.
- Inputs: For each impression, the context vector contains item-user features, while the qualified set contains signals passing eligibility preprocessing.X denotes the item-user feature space, and S denotes all possible signals.
- Selection objective: For placement i, the system selects and ranks R from qualified signals Q with |R| = min(k_i, |Q|).k_i is the number of display slots for placement i.
- Selection objective: The ranking function maps each context-signal pair to a score so qualified signals can be selected to maximize expected conversion probability.The conversion outcome Y is binary, such as a purchase event.
3 Methodology
Stageboost personalizes VI signal assignment by modeling both baseline conversion and signal-specific uplift, then ranking qualifying signals for each impression. It is trained and evaluated using randomized production data and a counterfactual metric designed to estimate deployment performance.
- 3.1 Overview: Stageboost uses two stages: M1 predicts baseline conversion for an item-user pair, while M2 estimates each signal’s incremental conversion uplift.The approach is implemented within a two-stage gradient boosting framework.
- 3.1 Overview: At inference, qualifying signals are ranked by M2’s predicted conversion probability and the top-k_i signals are selected for display.This ranking adapts assignments to each impression’s user context and item characteristics.
- 3.2 Data Preparation: Randomized VI experiments provide balanced training data by assigning qualifying signals uniformly rather than repeating CLE’s deterministic historical ranking.Stageboost uses two weeks of randomized data for training and one week for offline evaluation.
- 3.3 Training: A single XGBoost classifier jointly supports M1 and M2 through differential feature weighting and strategic feature subsampling.Signal indicators receive higher weights so later boosting rounds focus on signal-specific uplift after context features establish baseline patterns.
- 3.4 Evaluation Metrics: Offline evaluation estimates the conversion rate that each model’s predicted assignments would achieve using randomized experimental outcomes.The procedure partitions impressions by predicted signal and uses observed conversions for randomized instances assigned that signal, accounting for qualification-set confounding.
4 Empirical Results
Offline counterfactual evaluation favored full StageBoost for purchase uplift, while Lean StageBoost offered a latency-effective production trade-off. Online experimentation with Lean StageBoost produced a 0.08% overall GMB increase and stronger gains in the focus category.
- Counterfactual evaluation: Full StageBoost achieved the highest estimated purchase uplift, while Lean StageBoost balanced effectiveness with latency for production deployment.Lean StageBoost was selected online because of strict serving latency requirements.
- Online A/B experiment: 0.08% increase in Gross Merchandise Bought (GMB) was observed at the overall marketplace level, although it was not statistically significant.The result came from a large-scale online A/B experiment covering Urgency and Conversational placements.
- Online A/B experiment: 0.58% increase in GMB and 0.41% increase in Average Selling Price (ASP) occurred in the primary focus category.The approach was particularly effective at promoting higher-value purchases in that category.
- Production deployment: The production system invokes the model only when eligible signals outnumber available placement positions.This ensures that signals are displayed only when meaningful choices exist.
5 Conclusion and Future Work
Stageboost is a two-stage XGBoost approach for personalized signal assignment that models baseline conversion and signal-specific uplift. The paper addresses small uplift magnitudes, sparse qualification sets, and correlated signals, while identifying reinforcement learning, additional user features, and more placements as future directions.
- Conclusion: Stageboost personalizes View-Item signal recommendations by modeling both base conversion probability and signal-specific uplift.Its unified gradient boosting framework adapts recommendations to item characteristics and user context.
- Conclusion: The approach addresses extremely small uplift magnitudes, severe sparsity in rare qualification sets, and high feature correlation among signals.Training data is collected through randomized controlled trials, and evaluation uses counterfactual conversion rate estimation.
- Future Work: Future work includes reinforcement learning for online adaptation, additional user features, and extending the framework to more View-Item placements.
- Conclusion: Stageboost: Recommending Signals based on Counterfactual Estimations.