Source-linked AI summary
Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, Hongxia Yang
TL;DR
Large-scale DCG must learn from exposure-biased clicks over item collections far larger than language vocabularies, making faithful MLE fitting problematic. The paper proves a connection between contrastive learning and inverse propensity weighting, then develops CLRec and Multi-CLRec for scalable debiasing. In four-month A/B testing, CLRec consistently beat MIND across fairness and engagement metrics, while ensembling produced a +2% relative total-click improvement.
Problem
Exposure-biased clicks and extremely large candidate collections make standard MLE-based DCG vulnerable to underestimating under-explored items.
Method
The paper proves a contrastive-learning/IPW connection and develops CLRec and Multi-CLRec for efficient bias reduction in large-scale DCG.
Results
+2% relative improvement in total click number was achieved by ensembling CLRec and MIND versus MIND alone, while CLRec consistently outperformed MIND on fairness and engagement metrics.
Takeaways & Limitations
The methods were deployed as the default choice in a live system serving billions of page views daily and identified high-quality items neglected by existing systems.
Takeaways & Limitations
Explicit IPW requires a separate propensity model and suffers numerical instability and variance, while cached representations prevent back-propagation through prior-batch negatives.
Abstract
from arXiv · showhide
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning has become prevalent in industrial recommender systems. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe exposure bias and have a vocabulary several orders of magnitude larger than that of natural language, implying that MLE will preserve and even exacerbate the exposure bias in the long run in order to faithfully fit the observed samples. In this paper, we theoretically prove that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity weighting, which provides a new perspective for understanding the effectiveness of contrastive learning. Based on the theoretical discovery, we design CLRec, a contrastive learning method to improve DCG in terms of fairness, effectiveness and efficiency in recommender systems with extremely large candidate size. We further improve upon CLRec and propose Multi-CLRec, for accurate multi-intention aware bias reduction. Our methods have been successfully deployed in Taobao, where at least four-month online A/B tests and offline analyses demonstrate its substantial improvements, including a dramatic reduction in the Matthew effect.
1 INTRODUCTION
Industrial recommender systems use deep candidate generation to retrieve a few hundred relevant items from billion-scale collections, but MLE can preserve exposure bias in observed clicks. CLRec applies contrastive learning to reduce this bias, with queue-based implementations designed for scalable deployment.
- Motivation: Deep candidate generation learns item and user representations to enable fast retrieval from billion-scale collections.It narrows the corpus to a few hundred relevant entities in the first stage of a multistage recommendation pipeline.
- Problem: MLE-based DCG approximations are computationally necessary at large scale but can perpetuate exposure bias from popularity-skewed training data.Under-explored high-quality items may remain underestimated because the model fits observed clicks faithfully.
- Approach: CLRec establishes a contrastive-learning framework for debiased deep candidate generation in recommender systems.The paper connects contrastive learning with inverse propensity weighting as a bias-reduction perspective.
- Approach: A fixed-size FIFO queue accumulates positive samples and representations from previous batches for use as negatives in the next batch.The design supports efficient bias reduction while reusing computed results.
- Deployment: CLRec was deployed as the default live-system choice and consistently outperformed previous state-of-the-art baselines while recommending neglected high-quality items.The reported deployment served billions of page views each day.
2 OUR THEORECTICAL RESULTS ON CONTRASTIVE LEARNING IN DCG
The paper formulates DCG as multinomial relevance estimation, then analyzes how sampling-based contrastive losses relate to inverse propensity weighting under exposure-biased clicks. Its theorem shows that both objectives target the same KL-divergence solution, motivating contrastive debiasing without explicit propensity estimation.
- Problem Formulation: DCG learns user and item encoders whose similarity supports k-nearest-neighbor retrieval, while MLE models a multinomial distribution over items.Exact softmax computation over million- or billion-scale collections is infeasible and therefore requires sampling.
- Problem Formulation: Observed clicks reflect items exposed by a previous recommender, so MLE can leave high-quality items with few clicks under-recommended.This is characterized as exposure bias, or missing not at random.
- Sampling Objectives: Sampled softmax corrects sampling with a proposal-probability term, whereas the studied contrastive loss omits that correction and therefore optimizes a different objective.The proposal distribution is used to draw negative samples paired with each positive example.
- Contrastive Learning and IPW: The paper interprets contrastive learning as a sampling-based approximation to inverse propensity weighting for exposure-bias reduction.IPW uses exposure probabilities to model missing-not-at-random data and target oracle user preference.
- Practical Boundary: Explicit IPW is inefficient for large-scale production because it requires a separate propensity model and suffers numerical instability and variance from inverse propensities.The paper distinguishes its multinomial propensity formulation from prior work focused mainly on Bernoulli attention or position propensities.
- Theoretical Result: Theorem 1 states that contrastive-loss and IPW optima both minimize KL divergence from p_θ(y|x) to r(y|x) = p_data(y|x)/q(y|x).Here q(y|x) is the proposal or propensity distribution, and p_data(y|x) is the observed data distribution.
- Theoretical Result: Setting the proposal distribution to the old system’s propensity score can approximately reduce exposure bias through contrastive learning.This implication is stated directly as the practical meaning of the theorem.
3 QUEUE-BASED CONTRASTIVE LEARNING FOR BIAS REDUCTION
The paper proposes queue-based contrastive learning to reduce exposure bias efficiently in large-scale candidate generation, extending CLRec with intent-aware Multi-CLRec. Multi-CLRec routes items through disentangled intention prototypes and combines main and intention-specific queues to approximate smoothed propensity-based debiasing.
- CLRec and Multi-CLRec: CLRec and Multi-CLRec implement contrastive-loss debiasing with queues, avoiding explicit negative sampling while supporting large-scale candidate generation.CLRec uses a FIFO queue for negative samples; Multi-CLRec extends this design with multiple disentangled queues.
- Intent-aware debiasing: CLRec uses a main queue approximating q(y), while Multi-CLRec uses intention-specific queues to approximate q(y|user x’s current intention).Multi-CLRec clusters categories into H intentions; the system uses H = 64 intentions.
- CLRec and Multi-CLRec: Queue-based designs reduce sampling overhead and ensure that all items are sampled sometime in an epoch as negative examples.The queue also supports distributed training by avoiding explicit sampling communication costs.
- Efficiency and trade-offs: Cached negative representations prevent back-propagation through previous-batch negatives, mildly increasing convergence steps but potentially greatly reducing total runtime for expensive features.The efficiency benefit is especially relevant when negative-item features include raw images, texts, or structured data such as knowledge graphs.
- Intent-aware debiasing: Multi-CLRec routes clicked items into intention prototypes and selects the intention vector most likely to match the user’s current interest.The model uses trainable intention prototypes, routing probabilities, attention, and bias vectors shared across users.
- Multi-queue objective: The multi-queue contrastive loss combines a main queue with an intention-specific secondary queue, implicitly using q(y|intent h) + α·q(y) as its propensity score.The method also includes auxiliary losses for polarized routing, target-intention alignment, and balancing the intentions.
4 EXPERIMENT
Experiments show that CLRec reduces exposure-related popularity bias while improving diversity, online engagement, robustness, and training efficiency at industrial scale. Multi-CLRec further improves performance under changing user distributions.
- Debiasing Effects: Over 2× improvement in aggregated diversity was observed for CLRec versus sampled softmax.CLRec shifted recommendations toward under-explored items and alleviated the Matthew effect.
- Large-Scale Online Results: Over four months of A/B testing, CLRec outperformed MIND on aggregated diversity, average popularity index, click-through rate, and average dwell time.CLRec also recommended less-popular items while remaining attractive to users.
- Large-Scale Online Results: +2% relative total clicks resulted from ensembling CLRec and MIND compared with MIND alone.
- Multi-CLRec: 3% relative improvement in total clicks was achieved by Multi-CLRec while maintaining a comparable click-through rate.Multi-CLRec replaced CLRec as the default in mid-2020.
- Efficiency: CLRec’s queue-based implementation was more efficient than explicit-sampling methods because it reused computed positive representations as negatives.The design reduces computation and distributed communication costs, especially when negative-item features are expensive to encode.
- Public Benchmarks: CLRec retrieved more diverse item sets than sampled softmax on public benchmarks and explored previously under-explored items.
5 RELATED WORK
Related work covers deep candidate generation, recommender-system bias reduction, and contrastive learning. The paper situates its contribution at their intersection by connecting contrastive loss with inverse propensity weighting.
- Deep Candidate Generation: Deep candidate generation learns representations for fast retrieval, while existing methods commonly sample negative examples from proposal distributions.
- Bias Reduction and Fairness: Bias-reduction research addresses selection bias from missing-not-at-random recommendation data, but prior work mostly focuses on small-scale offline settings.
- Contrastive Learning: Contrastive learning uses self-supervised pretext tasks, and this paper provides a perspective connecting its loss to inverse propensity weighting.
6 CONCLUSION
The paper establishes a theoretical connection between contrastive learning and inverse propensity weighting, then uses it to develop CLRec and Multi-CLRec for bias reduction in large-scale candidate generation.
- The theory connects contrastive learning with inverse propensity weighting and motivates CLRec and Multi-CLRec for efficient, effective bias reduction.
A EXPERIMENTAL SETTINGS
The experiments use industrial sequential-recommendation data, distributed training, vector-based serving, and metrics covering offline accuracy, engagement, diversity, and popularity. The model uses feature-rich encoders with attention-based user representation and cosine-similarity retrieval.
- Data and Hyper-parameters: Training uses four days of platform data, with the following day’s clicks as test data and a minimum training-sequence length of five.
- Training Environment: The training environment contains four billion behavior sequences processed by 140 workers and 10 parameter servers.
- Online Serving: Serving precomputes item representations, indexes them with vector kNN, and retrieves a few hundred relevant items with approximately 10 ms query latency.
- Evaluation Metrics: Evaluation includes offline HitRate@50, online click-through rate, average dwell time, aggregate diversity, and popularity index.
- Encoders: Single-queue CLRec combines item ID, categorical features, time-bucket, and backward positional embeddings before user encoding.
- User Encoder: Simplified multi-head attention maps click embeddings to H attention-head embeddings, which weighted head aggregation combines into one user embedding.
- Similarity Function: The model uses l2-normalized embeddings and cosine similarity with temperature ρ=0.07 for contrastive learning.
B.4 Complex Pretext Tasks
The paper co-trains shared encoders on user-to-user and user-to-item pretext tasks. User-to-user prediction targets longer-term behavior, while user-to-item prediction remains necessary for item recommendation at serving time.
- The u2u task uses two sequences from the same user, split before and after a sampled timestamp.It is intended to improve long-term prediction.
- The encoders are co-trained on u2u and the original u2i task.u2i predicts a sequence’s next click and preserves the item-level recommendation objective.
- u2i remains necessary because the deployed recommender must recommend items rather than sequences.
C ADDITIONAL EMPIRICAL RESULTS
Additional empirical results examine Multi-CLRec’s interpretability. The top-K vectors selected by its user encoder represent different user intentions, with auxiliary disentanglement losses supporting this behavior.
- The analysis presents additional empirical results demonstrating Multi-CLRec’s interpretability.
- Multi-CLRec’s top-K selected vectors represent a user’s K different intentions.The example is described as randomly sampled rather than cherry-picked.
- Auxiliary losses for intention disentanglement are identified as the main reason the selected vectors represent different intentions.
C.2 Results about CLRec’s Encoders
The encoder studies evaluate positional and time features, head aggregation, and similarity choices. They find benefits from time bucket and backward positional embeddings, while WHA with cosine similarity improves HitRate and diversity in single-vector CLRec.
- Encoder feature ablations: Combining time bucket embedding with backward positional embedding benefits the encoders.The corresponding ablation studies were conducted in a small-traffic scenario.
- Head aggregation and similarity: Cosine similarity is needed with WHA to reach a higher HitRate.Table 12 compares different head aggregation strategies for the single-vector CLRec user encoder.
- Head aggregation and similarity: Combining WHA and cosine similarity enables a single-vector encoder to generate a diversified list reflecting different behavioral intents.
- Head aggregation and similarity: Without WHA, retrieved items can be dominated by one intent, producing limited diversity.
- Single-vector versus multi-vector encoders: Single-vector models can sometimes generate diverse recommendation lists, although Multi-CLRec is more controllable and reliable for balanced multiple-intention preservation.
D.1.2 Training with Biased Data: Inverse Propensity Weighting.
The section derives inverse propensity weighting for candidate generation trained on biased recommendation data. It contrasts the naïve biased estimator with IPW and motivates the multinomial policy formulation used for large-scale recommendation.
- Biased training data: Random exploration is expensive, so training data are collected under a biased old policy that recommends items according to multinomial propensities.The impression vector is one-hot, while user preferences are represented across all items.
- Naïve estimator: The naïve estimator learns a biased policy because its expected loss weights user preferences by the old policy’s recommendation probability.
- Inverse propensity weighting: Inverse propensity weighting replaces the biased exposure weighting so the estimator optimizes the ideal unbiased loss.Setting q(y|x) to the old policy propensity yields an unbiased estimator of the oracle loss.
- Policy formulation: The paper models candidate generation as a multinomial policy rather than a multivariate Bernoulli policy.The multinomial formulation does not model how many recommendations a user requests and is reported to perform better for large item sets.
D.2 Proof of Theorem 1
The proof shows that inverse propensity weighting and contrastive learning optimize toward the same reweighted target distribution. With sufficient model expressiveness, their global optima coincide across candidate supports.
- Theorem 2 states that contrastive and inverse-propensity-weighted losses both minimize KL divergence toward r(y|x) = pdata(y|x)/q(y|x).
- For contrastive learning, the candidate multiset contains the positive item and L negatives sampled from q(y|x).
- The proof relates the contrastive objective for each candidate support C to the same KL-divergence objective as inverse propensity weighting.
- If the scoring function is sufficiently expressive, minimizing over all candidate supports makes pθ(y|x) equal r(y|x) for every y.The paper justifies this expressiveness by implementing the score function as a neural network and invoking the universal approximation theorem.
- Therefore, contrastive learning and inverse propensity weighting have the same global optima under the stated expressiveness condition.