Source-linked AI summary
Debiased Contrastive Learning for Sequential Recommendation
Yuhao Yang, Chao Huang, Lianghao Xia, Chunzhen Huang, Da Luo, Kangyi Lin
TL;DR
Sparse, short, and noisy behavior sequences limit sequential recommendation, while existing contrastive learning methods inadequately address popularity bias and disentangle conformity from genuine interest. DCRec combines adaptive conformity-aware cross-view contrastive learning with sequential and collaborative relation modeling, and experiments report consistent superiority over state-of-the-art sequential recommendation methods. Its theoretical analysis further motivates dynamically reducing conformity-view influence for interest-driven interactions, while future work remains on automatically searching neural parameters.
Problem
Neural sequential recommenders depend on sufficient, semantically rich interaction sequences, and existing contrastive methods inadequately address popularity bias and the separation of conformity from genuine interest.
Method
DCRec uses adaptive conformity-aware cross-view contrastive learning, a multi-channel conformity weighting network, and unified sequential and collaborative views to model transitions and cross-sequence user dependencies.
Results
DCRec consistently outperforms state-of-the-art sequential recommendation methods on several real-world datasets while mitigating popularity bias.
Takeaways & Limitations
Debiased contrastive learning can reduce popularity bias in data augmentation while helping sequential recommendation discover users’ genuine interests from conformity.
Takeaways & Limitations
Without adaptive weighting, fixed positive and negative contributions make it difficult to discriminate among diverse samples, especially for interest-driven interactions.
Abstract
from arXiv · showhide
Current sequential recommender systems are proposed to tackle the dynamic user preference learning with various neural techniques, such as Transformer and Graph Neural Networks (GNNs). However, inference from the highly sparse user behavior data may hinder the representation ability of sequential pattern encoding. To address the label shortage issue, contrastive learning (CL) methods are proposed recently to perform data augmentation in two fashions: (i) randomly corrupting the sequence data (e.g. stochastic masking, reordering); (ii) aligning representations across pre-defined contrastive views. Although effective, we argue that current CL-based methods have limitations in addressing popularity bias and disentangling of user conformity and real interest. In this paper, we propose a new Debiased Contrastive learning paradigm for Recommendation (DCRec) that unifies sequential pattern encoding with global collaborative relation modeling through adaptive conformity-aware augmentation. This solution is designed to tackle the popularity bias issue in recommendation systems. Our debiased contrastive learning framework effectively captures both the patterns of item transitions within sequences and the dependencies between users across sequences. Our experiments on various real-world datasets have demonstrated that DCRec significantly outperforms state-of-the-art baselines, indicating its efficacy for recommendation. To facilitate reproducibility of our results, we make our implementation of DCRec publicly available at: https://github.com/HKUDS/DCRec.
1 INTRODUCTION
Sequential recommenders model future interactions from historical behavior, but sparse, short, and noisy sequences limit neural representation learning. DCRec addresses popularity bias and user conformity by combining conformity-aware contrastive learning with sequential and collaborative relation modeling.
- Sequential recommenders predict future item interactions from users’ historical behavior sequences.
- Neural sequence models capture evolving interests but rely on sufficient interaction data and semantically rich sequences, making sparsity, short sequences, and noise challenging.
- Existing contrastive methods augment sequences or item features and align representations across augmented views or semantically positive pairs.
- Popularity bias can confound augmentation when users share popular items for different reasons, such as genuine interest versus general preference for popular topics.The Reddit motivating case contrasts users subscribing to the popular “nba” topic alongside niche basketball topics or other popular topics.
- The model’s motivation is to capture user interest and conformity separately because popularity-biased methods can produce inaccurate recommendations.
- DCRec integrates conformity and interest disentanglement with contrastive learning, combining sequential transitions and inter-sequence user dependencies through popularity-aware augmentation.Its conformity weighting network uses three semantic channels to separate conformity from noisy item interactions.
2 METHODOLOGY
DCRec combines Transformer-based sequential encoding with graph-based transition and collaborative views to address sparse, short, noisy sequences and popularity bias. Its adaptive conformity-aware contrastive learning uses interaction-level signals to separate conformity from genuine interest.
- Task Formulation: DCRec formulates sequential recommendation as predicting a user’s next item from their temporally ordered interaction sequence.Sequences are padded to mitigate variable lengths, and the last interacted item is used as the training label.
- Sequential Pattern Encoding: A Transformer-based sequential pattern encoder incorporates positional embeddings, multi-head self-attention, and feed-forward transformations to model temporal context.The encoder maps item sequences into latent representations while capturing correlations between items.
- Unifying Sequential and CF Views: DCRec unifies sequential item transitions and collaborative user-item interactions to transfer user-wise knowledge when short sequences lack contextual signals.This addresses the limitation that sequences with very few items provide insufficient information for sequence encoders.
- Unifying Sequential and CF Views: The model constructs an item transition graph from adjacent sequence items and a co-interaction graph from user-item co-occurrence, then applies graph neural message passing.Transition edges retain item-transition frequency, while top-k filtering keeps highly relevant co-interaction connections.
- Adaptive Cross-View Contrastive Learning: DCRec uses cross-view contrastive learning with a multi-channel conformity weighting network to adapt regularization strength and disentangle user conformity from genuine interest.The conformity estimate uses interaction-level signals derived from graph structure and is incorporated into adaptive augmentation.
3 EVALUATION
The evaluation compares DCRec with sequential, graph-based, and self-supervised baselines across public datasets and specialized cold-start, item-sparsity, ablation, and conformity analyses. DCRec outperforms baselines for cold-start users and most sparse-item groups, while case studies support its conformity estimates.
- Experimental Setup: Experiments use four public datasets from Reddit, Amazon, and MovieLens, with leave-one-out evaluation using HR@N and NDCG@N for N values 1, 5, and 10.The compared methods include non-GNN, graph-based, and self-supervised sequential recommenders.
- RQ1: Overall Performance: Graph-based models generally outperform non-GNN models, but their gains are less significant or negative on sparse Beauty and Sports data.The authors attribute this limitation to data scarcity and noise affecting graph construction.
- RQ2: Cold-Start Users: DCRec outperforms the strongest baselines on cold-start users using HR@1 and NDCG@5.The authors attribute this result to conformity-aware contrastive learning that transfers global transition and collaborative information while mitigating popularity bias.
- RQ2: Item Sparsity: DCRec outperforms baselines in the first four of five item-sparsity groups, indicating stronger performance for less popular items.Lower group numbers contain items with fewer user interactions, while larger group numbers indicate more popular items.
- RQ3: Key Components: The ablation study removes transition-graph contrastive learning, co-interaction-graph contrastive learning, all contrastive learning, or adaptive contrastive learning in separate variants.These variants are designed to examine the contribution of DCRec’s key components.
- RQ4: Case Study: MovieLens case studies assign conformity degrees of 0.68 and 0.25 to interactions respectively consistent with other users’ themes and diverse across genres.The first case is interpreted as conformity, whereas the second is interpreted as authentic interest with less commonality among users.
4 RELATED WORK
Related work includes neural sequential recommenders based on CNNs, RNNs, and attention mechanisms, alongside contrastive-learning methods for sequential recommendation and graph-based recommendation.
- Sequential Recommendation: Neural sequential recommenders use CNNs, RNNs, Transformers, or attention mechanisms to model relations and evolving interests in behavior sequences.Examples include Caser, GRU4Rec, and BERT4Rec.
- Self-Supervised Recommendation: Contrastive-learning approaches augment sequences or learn agreement between semantically positive sequence or item pairs for recommendation.Examples include CL4SRec, DuoRec, and related self-supervised methods.
- Motivation for DCRec: The related methods motivate DCRec’s use of contrastive learning across item semantic views while leaving popularity bias and conformity disentanglement as central concerns.The paper frames conformity as potentially entangling users’ real interests with observed behavior.
5 CONCLUSION
The paper concludes that DCRec discovers users’ genuine interests from conformity, improves sequential recommendation, and mitigates popularity bias. It reports superior results over baselines and identifies automatic neural-parameter search as future work.
- Conclusion: DCRec discovers users’ genuine interests from conformity to enhance sequential recommendation while mitigating popularity bias.The conclusion describes this as the role of its debiased contrastive learning framework.
- Conclusion: Comprehensive experiments show DCRec achieving superior results compared with other baselines.The conclusion presents this as evidence of the model’s effectiveness.
- Future Work: Future work could investigate automatically searching neural parameters together with interest and conformity disentanglement.The stated goal is further improving the sequential recommender.
A.1 The Learning Process of DCRec
DCRec builds transition and co-interaction graph views, encodes user sequences, and creates conformity-aware contrastive objectives. It then fuses three item-representation views and optimizes recommendation, contrastive, and conformity-distribution losses jointly.
- Inputs and Outputs: The algorithm takes all users’ item sequences as input and outputs the overall training loss for backpropagation.Each user sequence is represented temporally as a sequence of interacted items.
- Graph Views: DCRec constructs an item-transition graph and an item co-interaction graph, then applies graph convolution to produce transition-pattern and co-interaction embeddings X and Z.The co-interaction adjacency matrix is defined as AG_c = R^T R.
- Sequence and Augmentation: The model encodes each user sequence into H_u from sequential patterns and masks interacted items to derive an augmented co-interaction graph.These operations provide the sequential and augmented collaborative views used later in training.
- Conformity-Aware Contrastive Learning: Interaction-level conformity weights ω(u,v) constrain the distribution of conformity scores before conformity-aware contrastive learning.The algorithm computes a conformity regularization loss L_w and uses the weights in subsequent contrastive objectives.
- Conformity-Aware Contrastive Learning: Contrastive learning aligns H with X using ω and X with Z using ψ(u,v) = 1 − ω(u,v), producing losses L_u and L_v.The two weighted objectives distinguish the contributions of the model’s views during augmentation.
- View Aggregation and Training: DCRec fuses H, X, and Z into final item representations p_v, computes recommendation loss L_rec, and trains with L = L_rec + λ1(L_u + L_v) + λ2(L_w).The overall objective combines recommendation, contrastive, and conformity-distribution losses.
A.2.1 Hyperparameter Sensitivity.
DCRec’s performance depends on carefully chosen co-interaction size, conformity-weight mean, and contrastive temperature. Across four datasets, moderate settings performed best, while excessive neighborhood size introduced noise.
- Top co-interaction size: Performance generally rises then falls as k increases from 2 to 10, with the best performance achieved at k=4.Larger k adds collaborative signals initially but can later introduce less-relevant noise.
- Conformity-weight mean: The best conformity-weight means are μ_c=0.4 or 0.5 across the four datasets.Because μ_c characterizes average user conformity, the paper recommends adjusting it for each dataset’s user distribution.
- Contrastive temperature: The best contrastive-learning temperature settings are reported closer to 1.0.The authors speculate that higher τ reduces the negative impact of inaccurate hard negatives in sequential recommendation.
A.2.2 Quality of Learned Item Embedding.
The paper evaluates DCRec’s learned item embeddings by visualizing their distribution on the Beauty dataset. Compared with sequential baselines, DCRec produces a more even and uniform embedding distribution.
- Visualization method: Item embeddings are visualized on the Beauty dataset using 2-D KDE graphs.The visualization uses t-SNE and Gaussian kernel density estimation over all items.
- Embedding quality: DCRec’s item embeddings are more evenly and uniformly distributed than those learned by other sequential baselines.This comparison is based on the embedding distributions shown in Figure 8.
- Interpretation: The analysis presents embedding distribution as evidence of the quality of the representations learned by DCRec.The section explicitly frames the visualization as demonstrating the superiority of the learned item embeddings.
A.3 Formula Derivation Details
This section derives Equations 14–15 by analyzing how positive and negative samples contribute to contrastive-model learning. The derivation starts from a normalized-vector contrastive objective and concludes by recovering the target equations.
- Derivation setup: The derivation begins with the single-interaction contrastive-learning objective expressed using normalized vectors.The two contrastive views provide normalized representations denoted by h̄ and x̄.
- Gradient analysis: The gradient with respect to h_v is decomposed into left and right terms for separate analysis.The text then derives each term individually before combining them.
- Final result: The derivation uses the unit matrix I and ultimately obtains the expressions corresponding to Equations 14–15.The final statement identifies the derived result as equivalent to Equations 14–15.