PaperlayerView exact revision on arXiv

Source-linked AI summary

XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation

Junliang Yu, Xin Xia, Tong Chen, Lizhen Cui, Nguyen Quoc Viet Hung, Hongzhi Yin

arXiv:2209.02544v4cs.IR

TL;DR

The paper asks what drives the gains of contrastive learning in recommendation and whether graph augmentations are necessary. It identifies InfoNCE-induced representation uniformity as central, then proposes noise-based XSimGCL with a streamlined architecture. Experiments on four large, sparse benchmarks show that XSimGCL outperforms graph augmentation-based counterparts in recommendation effectiveness and efficiency.

  • Problem

    The mechanism behind contrastive learning's recommendation gains remains unclear, including whether graph augmentations are necessary.

  • Method

    XSimGCL replaces graph augmentations with controllable noise-based embedding augmentation and uses a shared single pass for recommendation and contrastive learning.

  • Results

    XSimGCL outperforms graph augmentation-based counterparts in recommendation effectiveness and training efficiency across four large, highly sparse benchmark datasets.

  • Takeaways & Limitations

    InfoNCE-driven uniformity is more important than elaborate graph augmentations, while XSimGCL provides a simpler alternative for contrastive recommendation.

  • Takeaways & Limitations

    Graph augmentation choices can require lengthy trial and error, and contrastive recommendation models may incur nearly triple the computational cost of conventional models.

Abstract

from arXiv · show

Contrastive learning (CL) has recently been demonstrated critical in improving recommendation performance. The underlying principle of CL-based recommendation models is to ensure the consistency between representations derived from different graph augmentations of the user-item bipartite graph. This self-supervised approach allows for the extraction of general features from raw data, thereby mitigating the issue of data sparsity. Despite the effectiveness of this paradigm, the factors contributing to its performance gains have yet to be fully understood. This paper provides novel insights into the impact of CL on recommendation. Our findings indicate that CL enables the model to learn more evenly distributed user and item representations, which alleviates the prevalent popularity bias and promoting long-tail items. Our analysis also suggests that the graph augmentations, previously considered essential, are relatively unreliable and of limited significance in CL-based recommendation. Based on these findings, we put forward an eXtremely Simple Graph Contrastive Learning method (XSimGCL) for recommendation, which discards the ineffective graph augmentations and instead employs a simple yet effective noise-based embedding augmentation to generate views for CL. A comprehensive experimental study on four large and highly sparse benchmark datasets demonstrates that, though the proposed method is extremely simple, it can smoothly adjust the uniformity of learned representations and outperforms its graph augmentation-based counterparts by a large margin in both recommendation accuracy and training efficiency. The code and used datasets are released at https://github.com/Coder-Yu/SELFRec.

1 INTRODUCTION

The paper revisits why contrastive learning improves recommendation, finding that InfoNCE-driven representation uniformity matters more than graph augmentations. It introduces XSimGCL, a noise-based and computationally streamlined alternative that improves effectiveness and efficiency.

  • Motivation and findings: Graph augmentations are not essential: models without them can remain competitive, although some augmentations help and poor choices may degrade performance.Selecting effective augmentations requires trial and error, while repeatedly constructing augmented graphs adds training expense.
  • Motivation and findings: CL improves recommendation primarily by learning more uniformly distributed user and item representations, which reduces popularity bias and promotes long-tail items.The analysis identifies InfoNCE as the primary contributor to these gains, while graph augmentations play a secondary role.
  • Prior approach: SimGCL removes graph augmentations and uses uniform noise for representation-level augmentation, making representation uniformity more controllable.SimGCL was proposed as a graph-augmentation-free contrastive learning method based on the earlier findings.
  • XSimGCL: XSimGCL builds on noise-based augmentation and streamlines SimGCL through a shared single pass for recommendation and contrastive learning.This design addresses the extra forward/backward passes and full-graph training requirements that hinder the scalability of contrastive recommendation models.
  • XSimGCL: XSimGCL surpasses SimGCL in effectiveness and efficiency and is presented as an alternative to graph augmentation-based counterparts.The study evaluates the method on four large, highly sparse benchmark datasets.

2 REVISITING GRAPH CL FOR RECOMMENDATION

This section revisits graph-based contrastive recommendation by examining SGL’s losses, encoder, augmentations, and the representation distributions associated with performance gains. The evidence indicates that InfoNCE-driven uniformity matters more than graph augmentation itself, although excessive uniformity can harm recommendation performance.

  • Contrastive Recommendation with Graph Augmentations: SGL jointly optimizes recommendation and contrastive losses, using BPR for recommendation and InfoNCE for contrastive learning.The contrastive task is auxiliary and its effect is controlled by λ.
  • Contrastive Recommendation with Graph Augmentations: SGL uses LightGCN to learn representations from the user-item graph, with normalized adjacency propagation across layers.Graph augmentations replace the normalized adjacency matrix, and the augmented matrix is reconstructed every epoch.
  • Necessity of Graph Augmentations: SGL-WA is competitive with augmented SGL variants and outperforms SGL-ND across three benchmark datasets, showing that graph augmentation is less decisive than expected.All graph augmentation-based variants outperform LightGCN, while edge-dropout variants hold a slight advantage over SGL-WA.
  • Necessity of Graph Augmentations: Graph augmentation quality is inconsistent: node dropout can remove hubs and disconnect subgraphs, whereas edge dropout perturbs the original graph less severely.Finding useful augmentations requires extensive trial and error, and random selection may degrade recommendation performance.
  • Uniformity Is What Really Matters: LightGCN produces clustered representations and popularity-biased distributions, while CL-based models produce more even distributions across users and items.Hot users and items have similar distributions, and cold users tend to cluster near hot items in the LightGCN representation space.
  • Uniformity Is What Really Matters: Optimizing InfoNCE pushes different node representations apart, and improved uniformity is hypothesized to drive gains by mitigating popularity bias and promoting long-tail items.The relationship is not linear: excessive uniformity can weaken alignment of interacted pairs and similar users or items, reducing recommendation performance.

3 PROPOSED METHOD

XSimGCL replaces graph augmentations with controllable random noise in the embedding space and streamlines SimGCL through cross-layer contrast and a shared encoder pass. Its design targets more uniform representations while reducing graph-encoding cost.

  • 3.1 Noise-Based Augmentation: Random uniform noises perturb node embeddings within a small magnitude constraint, retaining most original information while introducing view differences.The perturbations are constrained by ∥∆∥2 = ϵ and are designed to spread representations through the embedding space.
  • 3.2.2 XSimGCL - Simpler Than Simple: XSimGCL contrasts embeddings from different layers because they share information but differ in aggregated neighbors and added noises.The design follows a mutual-information sweet spot and applies noises only during training, reverting to ordinary representations at test time.
  • 3.2.2 XSimGCL - Simpler Than Simple: A shared computation pass merges recommendation and contrastive encoding, reducing XSimGCL to one forward/backward pass per mini-batch.The joint architecture streamlines SimGCL’s separate encoding processes while preserving its perturbed representation learning.
  • 3.2.2 XSimGCL - Simpler Than Simple: Larger ϵ promotes greater representation uniformity, providing explicit control over the deviation of augmented representations from their originals.The paper evaluates this relationship using uniformity and converged Recall@20 across different ϵ values.
  • 3.3 Theoretical Analysis with Graph Spectrum: Cross-layer contrast is theoretically motivated by graph-spectrum analysis that harnesses high-frequency information in representation-level augmentations.LightGCN’s stacked convolution attenuates high-frequency information, while differences across layers provide stronger high-frequency variation for contrast.
  • 3.4 Complexity: XSimGCL has LightGCN-level graph-encoding expense, whereas SGL-ED and SimGCL require nearly three times LightGCN’s encoding cost.Without contrastive-task computation, XSimGCL spends only one-third of SimGCL’s graph-encoding expense and requires neither graph augmentations nor extra encoders.

4.1 Experimental Settings

The experiments evaluate XSimGCL and SimGCL on four large public datasets using standardized data splits and tuned hyperparameters. Comparisons include graph contrastive, feature-masking, positive-only, and hard-negative recommendation methods.

  • Datasets: Experiments use Yelp2018, Amazon-kindle, Alibaba-iFashion, and Amazon-Electronics, split into training, validation, and test sets at a 7:1:2 ratio.Hyperparameters are searched on validation data before retraining on the merged training and validation sets.
  • Baselines: The comparison includes DNN+SSL, which uses feature masking for contrastive learning in a DNN-based recommendation architecture.DNN+SSL adopts an architecture similar to the graph contrastive recommendation paradigm.
  • Baselines: BUIR provides a positive-only self-supervised baseline with target and online network branches.Its architecture consists of separate target and online networks.
  • Baselines: MixGCL synthesizes hard negatives for graph collaborative filtering through hop mixing and embedding interpolation.The method uses hop mixing to construct contrastive negatives.
  • Baselines: NCL uses a prototypical contrastive objective to model correlations between users or items and their contexts.The method is described as a recent contrastive recommendation model.
  • Hyperparameters: General settings use 64-dimensional Xavier-initialized embeddings, Adam with learning rate 0.001, L2 regularization 10^-4, and batch size 2048.The contrastive temperature is generally τ = 0.2, with τ = 0.15 for XSimGCL on Yelp2018.

4.2 SGL vs. XSimGCL: A Comprehensive Perspective

Across performance, efficiency, and long-tail recommendation, XSimGCL generally outperforms graph-augmentation methods while retaining a lightweight architecture. The results associate its gains with noise-based contrastive learning and more evenly distributed representations.

  • Performance Comparison: SGL-ED and SGL-RW outperform SGL-ND, supporting InfoNCE as the main contributor while heuristic graph augmentations may degrade performance.
  • Performance Comparison: XSimGCL and SimGCL remain relatively stable with deeper models, whereas LightGCN declines on three datasets, suggesting reduced over-smoothing.
  • Performance Comparison: XSimGCL and SimGCL achieve the best and second-best performance in almost all cases, while XSimGCL improves on SimGCL.
  • Training Efficiency: SGL-ED requires nearly four times LightGCN’s per-batch time, while XSimGCL has similar per-batch cost but reaches convergence fastest.
  • Training Efficiency: XSimGCL’s total training cost is about half that of SGL-WA, and its cross-layer contrast is linked to faster convergence.
  • Long-tail Recommendation: LightGCN favors popular items, whereas XSimGCL and SimGCL perform better on most long-tail groups, with XSimGCL strong on iFashion.

4.3 Hyperparameter Investigation

XSimGCL’s performance depends on the contrastive coefficient, noise magnitude, and contrasted layer. Moderate noise is consistently beneficial, while the best layer pairing varies across datasets but can be tuned with limited effort.

  • Hyperparameters: XSimGCL has three key hyperparameters: λ controls contrastive-task weight, ϵ controls noise magnitude, and l* selects the contrasted layer.
  • Noise and Contrastive Weight: The best values of λ and ϵ found in the 2-layer setting also apply to other settings.
  • Noise and Contrastive Weight: XSimGCL performs best when ϵ is 0.05, 0.1, or 0.2 across datasets, while zero, very small, or very large noise reduces performance.
  • Layer Selection: The optimal contrasted-layer pair varies by dataset but usually combines the final layer with one preceding layer.
  • Layer Selection: With a 3-layer setting, XSimGCL usually reaches its best performance, reducing manual effort for tuning l*.

4.4 Applicability Investigation

Noise-based contrastive learning extends beyond LightGCN to several common recommendation backbones. Signed uniform noise performs best among the tested noise variants, while adversarial noise can substantially hurt performance.

  • Backbone Applicability: Noise-based CL is evaluated with Linear MF, vanilla GCN, and a two-tower DNN, demonstrating applicability across these backbones.
  • Noise Types: Signed uniform-noise XSimGCL outperforms positive-uniform, Gaussian, and adversarial-noise variants.
  • Noise Types: Adversarial noise unexpectedly causes a large performance drop, indicating that only some noise distributions are helpful.
  • Noise Types: XSimGCL outperforms its positive-noise variant, showing that the sign constraint contributes to its effectiveness.

5 RELATED WORK

The related work situates XSimGCL within graph neural network recommendation and the growing use of contrastive learning for self-supervised learning under data sparsity.

  • Graph Neural Networks: GNNs have become routine in recommender systems because they model user behavior data effectively.
  • Contrastive Learning: Contrastive learning has attracted attention for learning from massive unlabeled data and has been integrated into recommendation.
  • Contrastive Learning: S3-Rec is identified as the first work combining contrastive learning with sequential recommendation.

6 CONCLUSION

The paper identifies InfoNCE loss as the main source of contrastive recommendation gains and proposes XSimGCL, a simple noise-based alternative to graph augmentation. Experiments on four large, sparse datasets support its effectiveness as an ultralight model.

  • InfoNCE loss accounts for most performance gains, while elaborate graph augmentations play a secondary role.
  • Noise-based augmentation smoothly adjusts representation uniformity through contrastive learning and promotes long-tail items.
  • XSimGCL provides an ultralight architecture and serves as an alternative to graph augmentation-based recommendation models.
  • Experiments on four large and highly sparse datasets demonstrate that XSimGCL is an effective alternative to graph augmentation-based counterparts.
Loading 2209.02544v4…