PaperlayerView exact revision on arXiv

Source-linked AI summary

Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation

Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, Quoc Viet Hung Nguyen

arXiv:2112.08679v4cs.IR

TL;DR

CL-based recommendation models improve performance, but the role of graph augmentations remains unclear. This paper finds that representation uniformity matters more, then replaces graph perturbations with uniform embedding noise, achieving better accuracy and training efficiency than graph-augmentation-based counterparts.

  • Problem

    The paper asks whether graph augmentations are necessary in contrastive learning for recommendation, since their contribution to observed performance gains remains unclear.

  • Method

    The proposed method discards graph dropout and creates contrastive views by adding controlled random uniform noise directly to node representations.

  • Results

    Across three benchmark datasets, the graph-augmentation-free method has advantages over graph-augmentation-based counterparts in recommendation accuracy and training efficiency.

  • Takeaways & Limitations

    Contrastive loss, rather than graph augmentation, is the decisive factor associated with more evenly distributed representations and improved recommendation performance.

Abstract

from arXiv · show

Contrastive learning (CL) recently has spurred a fruitful line of research in the field of recommendation, since its ability to extract self-supervised signals from the raw data is well-aligned with recommender systems' needs for tackling the data sparsity issue. A typical pipeline of CL-based recommendation models is first augmenting the user-item bipartite graph with structure perturbations, and then maximizing the node representation consistency between different graph augmentations. Although this paradigm turns out to be effective, what underlies the performance gains is still a mystery. In this paper, we first experimentally disclose that, in CL-based recommendation models, CL operates by learning more evenly distributed user/item representations that can implicitly mitigate the popularity bias. Meanwhile, we reveal that the graph augmentations, which were considered necessary, just play a trivial role. Based on this finding, we propose a simple CL method which discards the graph augmentations and instead adds uniform noises to the embedding space for creating contrastive views. A comprehensive experimental study on three benchmark datasets demonstrates that, though it appears strikingly simple, the proposed method can smoothly adjust the uniformity of learned representations and has distinct advantages over its graph augmentation-based counterparts in terms of recommendation accuracy and training efficiency. The code is released at https://github.com/Coder-Yu/QRec.

1 INTRODUCTION

The paper finds that contrastive learning improves recommendation primarily through the InfoNCE loss, which produces more evenly distributed user/item representations, while graph augmentations play a trivial role. It therefore proposes a graph-augmentation-free method using uniform representation noise and reports advantages in accuracy and training efficiency on three benchmark datasets.

  • Findings: Recommendation performance remains comparable without graph augmentations, indicating that their role is trivial in contrastive learning.The paper compares methods with and without graph augmentations and finds comparable performance.
  • Method: The proposed method discards dropout-based graph augmentation and adds random uniform noise to original representations to control representation uniformity smoothly.This creates contrastive views through representation-level data augmentation rather than graph structure perturbation.
  • Findings: InfoNCE, rather than graph augmentation, is the decisive factor behind contrastive learning’s recommendation gains.The authors associate performance with more evenly distributed user/item representations learned by optimizing InfoNCE, regardless of whether graph augmentations are applied.
  • Evaluation: Three benchmark datasets show distinct advantages over graph-augmentation-based counterparts in recommendation accuracy and model training efficiency.The study presents the method as a simple and effective alternative to cumbersome graph augmentation-based contrastive learning methods.

2 INVESTIGATION OF GRAPH CONTRASTIVE LEARNING IN RECOMMENDATION

The investigation finds that contrastive learning improves recommendation primarily by producing more evenly distributed representations, while dropout-based graph augmentations contribute little. Removing augmentation and applying CL directly on the original graph can retain or improve performance over several augmented SGL variants.

  • Necessity of graph augmentation: SGL-WA removes graph augmentation by learning contrastive representations directly from the original user-item graph.The variant is evaluated on Yelp2018 and Amazon-Book against SGL variants using node dropout, edge dropout, and random-walk augmentation.
  • Necessity of graph augmentation: All SGL variants substantially outperform LightGCN, showing that contrastive learning improves recommendation performance.The comparison includes SGL-ND, SGL-ED, SGL-RW, SGL-WA, and LightGCN.
  • Necessity of graph augmentation: SGL-WA retains remarkable gains and surpasses SGL-ND and SGL-RW, indicating that graph augmentation is not necessary for the observed improvement.The paper conjectures that node dropout and random-walk augmentation may remove key nodes and associated edges, disrupting correlated signals.
  • Representation uniformity: LightGCN produces highly clustered representations, whereas contrastive learning yields notably more even feature and density distributions.The distributions are visualized after mapping sampled user representations to normalized two-dimensional vectors on the unit hypersphere.
  • Representation uniformity: Optimizing CL minimizes cosine similarity between different node embeddings, pushing connected nodes away from high-degree hubs and producing a more even distribution.The resulting uniformity is identified as the decisive factor for recommendation performance and as an implicit way to mitigate popularity bias.

3 SIMGCL: SIMPLE GRAPH CONTRASTIVE LEARNING FOR RECOMMENDATION

SimGCL replaces graph-structure augmentations with representation-level random noise to regulate embedding uniformity efficiently. Its noise magnitude controls how far augmented representations deviate, producing more even representations and practical time-efficiency advantages over graph-augmentation methods.

  • Representation-level augmentation: SimGCL adds random noises directly to node representations instead of manipulating graph structure, providing efficient and effective contrastive augmentation.Graph-structure manipulation is described as intractable and time-consuming.
  • Representation-level augmentation: The noise satisfies a fixed L2 magnitude and same-hyperoctant constraint, limiting deviation while retaining information and variance in augmented representations.The perturbation lies on a hypersphere of radius 𝜖 and avoids large deviations from the original representation.
  • Model implementation: SimGCL uses LightGCN with differently scaled random noises imposed at each layer, skips input embeddings in final representations, and optimizes unified BPR and CL losses with Adam.Skipping the input embedding slightly improves performance in the reported setting, whereas without CL it harms LightGCN.
  • Uniformity control: Increasing 𝜖 makes SimGCL learn more evenly distributed representations, enabling direct control of representation uniformity beyond tuning 𝜆 alone.The perturbation magnitude 𝜖 directly controls the distance between augmented and original representations.
  • Uniformity control: With 𝜖=0.1, SimGCL’s representation distributions are more even than those of SGL variants and LightGCN, supporting its ability to control uniformity and debias.The comparison uses popular items and sampled users from Yelp2018 under a three-layer setting with 𝜆=0.1.
  • Time complexity: SimGCL avoids graph augmentations and adjacency construction overhead, yet is observed to be much more time-efficient than SGL-ED in practice because graph perturbation runs on CPUs.Its graph convolution cost is almost three times LightGCN’s, while SGL-ED’s theoretical graph-convolution advantage does not translate into practical efficiency.

4 EXPERIMENTAL RESULTS · 4.1 Experimental Settings

The experiments evaluate SimGCL on three public benchmark datasets under Top-N recommendation settings, using established baselines and standardized training configurations. Hyperparameters are selected through reported settings followed by grid-search fine-tuning for fair comparison.

  • 4.1 Experimental Settings: The study evaluates SimGCL on Douban-Book, Yelp2018, and Amazon-Book benchmark datasets.The datasets contain 13,024, 31,668, and 52,463 users, respectively.
  • 4.1 Experimental Settings: The datasets contain 22,347, 38,048, and 91,599 items for Douban-Book, Yelp2018, and Amazon-Book, respectively.Their interaction counts are 792,062, 1,561,406, and 2,984,108, respectively.
  • 4.1 Experimental Settings: The evaluation focuses on Top-N recommendation and discards Douban-Book ratings below 4.Douban-Book uses a 1–5 rating scale.
  • 4.1 Experimental Settings: The compared methods include Mult-VAE, DNN+SSL, BUIR, and MixGCL.They represent variational autoencoder, feature-masking contrastive, positive-only self-supervised, and graph contrastive approaches, respectively.
  • 4.1 Experimental Settings: Baseline hyperparameters follow the original papers’ best settings and are then fine-tuned with grid search.This procedure is used for a fair comparison.

4.2 SGL vs. SimGCL: From a Comprehensive Perspective

SimGCL generally matches or surpasses SGL while improving LightGCN, converging faster, and preserving near-LightGCN training efficiency. Its recommendation gains primarily reflect stronger exploration of long-tail items, whereas SGL variants occupy an intermediate position.

  • Recommendation performance: SimGCL improves LightGCN by 25.6% on Recall and 30.2% on NDCG on Amazon-Book with three layers, while SGL-ED is the strongest SGL variant.SGL-ND is the least effective variant; SGL-WA outperforms SGL-ND in most two- or three-layer cases.
  • Convergence speed: SimGCL reaches peak test performance at epochs 25, 11, and 10 on Douban-Book, Yelp2018, and Amazon-Book, versus SGL-ED at epochs 38, 17, and 14.SimGCL uses only 2/3 of the epochs required by SGL variants.
  • Running time: SGL-WA runs nearly as fast per epoch as LightGCN because it uses no graph augmentation, whereas SGL-ED is 5.7 times slower than LightGCN on Amazon-Book.SGL-ED requires two graph augmentations, with most of that computation performed on CPUs.
  • Debiasing ability: SimGCL’s improvements come from lower-popularity items, compensating for its loss on the Popular group and giving it a stronger long-tail recommendation advantage than LightGCN and SGL variants.LightGCN favors popular items, while SGL variants fall between LightGCN and SimGCL in exploring long-tail items.

4.3 Parameter Sensitivity Analysis

SimGCL’s performance rises and then declines as λ increases, with dataset-specific peaks, while ε controls representation evenness but can become harmful when excessive. The analysis identifies λ as producing more dramatic performance changes than ε under the tested settings.

  • Impact of λ: SimGCL performance initially increases with λ, then declines after peaking at 0.2 on Douban-Book, 0.5 on Yelp2018, and 2 on Amazon-Book.The experiments fix ε at 0.1 while varying λ.
  • Parameter comparison: λ produces more dramatic performance changes than ε when both are tuned over the same scope.This comparison is drawn from the reported contrast between Fig. 8 and Fig. 9.
  • 4.3.2 Impact of ε: A larger ε promotes more even representations that can reduce bias, but excessive ε hinders recommendation by obscuring high similarity between connected nodes.The study fixes λ at its dataset-specific best values before varying ε.
  • 4.3.2 Impact of ε: On all datasets, the ε performance curves behave as expected when ε is near 0.1.The supplied passage reports this observation from Fig. 9 but does not provide the complete curve details.

4.4 Performance Comparison with Other Methods

SimGCL outperforms four recent data augmentation-based methods, while the paper identifies limitations that weaken several competing approaches in the evaluated feature-free, full-node setting.

  • Comparison with Other Methods: SimGCL outperforms all four compared data augmentation-based baselines, with MixGCF ranking second.Some data augmentation-based methods are also outperformed by LightGCN in many cases.
  • Comparison with Other Methods: DNN-based recommenders struggle when datasets provide no user or item features and their learned embeddings are masked for self-supervised learning.The paper states that DNNs are effective when features are available, but cannot fulfill themselves under this setup.
  • Comparison with Other Methods: BUIR removes long-tail nodes for performance, whereas this study retains all users and items; its siamese structure may collapse on some long-tail nodes.The passage attributes this possible collapse to BUIR not using negative signals.

4.5 Performance Comparison with Different Types of Noises

The study compares uniform, positive uniform, Gaussian, and adversarial noises as alternatives for SimGCL’s data augmentation. Gaussian noise performs comparably to uniform noise, whereas adversarial noise is less effective, likely because L2 normalization makes Gaussian noise approximate a uniform distribution.

  • Performance Comparison: Gaussian-noise SimGCL shows comparable performance, while adversarial-noise SimGCL is less effective.These results are reported in Table 6.
  • Performance Comparison: L2 normalization makes standard-Gaussian noises fit a flatter distribution that approximates a uniform distribution.This approximation is offered as the possible reason for Gaussian noise’s comparable results.

5 RELATED WORK

Related work builds on GNN-based recommendation models, particularly LightGCN, and increasingly integrates contrastive learning to address data sparsity through self-supervision.

  • GNN-based Recommendation: GNNs have become widely acknowledged architectures for modeling recommendation data and underpin many recommendation models claiming state-of-the-art performance.The passage identifies GNNs as a new neural-network paradigm that advanced neural recommender systems.
  • LightGCN: LightGCN is especially popular because its simple structure delivers decent performance.LightGCN removes transformation matrices and nonlinear activation functions, a design described as efficient and effective.
  • Contrastive Learning for Recommendation: Contrastive learning is increasingly integrated with recommendation because its self-supervised nature may help address recommender systems’ data sparsity issue.Prior work applies CL to recommendation, including sequence augmentations created through random masking of attributes and items.

6 CONCLUSION

The paper finds that contrastive learning, rather than graph augmentation, drives improved recommendation by producing more evenly distributed representations that help reduce bias. It therefore develops a graph-augmentation-free contrastive learning method to regulate representation uniformity.

  • Contrastive learning is the core component of CL-based recommendation models, while graph augmentation plays only a secondary role.
  • Optimizing the contrastive learning loss produces a more even representation distribution that helps debias recommendation.
  • The paper develops a simple graph-augmentation-free contrastive learning method to regulate representation uniformity.
Loading 2112.08679v4…