PaperlayerView exact revision on arXiv

Source-linked AI summary

LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation

Xuheng Cai, Chao Huang, Lianghao Xia, Xubin Ren

arXiv:2302.08191v3cs.IRcs.LG

TL;DR

Graph contrastive recommenders need augmentation that handles sparse interactions without losing semantic structure or relying heavily on noisy heuristic views. LightGCL uses SVD-guided augmentation with global collaborative relation modeling, and experiments report state-of-the-art performance, robustness to sparsity and popularity bias, and improved efficiency.

  • Problem

    Existing graph contrastive recommendation methods rely on stochastic or heuristic augmentations that may lose useful structure, be vulnerable to noisy behaviors, and suffer from over-smoothing.

  • Method

    LightGCL uses SVD-guided augmentation alongside a GCN backbone to distill global collaborative signals and align representations while preserving interaction semantics.

  • Results

    Experiments on several public evaluation datasets report new state-of-the-art results, with additional analyses showing robustness against data sparsity and popularity bias.

  • Takeaways & Limitations

    LightGCL provides a lightweight graph contrastive learning paradigm that improves recommendation performance and training efficiency while incorporating global collaborative relations.

  • Takeaways & Limitations

    The approach assumes that SVD-reconstructed unobserved interactions have smaller weights, with 0.01 already considered large, and future work targets confounding effects.

Abstract

from arXiv · show

Graph neural network (GNN) is a powerful learning approach for graph-based recommender systems. Recently, GNNs integrated with contrastive learning have shown superior performance in recommendation with their data augmentation schemes, aiming at dealing with highly sparse data. Despite their success, most existing graph contrastive learning methods either perform stochastic augmentation (e.g., node/edge perturbation) on the user-item interaction graph, or rely on the heuristic-based augmentation techniques (e.g., user clustering) for generating contrastive views. We argue that these methods cannot well preserve the intrinsic semantic structures and are easily biased by the noise perturbation. In this paper, we propose a simple yet effective graph contrastive learning paradigm LightGCL that mitigates these issues impairing the generality and robustness of CL-based recommenders. Our model exclusively utilizes singular value decomposition for contrastive augmentation, which enables the unconstrained structural refinement with global collaborative relation modeling. Experiments conducted on several benchmark datasets demonstrate the significant improvement in performance of our model over the state-of-the-arts. Further analyses demonstrate the superiority of LightGCL's robustness against data sparsity and popularity bias. The source code of our model is available at https://github.com/HKUDS/LightGCL.

1 INTRODUCTION

LightGCL addresses sparsity and robustness challenges in graph contrastive recommendation by replacing stochastic or heuristic view generation with SVD-guided augmentation. The method injects global collaborative relations while preserving interaction semantics, and reports superior performance and improved training efficiency.

  • Motivation: Existing graph contrastive recommenders use stochastic perturbations or heuristic view generators that can lose structural information, depend on noisy behaviors, and suffer from over-smoothing.These limitations affect structural fidelity, generality, robustness, and representation quality.
  • Method: LightGCL uses singular value decomposition to distill useful interaction information and inject global collaborative context into contrastive representation alignment.The augmentation is designed to preserve important interaction semantics without two handcrafted augmented views.
  • Method: The framework combines a GCN backbone for local collaborative signals with SVD-guided augmentation for global collaborative relation analysis.This design targets user and item representations that reflect both user-specific preferences and cross-user dependencies.
  • Results: LightGCL is reported to achieve performance superiority on several real-world datasets and improved training efficiency over existing GCL-based approaches.The introduction identifies both performance and efficiency as headline outcomes.

2 RELATED WORK

Related work applies contrastive and self-supervised learning to graph recommendation and graph representation learning, often using augmentation to address sparse supervision. The cited approaches motivate LightGCL’s focus on more reliable graph augmentation.

  • Graph Contrastive Learning for Recommendation: SGL and SimGCL augment graph structure or embeddings with random dropout, which may remove important information and worsen sparsity for inactive users.These methods represent stochastic augmentation within graph contrastive recommendation.
  • Graph Contrastive Learning for Recommendation: Other graph contrastive recommenders use alternative view-generation strategies, including heuristic approaches for identifying semantic neighbors.The passage positions these methods alongside stochastic augmentation in the related-work landscape.
  • Self-Supervised Learning on Graphs: Self-supervised learning on graphs enhances node representations from unlabeled graph data and includes automated combinations of augmentation-oriented pretext tasks.AutoSSL is cited as an example of automatically combining multiple pretext tasks.

3 METHODOLOGY

LightGCL combines local GCN-based dependency modeling with SVD-guided global collaborative relation learning, then directly contrasts augmented and main-view embeddings.

  • Local Graph Dependency Modeling: A two-layer GCN aggregates neighboring information to extract local graph dependencies for user and item representations.Embedding propagation uses the normalized adjacency matrix with edge dropout to mitigate overfitting.
  • Local Graph Dependency Modeling: The final node embedding sums representations across layers, and user-item inner products predict user preferences.
  • Efficient Global Collaborative Relation Learning: LightGCL applies SVD to the normalized adjacency matrix and retains its largest q singular values to distill important global collaborative signals.The truncated factors reconstruct a low-rank approximation of the normalized adjacency matrix.
  • Efficient Global Collaborative Relation Learning: The reconstructed graph emphasizes principal components and preserves global collaborative signals by considering each user-item pair during message propagation.
  • Efficient Global Collaborative Relation Learning: Randomized SVD makes the decomposition practical for large user-item matrices by approximating the matrix range before decomposing a smaller matrix.Low-dimensional factors are stored instead of the large dense reconstructed matrix, with pre-calculation improving efficiency.
  • Simplified Local-Global Contrastive Learning: LightGCL simplifies contrastive learning by directly contrasting SVD-augmented view embeddings with main-view embeddings in the InfoNCE loss.The contrastive objective is jointly optimized with the recommendation objective, while random node dropout excludes some nodes from each batch.

4 EVALUATION

LightGCL is evaluated against diverse recommender baselines across five datasets, with analyses targeting performance, efficiency, robustness, embedding distributions, and noisy-interaction handling. Results report consistent gains over contrastive baselines, lower training complexity, robustness to sparse users and popularity bias, and balanced embedding distributions.

  • Performance Validation: LightGCL consistently outperforms all compared contrastive learning baselines, which the authors attribute to global collaborative contextual signals reducing misleading self-supervised signals.The compared contrastive methods include SGL, HCCF, and SimGCL.
  • Efficiency Study: LightGCL’s training complexity is less than half that of SimGCL on the evaluated datasets because its low-rank SVD graph encoder uses O[2q(I + J)Ld] time.The one-time SVD preprocessing cost is O(qE), while the method uses only two contrastive views.
  • Resistance Against Data Sparsity and Popularity Bias: For users with fewer than 15 interactions, LightGCL’s Recall@20 is not much below overall performance and is even higher on Gowalla, while consistently outperforming HCCF and SimGCL.This comparison is reported for Yelp and Gowalla.
  • Resistance Against Data Sparsity and Popularity Bias: LightGCL performs better in most popularity-bias cases than HCCF and SimGCL, whose performance fluctuates substantially under popularity effects.The Gowalla sparsity-group distribution also affects the aggregate Recall@20 trend because extremely sparse users are numerous.
  • Balancing Between Over-Smoothing and Over-Uniformity: t-SNE visualizations show LightGCL preserving clear community structures with reasonably dispersed within-community embeddings, balancing the over-uniformity and over-smoothing patterns of baselines.The authors report that LightGCL’s MAD lies between the two baseline types.
  • Case Study: The case study shows SVD augmentation assigning negative weight to a likely redundant car-rental interaction and preserving a smaller Arizona community for recommending minor-interest venues.The reconstructed view also assigns large weights to potential venues of interest.

5 CONCLUSION

LightGCL introduces SVD-based graph augmentation for recommendation and reports strong resistance to data sparsity and popularity bias. Experiments on public datasets show new state-of-the-art results.

  • LightGCL uses singular value decomposition to augment user-item interaction graph structures.The method is designed as a simple augmentation method within graph contrastive learning.
  • LightGCL exhibits strong ability to resist data sparsity and popularity bias.
  • Extensive experiments show that LightGCL achieves new state-of-the-art results on several public evaluation datasets.

A DETAILS OF THE BASELINES

The appendix describes baselines spanning collaborative filtering, GNN-based, self-supervised, disentangled, and hypergraph-based recommendation models. These baselines use varied neural, graph, hypergraph, masking, dropout, and contrastive-learning designs.

  • MLP-enhanced Collaborative Filtering: NCF uses a neural network to exploit non-linearity in collaborative filtering.The evaluation uses two hidden layers.
  • GNN-based Collaborative Filtering: GCCF strengthens GNN-based collaborative filtering with a residual network and reduced non-linear transformation.
  • GNN-based Collaborative Filtering: LightGCN simplifies GCN by removing embedding weight matrices and non-linear projection.
  • DGCF segments embedding vectors to represent multiple latent intentions, while HyRec uses hypergraphs to encode multi-order user-item information.
  • Self-Supervised Learning Recommender Systems: Self-supervised baselines generate signals through graph corruption, feature or parameter perturbation, masking, graph infomax, or neighborhood prediction.The listed methods include GraphCL, GRACE, GCA, MHCN, SAIL, AutoGCL, and SimGRACE.
  • Self-Supervised Learning Recommender Systems: SGL, HCCF, SHT, and SimGCL use, respectively, dropout-based views, hypergraph-global/local contrasts, hypergraph transformers, and random feature noise.The SGL-ED variant is adopted experimentally, while HCCF and SHT use 128 hyper-edges.

B PERFORMANCE COMPARISON WITH BASELINES (CONTINUED)

The continued baseline comparison reports that LightGCL consistently outperforms the additional methods omitted from the main table. The comparison covers five datasets.

  • LightGCL consistently outperforms NCF, GCCF, GraphCL, SAIL, GRACE, and AutoGCL in the appendix comparison.These baselines were not shown in Table 1 because of space limitations.
  • The continued performance comparison is conducted across five datasets.

C THEORETICAL ANALYSIS

The theoretical analysis explains how LightGCL’s local-global contrastive learning uses SVD-based global relations to update embeddings of potentially related, nonadjacent nodes. These gradients inject learned relatedness into optimization.

  • LightGCL’s local-global contrastive objective maximizes similarity between embeddings of potentially related nodes using SVD-based global relation learning.
  • Unlike vanilla InfoNCE, the local-global objective assigns gradients to SVD-related nonadjacent nodes.For these nodes, the additional gradients pull their embeddings toward the local representation and inject SVD-learned relatedness into optimization.

D.1 ADJACENCY MATRIX NORMALIZATION

The section derives computational costs for normalization, graph convolution, contrastive training, and score calculation. LightGCL’s layer-wise graph operations are compared with the higher cost of traditional three-view contrastive methods.

  • Normalization: Sparse COO normalization visits each nonzero entry, giving O(E) complexity; SGL normalizes two augmented graphs with ρE edges each, yielding O(2ρE) per batch.The SGL cost varies slightly depending on ρ.
  • Graph convolution: Multiplying a sparse adjacency matrix by a dense matrix costs O(Ed), and two graph-convolution multiplications per layer require O(2Ed).
  • Graph convolution: Across L layers, graph convolution requires O(2ELd), while three-view SGL and SimCGL require O(12ELd).For SGL, the complexity can vary somewhat with ρ.
  • Factorized computation: LightGCL’s factorized operations take O(2q(I + J)d) after using the precalculated factor product.The two terms arise from multiplying by the sparse-side and item-side factors.
  • Score calculation: For a batch of B users, positive and negative item score calculations together require O(2Bd).
  • Contrastive loss: Per-layer InfoNCE loss calculation costs O(Bd) for the numerator and O(BMd) for the denominator, with an additional factor of L.M denotes the total number of nodes in the batch.

E PERFORMANCE RESULTS UNDER THE NEW SETTING

The paper changes its sampling setting after finding that the original procedure produced sub-optimal performance because graph interactions were not fully sampled and trained. Table 6 reports LightGCL and SimGCL under the new setting.

  • Sampling setting: The original setting sampled users first, then fixed numbers of positive and negative items for each user.
  • Sampling setting: The authors found this sampling method sub-optimal because graph interactions were not fully sampled and trained.
  • Performance results: Table 6 compares LightGCL and SimGCL under the new experimental setting.
Loading 2302.08191v3…