PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Graph Contrastive Learning for Recommendation

Yangqin Jiang, Chao Huang, Lianghao Xia

arXiv:2305.10837v3cs.IR

TL;DR

Noisy, sparse, and skewed user behavior data, together with trial-and-error contrastive augmentations, challenge graph-based collaborative filtering. AdaGCL introduces trainable graph generative and denoising view generators, and experiments report superiority over competitive baselines across multiple datasets.

  • Problem

    Graph-based CF requires sufficient, high-quality interaction data, but recommendation behavior is noisy, sparse, and skewed, while existing contrastive views rely on burdensome augmentation selection.

  • Method

    AdaGCL uses a graph generative model and a graph denoising model as two trainable generators of adaptive contrastive views for graph-based CF.

  • Results

    AdaGCL outperforms several competitive baseline recommendation models across multiple datasets and demonstrates improved robustness against noise perturbation.

  • Takeaways & Limitations

    Adaptive contrastive views provide additional self-supervision for graph-based recommendation while addressing data noise and user data scarcity within the evaluated settings.

  • Takeaways & Limitations

    Future work is needed to explore causal factors for contrastive self-supervised signals and the model’s transferability through transfer learning.

Abstract

from arXiv · show

Graph neural networks (GNNs) have recently emerged as an effective collaborative filtering (CF) approaches for recommender systems. The key idea of GNN-based recommender systems is to recursively perform message passing along user-item interaction edges to refine encoded embeddings, relying on sufficient and high-quality training data. However, user behavior data in practical recommendation scenarios is often noisy and exhibits skewed distribution. To address these issues, some recommendation approaches, such as SGL, leverage self-supervised learning to improve user representations. These approaches conduct self-supervised learning through creating contrastive views, but they depend on the tedious trial-and-error selection of augmentation methods. In this paper, we propose a novel Adaptive Graph Contrastive Learning (AdaGCL) framework that conducts data augmentation with two adaptive contrastive view generators to better empower the CF paradigm. Specifically, we use two trainable view generators - a graph generative model and a graph denoising model - to create adaptive contrastive views. With two adaptive contrastive views, AdaGCL introduces additional high-quality training signals into the CF paradigm, helping to alleviate data sparsity and noise issues. Extensive experiments on three real-world datasets demonstrate the superiority of our model over various state-of-the-art recommendation methods. Our model implementation codes are available at the link https://github.com/HKUDS/AdaGCL.

1 INTRODUCTION

Graph-based collaborative filtering improves interaction modeling but remains vulnerable to noisy, sparse, and skewed data. AdaGCL addresses these challenges with two trainable adaptive view generators that provide higher-quality contrastive signals.

  • Motivation: Graph neural CF models propagate information over user-item interaction graphs to learn richer representations than traditional matrix factorization.Examples include PinSage, NGCF, and LightGCN.
  • Challenges: Noise from irrelevant interactions and multi-hop propagation can distort user representations, while sparse and skewed data further challenge recommendation accuracy.Directly aggregating all interaction edges may capture inaccurate user interests.
  • Challenges: Existing contrastive methods create views through random masking or noise, which may retain noisy interactions or remove important training signals.These limitations restrict the applicability and potential of contrastive learning.
  • Proposed approach: AdaGCL uses a graph generative model and a graph denoising model to create two trainable contrastive views adapted to the data distribution.The framework is designed to introduce high-quality training signals into graph-based CF.
  • Results: Experiments show that AdaGCL outperforms various baseline models on multiple datasets and addresses data noise and user data scarcity challenges.The reported evaluation covers multiple datasets rather than a single benchmark setting.

2 PRELIMINARIES AND RELATED WORK

The related work situates AdaGCL within graph-based CF and self-supervised graph learning. Prior methods use graph propagation, contrastive views, or graph autoencoding to improve recommendation representations.

  • Graph-based CF: Graph neural recommendation architectures propagate embeddings over user-item graphs and capture multi-hop connections between users and items.These architectures model complex user-item relationships through graph embeddings.
  • Graph-based CF: Fine-grained relational methods learn disentangled or behavior-aware user representations by exploiting graph-structured multi-intent information.Examples include DGCF, DCCF, and DRAN.
  • Self-supervised graph learning: Self-supervised graph learning generates auxiliary signals from graph data to improve representations for tasks including node classification and link prediction.Contrastive learning is one such self-supervised strategy.
  • Recommendation SSL: SGL creates contrastive user-item graph views through random node and edge dropout, whereas GFormer reconstructs masked interactions with a graph autoencoder.These approaches augment training data for recommendation representation learning.
  • Recommendation SSL: The broader recommendation literature also applies self-supervised objectives to attributes, subsequences, cross-domain representations, and multimodal patterns.Examples include S3-Rec, C2DSR, and SLMRec.

3 METHODOLOGY

AdaGCL has three parts: a graph message-passing encoder, an adaptive self-supervised framework with variational and denoising view generators, and a model-optimization phase.

  • Framework components: The graph message-passing encoder captures local collaborative relationships among users and items.It forms the representation-learning component of the framework.
  • Framework components: Two trainable view generators use variational and denoising graph models to produce adaptive self-supervised views.These generators form the framework’s adaptive contrastive-learning component.
  • Framework components: The final framework component performs model optimization, and Figure 1 presents the overall AdaGCL architecture.The passage identifies optimization as a distinct phase.

3.1 Local Collaborative Relation Learning

AdaGCL learns local collaborative representations by propagating user and item embeddings across multiple graph layers using normalized interaction structure, then combines layer embeddings for preference prediction.

  • Embedding propagation: The encoder is inspired by simplified graph convolution in LightGCN for modeling local collaborative relations.The method follows a graph-based CF propagation design.
  • Embedding propagation: The propagation uses aggregated neighbor vectors derived from the normalized user-item adjacency matrix and user/item degree matrices.Neighbor sets identify the items connected to each user and the users connected to each item.
  • Embedding propagation: User and item embeddings are propagated through multiple GNN layers to aggregate local neighborhood information and refine contextual representations.The layer-specific embeddings are denoted e_i,l^(u) and e_j,l^(v).
  • Prediction: Final node embeddings are obtained by summing embeddings across all layers, and a user-item inner product predicts preference.This combines information from the propagation hierarchy for recommendation scoring.

3.2 Adaptive View Generators for Graph Contrastive Learning

AdaGCL replaces manually selected graph augmentations with two trainable view generators: a graph generative model and a graph denoising model. Their complementary views support contrastive learning while filtering noisy interactions and preserving task-relevant signals.

  • Dual-View GCL Paradigm: Existing graph contrastive learning methods rely on manually selected augmentations such as random edge or node dropping.Choosing suitable view-generation methods can require tedious trial-and-error.
  • Dual-View GCL Paradigm: AdaGCL uses two distinct trainable generators—a graph generative model and a graph denoising model—to create views from different perspectives.Using distinct generators addresses the risk that identical generators produce similarly distributed views and weaken contrastive optimization.
  • Dual-View GCL Paradigm: The contrastive objective treats views of the same node as positive pairs and views of different nodes as negative pairs.Cosine similarity measures vector similarity, with τ serving as the softmax temperature.
  • Graph Generative Model: The graph generative view uses VGAE to encode graph embeddings, sample Gaussian noise, and decode a new graph.VGAE adds KL-divergence regularization, enabling more diverse generated graphs and reducing overfitting risk relative to GAE.
  • Graph Denoising Model: The denoising generator learns binary edge-selection matrices and forms each layer’s subgraph as A_l = A ⊙ M_l.Non-zero entries in M_l are penalized to reduce noisy edges, while reparameterization enables gradient-based optimization of relaxed binary selections.

3.3 Learning Task-aware View Generators

AdaGCL makes its view generators task-aware by optimizing them with recommendation supervision. BPR losses computed from the generative and denoising networks align both generated views with observed user-item preferences.

  • Task-aware Generator Optimization: The two view generators may otherwise lack optimization signals connecting their generated views to the main collaborative-filtering task.AdaGCL therefore introduces the commonly used BPR loss.
  • Task-aware Generator Optimization: BPR training uses triples containing a user, an observed positive interaction, and an unobserved negative interaction.The training set is O = {(u, i, j) | (u, i) ∈ O+, (u, j) ∈ O−}.
  • Task-aware Generator Optimization: The graph generative model is trained with BPR loss computed from node embeddings produced by the VGAE encoder.Weight-decay regularization is included through λ2.
  • Task-aware Generator Optimization: The graph denoising model is trained with BPR loss computed from embeddings produced by its denoising neural network.

3.4 Model Training

AdaGCL uses a two-level training strategy that jointly optimizes the recommendation model and self-supervised objective, then updates both adaptive view generators.

  • Upper-level Training: Upper-level training jointly optimizes the classic recommendation task and the self-supervised learning task.The main-task parameters are those of LightGCN, while λ1 and λ2 control SSL and L2 regularization strengths.
  • Lower-level Training: Lower-level training optimizes the generative and denoising view generators using their respective task-aware objectives.

3.5 Time Complexity Analysis

AdaGCL’s computational cost is decomposed across local collaborative learning, VGAE generation, denoising layers, and contrastive learning, with costs expressed using graph, batch, and embedding dimensions.

  • Component-wise Complexity: Local collaborative relation learning costs O(L×|A|×d), matching LightGCN’s complexity.L is the number of graph layers, |A| the number of interaction-graph edges, and d the embedding dimension.
  • Component-wise Complexity: The VGAE graph generative model costs O(|A| × d^2), while denoising layers cost O(L× |A| × d^2).
  • Component-wise Complexity: Contrastive learning costs O(L×B× (I+ J) ×d), where B is batch size and I and J are the numbers of users and items.

4 EVALUATION

The evaluation tests AdaGCL against diverse recommender baselines on three datasets, using standard ranking metrics and an all-rank protocol. Results and ablations examine overall performance, self-supervised learning, and the value of adaptive view generators.

  • Experimental Setup: Experiments use Last.FM, Yelp, and BeerAdvocate, split 7:2:1 for training, validation, and testing.BeerAdvocate is processed with a 10-core setting.
  • Baselines: AdaGCL is compared with matrix-factorization, neural, graph-convolutional, hypergraph, and self-supervised recommender baselines.The baselines include BiasMF, NeuMF, AutoR, GCMC, PinSage, NGCF, LightGCN, GCCF, HCCF, SHT, SLRec, SGL, NCL, and DirectAU.
  • Overall Performance Comparison (RQ1): AdaGCL outperforms the baselines under both top-20 and top-40 settings, with t-tests validating significant performance improvements.The overall comparison is reported on Last.FM, Yelp, and BeerAdvocate using Recall and NDCG.
  • Overall Performance Comparison (RQ1): Self-supervised learning improves SLRec, SGL, and NCL by adding augmented learning tasks that provide beneficial regularization based on input data.These methods use stochastic augmentation or potential neighbors in contrastive pairs, but may lose useful interaction-pattern signals.
  • Ablation Study: Random edge-drop augmentation causes significant performance decay relative to AdaGCL, while its generative and denoising generators are described as critical for superior performance.The ablation study also replaces the denoising generator with a second generative generator and removes task-aware optimization.

4.3 Model Ablation Test (RQ2)

Ablation and robustness studies show that AdaGCL’s adaptive generators, denoising view, and task-aware optimization support performance under noisy and sparse interaction data.

  • Model ablation: Removing adaptive view generators causes significant performance decay versus AdaGCL, indicating random augmentation is insufficient for informative contrastive views.The generative view preserves interaction patterns, while the denoising network filters interfering noise signals.
  • Model ablation: Removing the denoising generator leaves adaptive augmentation insufficient to eliminate inherent data noise, whereas adding denoising produces significant performance improvements.This comparison is reported for the Gen+Gen variant.
  • Model ablation: The w/o Task variant performs worse than AdaGCL on all three datasets, supporting BPR-based task-aware optimization for generating more relevant contrastive views.General-purpose auto-encoding and denoising losses were less effective for this CF setting.
  • Robustness to noise: AdaGCL exhibits smaller performance degradation than LightGCN and SGL in most noise settings when 5%–25% of interaction edges are replaced with fake edges.The experiments retrain models on graphs corrupted at five noise ratios.
  • Robustness to sparsity: AdaGCL consistently performs better across user and item sparsity groups, while sparse item interaction vectors affect performance more strongly across methods.Groups are formed by interaction counts, including users with 0–10 interactions and items with 0–5 interactions in the first groups.
  • Embedding analysis: The embedding analysis compares AdaGCL and SGL using t-SNE and KMeans visualizations of 2,000 Yelp nodes across one main and two contrastive views.The visualizations examine whether adaptive views preserve informative collaborative structure under noise.

5 CONCLUSION

The paper concludes that AdaGCL uses adaptive graph generative and denoising models to create contrastive views for recommendation. Experiments report improved robustness to noise and performance over competitive baselines, while future work targets causal factors and transferability.

  • Conclusion: AdaGCL creates contrastive views with a graph generative model and a graph denoising model for self-augmented user-item interaction modeling.The framework is presented as an adaptive view-generator approach for contrastive recommenders.
  • Conclusion: Extensive experiments on multiple datasets show improved robustness to noise perturbation and performance over several competitive baselines.The conclusion presents these findings as validation for AdaGCL’s superiority in contrastive recommendation.
  • Future work: Future work includes exploring causal factors for contrastive self-supervised signals and testing transferability through domain adaptation and multi-task learning.These directions are proposed to improve interpretability and assess transfer across settings.
Loading 2305.10837v3…