PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-supervised Graph Learning for Recommendation

Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, Xing Xie

arXiv:2010.10783v4cs.IRcs.LG

TL;DR

Graph-based recommenders are limited by sparse supervision, degree bias against long-tail items, and sensitivity to noisy interactions. SGL adds self-supervised contrastive learning through graph augmentations, improving accuracy—especially for long-tail items—and robustness to interaction noise across three benchmark datasets.

  • Problem

    Graph-based recommenders face sparse supervision, skewed degree distributions that disadvantage long-tail items, and vulnerability to noisy interactions.

  • Method

    SGL supplements supervised recommendation with self-supervised contrastive learning over multiple graph-augmented views of users and items.

  • Results

    Across three benchmark datasets, SGL improves recommendation accuracy, particularly for long-tail items, and strengthens robustness against noisy interactions.

  • Takeaways & Limitations

    SGL provides a model-agnostic way to address degree bias and interaction noise while adding auxiliary supervision for graph-based recommendation.

Abstract

from arXiv · show

Representation learning on user-item graph for recommendation has evolved from using single ID or interaction history to exploiting higher-order neighbors. This leads to the success of graph convolution networks (GCNs) for recommendation such as PinSage and LightGCN. Despite effectiveness, we argue that they suffer from two limitations: (1) high-degree nodes exert larger impact on the representation learning, deteriorating the recommendations of low-degree (long-tail) items; and (2) representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme further enlarges the impact of observed edges. In this work, we explore self-supervised learning on user-item graph, so as to improve the accuracy and robustness of GCNs for recommendation. The idea is to supplement the classical supervised task of recommendation with an auxiliary self-supervised task, which reinforces node representation learning via self-discrimination. Specifically, we generate multiple views of a node, maximizing the agreement between different views of the same node compared to that of other nodes. We devise three operators to generate the views -- node dropout, edge dropout, and random walk -- that change the graph structure in different manners. We term this new learning paradigm as \textit{Self-supervised Graph Learning} (SGL), implementing it on the state-of-the-art model LightGCN. Through theoretical analyses, we find that SGL has the ability of automatically mining hard negatives. Empirical studies on three benchmark datasets demonstrate the effectiveness of SGL, which improves the recommendation accuracy, especially on long-tail items, and the robustness against interaction noises. Our implementations are available at \url{https://github.com/wujcan/SGL}.

1 INTRODUCTION

The paper introduces Self-supervised Graph Learning (SGL) to address sparse supervision and skewed degree distributions in GCN-based recommendation. SGL adds node self-discrimination through augmented views and contrastive learning, improving accuracy and robustness when implemented on LightGCN.

  • Limitations: GCN-based recommenders face sparse supervision because observed user-item interactions cover only a small fraction of the interaction space.This limits the signal available for learning high-quality representations.
  • Limitations: GCN-based recommenders also suffer from skewed data distributions, which can bias representation learning toward high-degree nodes and hurt long-tail items.The introduction identifies this as a separate limitation alongside sparse supervision.
  • SGL: SGL uses node self-discrimination as an auxiliary task, combining data augmentation that generates multiple node views with contrastive learning.The auxiliary task distills additional signal from the input data itself.
  • SGL: SGL supplements observed-interaction supervision, reduces degree bias through augmentation operators, and improves robustness by exposing nodes to different local structures and neighborhoods.The introduction particularly identifies edge dropout as reducing high-degree nodes’ influence.
  • Evaluation: SGL is model-agnostic, is implemented on LightGCN, and improves recommendation accuracy—especially for long-tail items—while enhancing robustness against interaction noises on three benchmark datasets.The paper also states that SGL theoretically encourages learning from hard negatives through the temperature hyper-parameter in the softmax loss.

2 PRELIMINARIES

The preliminaries define GCN-based collaborative filtering on a user-item bipartite graph whose edges are observed interactions. They then describe neighborhood aggregation, representation readout, inner-product prediction, and supervised optimization with BPR selected as the main task.

  • GCN-based Collaborative Filtering: GCN-based collaborative filtering represents users and items as nodes in a bipartite graph, with observed interactions forming its edges.The graph is G = (V, E), where V = U ∪ I and E = O+.
  • GCN-based Collaborative Filtering: At each layer, a node aggregates neighbor representations and combines them with its own representation, encoding increasingly higher-order neighbors.After L layers, representations encode L-order neighbors in the graph.
  • GCN-based Collaborative Filtering: Final node representations can use the last layer, concatenation, or a weighted sum before prediction.These are the common readout designs listed in the preliminaries.
  • Supervised Learning Loss: An inner product between user and item representations predicts how likely a user is to adopt an item and supports fast retrieval.The prediction layer is built on the final representations.
  • Supervised Learning Loss: Training uses observed interactions as supervision through point-wise objectives or pairwise BPR, with BPR chosen as the main supervised task.BPR compares an observed item against an unobserved item for the same user.

3 METHODOLOGY

SGL augments supervised graph recommendation with self-supervised contrastive learning over multiple graph-derived views of each node. The methodology combines three structural augmentations with multi-task optimization, analyzes automatic hard-negative mining, and evaluates computational complexity.

  • SGL framework: SGL supplements the supervised recommendation task with an auxiliary self-supervised task that constructs supervision from correlations within the input graph.The framework reinforces node representation learning through self-discrimination.
  • Graph augmentation: Node dropout, edge dropout, and random walk generate multiple correlated views by changing graph structure in distinct ways.Node dropout removes nodes and connected edges; edge dropout retains partial neighborhood connections and improves robustness to noisy interactions; random walk changes graph structure across layers.
  • Contrastive pretext task: Contrastive learning treats views of the same node as positive pairs and views of different nodes as negative pairs, encouraging within-node consistency and between-node divergence.The method follows SimCLR and uses cosine similarity with a temperature parameter.
  • Multi-task optimization: SGL jointly optimizes the classical recommendation objective and the self-supervised learning objective through multi-task training.The self-supervised task is combined with the classical GCN recommendation model without adding model parameters.
  • Theoretical analysis: With decreasing temperature, influential negatives approach similarity 0.9 and their contributions approach e8, enabling SGL to automatically mine hard negatives.Hard negatives have similarity 0 < x ≤ 1, whereas easy negatives have −1 ≤ x < 0.
  • Complexity analysis: 3.7x larger is the time complexity of SGL-ED (alternative) than LightGCN on Yelp2018 when the keep probability is 0.8.The passage describes this overhead as acceptable in light of the reported convergence-speed improvement.

4 EXPERIMENTS

The experiments evaluate SGL’s superiority and effectiveness through three research questions covering top-K recommendation performance, the benefits of self-supervised learning, and the influence of different settings.

  • RQ1: The experiments compare SGL’s top-K recommendation performance with state-of-the-art collaborative filtering models.This addresses RQ1.
  • RQ2: The experiments investigate the benefits of applying self-supervised learning in collaborative filtering.This addresses RQ2.
  • RQ3: The experiments examine how different settings influence the effectiveness of SGL.This addresses RQ3.

4.1 Experimental Settings

Experiments use three benchmark datasets, a 7:1:2 interaction split, and all-ranking evaluation reporting Recall@20 and NDCG@20. SGL variants are compared with established collaborative-filtering baselines under standardized training and tuned hyperparameters.

  • Datasets: Experiments use Yelp2018, Amazon-Book, and Alibaba-iFashion, with 10-core filtering for the first two and 300k sampled users for the sparse third dataset.Alibaba-iFashion retains all sampled users’ fashion-outfit interactions.
  • Evaluation: Interactions are split into training, validation, and testing sets at a 7:1:2 ratio.Testing follows the all-ranking protocol for top-K recommendation, reporting average Recall@K and NDCG@K with K=20.
  • Compared methods: SGL is compared with NGCF, LightGCN, and Mult-VAE, while previously validated MF, NeuMF, GC-MC, and PinSage baselines are excluded.The SGL implementations on LightGCN are SGL-ND, SGL-ED, and SGL-RW, using Node Dropout, Edge Dropout, and Random Walk.
  • Implementation: All models are trained from scratch with Xavier initialization, Adam at learning rate 0.001, mini-batches of 2048, and the same early-stopping strategy as NGCF and LightGCN.SGL inherits optimal shared hyperparameters, while its unique λ1, τ, and ρ values are tuned over specified candidate ranges.

4.2 Performance Comparison (RQ1)

SGL generally outperforms LightGCN and other baselines, supporting the benefit of adding self-supervised learning to recommendation. Within SGL, edge dropout performs best most often, while random walk consistently exceeds node dropout across the three datasets.

  • Most SGL implementations outperform LightGCN by a large margin, indicating the benefit of supplementing recommendation with self-supervised learning.
  • SGL-ED achieves the best performance in 10 of 18 cases within the SGL family.
  • SGL-RW outperforms SGL-ND across all three datasets, suggesting edge-dropout-like operators better capture inherent graph-structure patterns.
  • SGL-ED consistently outperforms all evaluated baselines across the board.
  • LightGCN performs better than NGCF and Mult-VAE, while Mult-VAE is on par with NGCF and LightGCN on Alibaba-iFashion and outperforms NGCF on Amazon-Book.

4.3 Benefits of SGL (RQ2)

SGL-ED benefits recommendation across long-tail exposure, training efficiency, and robustness to noisy interactions. It converges substantially faster than LightGCN and degrades less under increasing interaction noise, with stronger robustness on Yelp2018.

  • Long-tail recommendation: LightGCN disproportionately recommends high-degree items, with the 10-th group contributing 39.72%, 39.92%, and 51.92% of total Recall across three datasets despite representing only 0.83%, 0.83%, and 0.22% of item spaces.SGL shows potential to alleviate this long-tail recommendation issue.
  • Training efficiency: SGL reaches its best performance at the 18-th and 16-th epochs on Yelp2018 and Amazon-Book, whereas LightGCN requires 720 and 700 epochs, respectively.The faster convergence is attributed to InfoNCE using multiple negative samples and dynamic hard negative mining.
  • Training efficiency: InfoNCE enables SGL to learn from multiple negative samples, while LightGCN’s BPR loss uses one negative sample and limits its perception field.With a proper τ, dynamic hard negative mining supplies meaningful, larger optimization gradients.
  • Robustness to noises: As adversarial negative interactions increase from 5% to 20%, SGL’s performance degradation remains lower than LightGCN’s, and the gap becomes more apparent.Comparing augmented node views helps SGL identify useful graph patterns and reduce dependence on particular edges.
  • Robustness to noises: SGL is more robust to noise on Yelp2018 than on Amazon-Book, possibly because Amazon-Book’s greater sparsity makes noisy data more influential on its graph structure.The testing set remains unchanged while adversarial examples are added to training.

4.4 Study of SGL (RQ3)

The study examines SGL’s temperature, pretraining strategy, and negative-sample design. Results favor moderate temperature tuning, joint optimization over pretraining alone, and type-specific negatives within mini-batches.

  • Temperature τ: Increasing τ to 1.0 worsens performance and slows convergence by weakening discrimination between hard and easy negatives.Conversely, τ=0.1 also hurts performance because a few negatives dominate optimization, reducing the benefit of multiple negative samples.
  • Pretraining: Self-supervised pretraining provides a better initialization for LightGCN, but joint training achieves better recommendation performance.Joint training benefits from mutual enhancement between representations learned by the main and auxiliary tasks.
  • Negative samples: SGL-ED-batch outperforms SGL-ED-merge, showing that negative samples should distinguish users and items.SGL-ED-batch separates users and items as negative views, whereas SGL-ED-merge treats all nodes within a mini-batch as negatives.

5 RELATED WORK

The paper situates its work among model- and graph-level graph-based recommendation methods and generative or contrastive self-supervised learning. It distinguishes its approach from prior supervised graph recommendation and limited SSL-for-recommendation efforts.

  • Graph-based recommendation: Graph-based recommendation methods span model-level designs, evolving from random walks to GCN propagation and attention-enhanced GCNs.Model-level methods focus on mining the user-item graph; the passage also identifies graph-level methods as a separate category.
  • Graph-based recommendation: Existing graph-based recommendation methods use supervised training, whereas this work explores self-supervised learning as a new direction.The paper characterizes self-supervised learning as orthogonal to the established supervised paradigm.
  • Self-supervised learning: Self-supervised learning broadly includes generative models that reconstruct inputs and contrastive models that compare samples through Noise Contrastive Estimation.Generative approaches may intentionally add noise to improve robustness, while contrastive approaches typically require multiple sample views.
  • Self-supervised learning: Graph SSL methods learn node or graph representations using mutual information, contrastive comparisons, or GCN-based extensions.Examples include InfoGraph, DGI, Hu et al.’s GCN extension, and Kaveh et al.’s node- and graph-level contrastive learning.
  • SSL for recommendation: Only limited work combines SSL with recommendation, including S3-Rec for sequential recommendation and another multi-task approach using a two-tower DNN encoder.The paper contrasts these efforts with its graph-based recommendation setting.

6 CONCLUSION AND FUTURE WORK

The paper presents SGL as a model-agnostic framework that supplements supervised recommendation with self-supervised learning and graph-based augmentations. Future work targets tighter SSL integration, counterfactual learning, pre-training and fine-tuning, and long-tail recommendation.

  • Contributions: SGL supplements supervised recommendation with self-supervised learning on user-item graphs through three graph-structure-based data augmentations for an auxiliary contrastive task.The framework is described as model-agnostic and motivated by limitations of graph-based recommendation under supervised learning.
  • Future Work: Future work will further integrate self-supervised learning with recommendation, including counterfactual learning beyond stochastic graph-structure selection.The authors characterize this work as an initial attempt and identify counterfactual learning as a new perspective.
  • Future Work: The authors plan to pre-train recommendation models on transferable user patterns across domains or datasets and fine-tune them on upcoming domains or datasets.This direction is intended to support universal, transferable representations for recommendation.
  • Future Work: Another future direction is using self-supervised learning to address long-tail recommendation and improve recommender-model generalization and transferability.The authors state that developing SGL may help realize these benefits.

A GRADIENT OF INFONCE LOSS W.R.T. NODE REPRESENTATION

This section derives Equation (12) for the user contrastive loss by decoupling cosine-similarity normalization and applying the scalar-by-vector chain rule.

  • Contrastive-loss derivation: The derivation starts from the user contrastive loss with cosine similarity and decouples normalization to expose its superiority.The decoupling is motivated by prior work.
  • Gradient calculation: The resulting gradient is calculated using the chain rule of scalar-by-vector differentiation.
Loading 2010.10783v4…