PaperlayerView exact revision on arXiv

Source-linked AI summary

LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, Meng Wang

arXiv:2002.02126v4cs.IRcs.LG

TL;DR

The paper examines why GCN components effective for graph classification may be unnecessary or harmful in collaborative filtering. It proposes LightGCN, which retains neighborhood aggregation and layer combination, and reports substantial improvements over NGCF under identical settings.

  • Problem

    Existing GCN-based collaborative-filtering models lack thorough ablations establishing which inherited operations contribute to recommendation performance.

  • Method

    LightGCN learns user and item embeddings through linear neighborhood propagation and combines embeddings from all layers using a weighted sum.

  • Results

    About 16.0% relative improvement on average over NGCF is reported under exactly the same experimental setting.

  • Takeaways & Limitations

    Feature transformation and nonlinear activation can be discarded for collaborative filtering, while LightGCN provides a simpler model with reported training and generalization advantages.

  • Takeaways & Limitations

    The paper flags that GCN recommender models incorporating auxiliary information may also suffer from issues similar to those identified in NGCF.

Abstract

from arXiv · show

Graph Convolution Network (GCN) has become new state-of-the-art for collaborative filtering. Nevertheless, the reasons of its effectiveness for recommendation are not well understood. Existing work that adapts GCN to recommendation lacks thorough ablation analyses on GCN, which is originally designed for graph classification tasks and equipped with many neural network operations. However, we empirically find that the two most common designs in GCNs -- feature transformation and nonlinear activation -- contribute little to the performance of collaborative filtering. Even worse, including them adds to the difficulty of training and degrades recommendation performance. In this work, we aim to simplify the design of GCN to make it more concise and appropriate for recommendation. We propose a new model named LightGCN, including only the most essential component in GCN -- neighborhood aggregation -- for collaborative filtering. Specifically, LightGCN learns user and item embeddings by linearly propagating them on the user-item interaction graph, and uses the weighted sum of the embeddings learned at all layers as the final embedding. Such simple, linear, and neat model is much easier to implement and train, exhibiting substantial improvements (about 16.0\% relative improvement on average) over Neural Graph Collaborative Filtering (NGCF) -- a state-of-the-art GCN-based recommender model -- under exactly the same experimental setting. Further analyses are provided towards the rationality of the simple LightGCN from both analytical and empirical perspectives.

1 INTRODUCTION

The introduction argues that GCNs are unnecessarily complex for collaborative filtering: feature transformation and nonlinear activation add little value, while removing them improves accuracy. It motivates LightGCN, which retains only essential GCN components for recommendation and substantially outperforms NGCF under identical settings.

  • Collaborative filtering: Collaborative filtering predicts user-item interactions from learned user and item embeddings, commonly incorporating users’ interaction histories.Matrix factorization represents users and items with latent features, while methods such as SVD++ augment user IDs with interaction history.
  • Motivation: NGCF applies GCN-style feature transformation, neighborhood aggregation, and nonlinear activation to exploit high-hop neighbors for collaborative filtering.NGCF is presented as a state-of-the-art CF model inspired by GCN propagation rules.
  • Motivation: GCN operations inherited by NGCF are burdensome and may be unjustified because GCN was originally designed for node classification on attributed graphs.The introduction specifically questions whether these designs are useful for collaborative filtering, where the task setting differs from attributed-graph classification.
  • Ablation findings: Feature transformation and nonlinear activation contribute nothing to NGCF’s effectiveness, and removing them yields significant accuracy improvements in controlled experiments.The ablations use the same data splits and evaluation protocol, isolating the effect of the two operations.
  • Contributions: LightGCN simplifies GCN-based recommendation by retaining only essential components, and it substantially improves over NGCF under the same experimental setting.The paper also provides in-depth analyses of LightGCN’s rationale.

2 PRELIMINARIES

This section introduces NGCF and uses controlled ablations to show that feature transformation and nonlinear activation are unnecessary or harmful for collaborative filtering. Removing both operations improves recall and reduces training difficulty, motivating simpler recommendation models.

  • NGCF: NGCF propagates user and item ID embeddings through the interaction graph using nonlinear activation and trainable feature-transformation matrices.After L layers, it concatenates the resulting L + 1 embeddings and uses an inner product for prediction.
  • Ablation setup: The ablation study removes feature transformation, nonlinear activation, or both while keeping NGCF’s hyperparameters fixed and evaluating the 2-layer setting.The variants are NGCF-f, NGCF-n, and NGCF-fn, evaluated on the Gowalla and Amazon-Book datasets.
  • Ablation findings: Removing feature transformation consistently improves NGCF, whereas removing nonlinear activation has little effect when feature transformation remains included.Nonlinear activation becomes harmful when feature transformation is disabled.
  • Ablation findings: 9.57% relative improvement on recall is achieved by NGCF-fn over NGCF when both feature transformation and nonlinear activation are removed.The results indicate that both operations impose a rather negative overall effect on NGCF.
  • Training analysis: NGCF-fn maintains lower training loss throughout training, and this lower loss aligns with better testing recall and recommendation accuracy.The evidence attributes NGCF’s deterioration to training difficulty rather than overfitting.
  • Interpretation: Although NGCF has higher representation power in theory, feature transformation and nonlinear activation worsen practical generalization, underscoring the need for rigorous ablation studies.The cited theoretical comparison notes that identity feature-transformation matrices can recover NGCF-f.

3 METHOD

Section 3 introduces LightGCN as a simpler, effective alternative motivated by NGCF’s burdensome design. It presents the model, analyzes the rationale for its simplicity, and describes recommendation training.

  • Method motivation: LightGCN is designed to include the most essential GCN ingredients for recommendation while remaining light and effective.The design goal follows the finding that NGCF is heavy and burdensome for collaborative filtering.
  • Method motivation: Simplicity aims to improve interpretability, training and maintenance, model analysis, and subsequent revision toward greater effectiveness.These advantages are presented as severalfold benefits of a simpler model.
  • Section organization: The section presents LightGCN, analyzes the rationale behind its simple design, and explains model training for recommendation.These are the section’s three stated components.

3.1 LightGCN

LightGCN simplifies graph convolution for collaborative filtering to weighted neighborhood aggregation, removing feature transformation, nonlinear activation, and self-connection. It propagates trainable initial embeddings across the user-item graph, combines all layer embeddings with uniformly weighted coefficients, and uses inner products for recommendation scores.

  • Model architecture: LightGCN retains only weighted neighborhood aggregation, abandoning feature transformation and nonlinear activation for collaborative filtering.Its architecture also removes self-connection and performs normalized neighbor aggregation toward the next layer.
  • Model architecture: LightGCN aggregates only connected neighbors, while layer combination captures the effect of self-connections.Unlike many graph convolution operations, it does not integrate the target node itself during propagation.
  • Embedding propagation: The only trainable parameters are the user and item embeddings at layer 0; higher-layer embeddings are computed by propagation.After K propagation layers, embeddings from every layer are combined into final user and item representations.
  • Layer combination: Setting αk uniformly to 1/(K + 1) generally performs well, avoiding a separate component for optimizing layer-combination weights.The coefficients αk determine each layer embedding’s contribution and may otherwise be manually tuned or learned.
  • Prediction and matrix formulation: The prediction score is the inner product of final user and item representations, which serves as the recommendation ranking score.The model is also expressed using the user-item interaction matrix and its graph adjacency matrix for implementation and analysis.

3.2 Model Analysis

Model analysis shows that LightGCN’s layer combination makes adjacency self-connections unnecessary and enables it to recover APPNP-like propagation with controllable oversmoothing. Its linear propagation also yields an interpretable smoothing mechanism aligned with collaborative-filtering assumptions.

  • Relation to SGCN: LightGCN’s weighted layer combination is essentially equivalent to propagating embeddings with self-connections, so adding self-connections to its adjacency matrix is unnecessary.This establishes the connection between LightGCN and SGCN.
  • Relation to APPNP: By setting its layer weights α_k appropriately, LightGCN fully recovers APPNP’s prediction embedding and supports long-range modeling with controllable oversmoothing.This gives LightGCN the same oversmoothing-related strength attributed to APPNP.
  • Relation to APPNP: APPNP’s self-connection is likewise redundant in LightGCN because the weighted sum of embeddings from different propagation layers provides the same effect.The redundancy follows from the same layer-combination property used in the SGCN analysis.
  • Embedding Smoothness: In a two-layer LightGCN, second-order neighbors influence a user through shared interacted items, producing smoothing that reflects collaborative-filtering notions of user similarity.The influence increases with more co-interacted items and less popular shared items, while less-active neighboring users exert greater influence.

3.3 Model Training

LightGCN trains only its initial-layer embeddings with BPR and Adam, keeping optimization as simple as matrix factorization. It avoids dropout because L2 regularization suffices, while learned layer-combination coefficients provide negligible gains.

  • Training objective: LightGCN’s only trainable parameters are the 0-th-layer embeddings, so its model complexity matches standard matrix factorization.Training uses the Bayesian Personalized Ranking (BPR) pairwise loss.
  • Training objective: LightGCN uses the BPR loss with Adam optimization in a mini-batch training manner.BPR encourages observed entries to receive higher predictions than unobserved counterparts; λ controls L2 regularization strength.
  • Regularization and simplicity: LightGCN omits dropout because it has no feature-transformation weight matrices, making L2 regularization on embeddings sufficient to prevent overfitting.This reduces tuning compared with NGCF, which requires node-dropout and message-dropout ratios plus layer-wise embedding normalization.
  • Layer combination: Learning layer-combination coefficients on training data does not improve LightGCN, while learning them from validation data improves performance by less than 1%.The authors attribute the limited training-data benefit to insufficient signal for learning coefficients that generalize to unknown data.

4 EXPERIMENTS · 4.1 Experimental Settings

The experiments follow NGCF’s settings to ensure fair comparison, using NGCF as the main baseline alongside Mult-VAE and GRMF variants. They evaluate LightGCN through comparisons, ablations, embedding analyses, and hyper-parameter studies under specified implementation settings.

  • 4 EXPERIMENTS: The experimental program compares LightGCN with NGCF, other state-of-the-art methods, ablations, embedding analyses, and hyper-parameter studies.These components are presented in Sections 4.2–4.5 after the experimental settings.
  • 4.1 Experimental Settings: The study closely follows NGCF’s settings and uses author-provided datasets, including train/test splits, to reduce workload and preserve fairness.Gowalla and Amazon-Book match NGCF exactly, while Yelp2018 is a revised version.
  • 4.1 Experimental Settings: NGCF is the main competing method because it outperforms GC-MC, PinSage, NeuMF, CMN, MF, and HOP-Rec under the same evaluation protocol.Consequently, those methods are not compared again.
  • 4.1.1 Compared Methods: Mult-VAE is evaluated as an item-based variational-autoencoder collaborative-filtering method using its released code and a 600 →200 →600 architecture.Dropout is tuned in [0, 0.2, 0.5], and β is tuned in [0.2, 0.4, 0.6, 0.8].
  • 4.1.1 Compared Methods: GRMF smooths matrix factorization with a graph-Laplacian regularizer, uses BPR loss for item recommendation, and searches λд in [1e−5, 1e−4, ..., 1e−1].A normalized variant, GRMF-norm, is also evaluated.
  • 4.1.1 Compared Methods: GRMF and GRMF-norm benchmark embedding smoothing through Laplacian regularization, whereas LightGCN performs embedding smoothing in the predictive model.The normalized variant adds normalization to the graph Laplacian.
  • 4.1.1 Compared Methods: All models use embedding size 64 and Xavier initialization; LightGCN uses Adam with learning rate 0.001, mini-batch size 1024, and λ searched over {1e−6, 1e−5, ..., 1e−2}.Amazon-Book uses mini-batch size 2048 for speed, and the optimal λ is 1e−4 in most cases.

4.2 Performance Comparison with NGCF · 4.3 Performance Comparison with State-of-the-Arts

LightGCN substantially outperforms NGCF and competing methods, while its gains over NGCF are especially pronounced on recall and ndcg. Performance generally improves with additional propagation layers, though gains diminish, and LightGCN uses uniform layer weights in the state-of-the-art comparison to avoid over-tuning.

  • 4.2 Performance Comparison with NGCF: 16.56% higher recall: LightGCN reaches 0.1830 on Gowalla with four layers versus NGCF’s reported 0.1570.Across three datasets, average recall and ndcg improvements are 16.52% and 16.87%, respectively.
  • 4.2 Performance Comparison with NGCF: LightGCN outperforms NGCF by a large margin in all reported cases.The comparison records performance across layers 1 to 4.
  • 4.2 Performance Comparison with NGCF: Removing feature embedding and dropout from NGCF does not make it better than LightGCN, suggesting these operations may be useless for NGCF-fn.NGCF-fn is the NGCF variant that removes feature embedding in graph convolution and dropout.
  • 4.2 Performance Comparison with NGCF: Increasing layers improves performance, but benefits diminish; moving from zero to one layer gives the largest gain, while three layers are satisfactory in most cases.The zero-layer setting corresponds to matrix factorization.
  • 4.3 Performance Comparison with State-of-the-Arts: LightGCN consistently outperforms competing methods on all three datasets, demonstrating effectiveness with simple designs.The comparison reports the best score obtained for each method.
  • 4.3 Performance Comparison with State-of-the-Arts: LightGCN can be further improved by tuning α_k, but the comparison uses a uniform setting of 1/(K+1) to avoid over-tuning.The α_k tuning evidence is referenced through Figure 4.
  • 4.3 Performance Comparison with State-of-the-Arts: Among the baselines, Mult-VAE exhibits the strongest performance and is better than GRMF and NGCF.GRMF is on a par with NGCF and better than MF, supporting the utility of embedding smoothness with a Laplacian regularizer.
  • 4.3 Performance Comparison with State-of-the-Arts: GRMF-norm improves over GRMF on Gowalla but provides no benefits on Yelp2018 and Amazon-Book.GRMF-norm adds normalization to the Laplacian regularizer.

4.4 Ablation and Effectiveness Analyses

Ablation analyses show that layer combination and symmetric square-root normalization are central to LightGCN’s effectiveness. The results also support embedding smoothness as an explanation for its strong collaborative-filtering performance, while excessive propagation causes oversmoothing.

  • Layer Combination: LightGCN’s performance improves as layers increase, remaining strong through four layers, whereas LightGCN-single peaks at layer 2 and drops sharply by layer 4.LightGCN-single benefits from first- and second-order neighbors but suffers from oversmoothing with higher-order neighbors; layer combination avoids this degradation.
  • Symmetric Sqrt Normalization: Symmetric square-root normalization on both sides is generally best, while removing either side substantially reduces performance.Without normalization, training becomes numerically unstable and produces NaN issues.
  • Symmetric Sqrt Normalization: L1 normalization at the left side only is generally the second-best setting and corresponds to degree-based stochastic-matrix normalization.Symmetric normalization helps square-root normalization but harms L1 normalization.
  • Embedding Smoothness: Embedding smoothness is proposed as a key reason for LightGCN’s effectiveness because two-layer propagation smooths users connected through overlapping interacted items.The smoothing strength between users is defined by Equation (14), with an analogous definition for item embeddings.
  • Embedding Smoothness: The 2-layer LightGCN-single outperforms matrix factorization by a large margin in recommendation accuracy and has much lower smoothness loss.Smoothness loss uses the L2 norm to remove the impact of embedding scale; lower loss indicates smoother embeddings.

4.5 Hyper-parameter Studies

LightGCN is relatively insensitive to the L2 regularization coefficient λ and remains better than NGCF even when λ=0. Its optimal λ is dataset-dependent, while values larger than 1e−3 cause a rapid performance drop.

  • L2 regularization coefficient λ: LightGCN is relatively insensitive to λ and outperforms NGCF even when λ=0.Unlike NGCF, which uses dropout to prevent overfitting, LightGCN has only 0-th-layer ID embeddings as trainable parameters, making it easy to train and regularize.
  • L2 regularization coefficient λ: The optimal λ is 1e−3 for Yelp2018, 1e−4 for Amazon-Book, and 1e−4 for Gowalla.These dataset-specific values are reported for LightGCN.
  • L2 regularization coefficient λ: When λ is larger than 1e−3, performance drops quickly, indicating that overly strong regularization negatively affects normal model training.The study identifies λ as the most important hyper-parameter to tune after the learning rate when applying LightGCN to a new dataset.

5 RELATED WORK

Prior collaborative-filtering methods learn user and item embeddings from interaction data, while other approaches enrich representations with historical-item information and attention. A separate research line exploits user-item graph structure through label propagation and graph neural networks, motivating LightGCN’s simplification in relation to prior GNN analyses.

  • Collaborative filtering: Collaborative filtering commonly parameterizes users and items as embeddings learned by reconstructing historical user-item interactions.Matrix factorization, NCF, and LRML are cited as examples of this embedding-based paradigm.
  • Collaborative filtering: FISM and SVD++ represent users using weighted averages of historical-item ID embeddings, while attention mechanisms model differing item contributions to personal interest.These methods move beyond using only user and item ID information.
  • Graph-based recommendation: Graph-based recommendation methods exploit user-item structure through label propagation and graph neural networks that use high-hop neighbors to guide embedding learning.ItemRank propagates user preference scores over the graph, encouraging connected nodes to have similar labels.
  • GNN simplification: Insights into unnecessary GCN complexity inspired LightGCN, while SGCN simplifies GCNs by removing nonlinearities and collapsing multiple weight matrices into one.The passage distinguishes the models because they target different tasks, making their simplification rationales different.

6 CONCLUSION AND FUTURE WORK

The conclusion argues that GCNs for collaborative filtering are unnecessarily complicated and presents LightGCN as a simpler design centered on light graph convolution and layer combination. It also suggests that LightGCN offers broader inspiration as graph-based recommendation models increasingly exploit explicit entity relations.

  • Conclusion: LightGCN consists of two essential components: light graph convolution and layer combination.The model discards feature transformation and nonlinear activation from standard GCNs because they increase training difficulty.
  • Conclusion: Light graph convolution removes feature transformation and nonlinear activation, simplifying GCNs for collaborative filtering.These operations are described as standard in GCNs but unnecessarily complicate training.
  • Future Work: Graph-based recommendation models can explicitly exploit relations among entities, unlike factorization machines that model those relations implicitly.The passage presents this explicit relational modeling as an advantage over traditional supervised learning schemes such as factorization machines.
Loading 2002.02126v4…