PaperlayerView exact revision on arXiv

Source-linked AI summary

Interest-aware Message-Passing GCN for Recommendation

Fan Liu, Zhiyong Cheng, Lei Zhu, Zan Gao, Liqiang Nie

arXiv:2102.10044v2cs.IR

TL;DR

GCN-based recommendation faces over-smoothing because indiscriminate high-order propagation can mix information from users with dissimilar interests. IMP-GCN restricts high-order convolution to interest-based subgraphs, and experiments show stronger performance with deeper layers and significant gains over state-of-the-art GCN methods.

  • Problem

    Indiscriminate aggregation of high-order neighbors can make users with dissimilar interests obtain similar embeddings, worsening over-smoothing in deep GCN-based recommendation.

  • Method

    IMP-GCN groups users with similar interests and their interacted items into subgraphs, using user features and graph structure to restrict high-order message passing within subgraphs.

  • Results

    IMP-GCN outperforms state-of-the-art GCN-based recommendation methods and achieves better performance with more layers, reaching improvement through 7 layers.

  • Takeaways & Limitations

    Filtering negative information from high-order propagation helps preserve user uniqueness and alleviate over-smoothing in deeper recommendation GCNs.

  • Takeaways & Limitations

    The first-order propagation includes all first-order neighbors because direct user-item interactions are treated as the most important and reliable interest information.

Abstract

from arXiv · show

Graph Convolution Networks (GCNs) manifest great potential in recommendation. This is attributed to their capability on learning good user and item embeddings by exploiting the collaborative signals from the high-order neighbors. Like other GCN models, the GCN based recommendation models also suffer from the notorious over-smoothing problem - when stacking more layers, node embeddings become more similar and eventually indistinguishable, resulted in performance degradation. The recently proposed LightGCN and LR-GCN alleviate this problem to some extent, however, we argue that they overlook an important factor for the over-smoothing problem in recommendation, that is, high-order neighboring users with no common interests of a user can be also involved in the user's embedding learning in the graph convolution operation. As a result, the multi-layer graph convolution will make users with dissimilar interests have similar embeddings. In this paper, we propose a novel Interest-aware Message-Passing GCN (IMP-GCN) recommendation model, which performs high-order graph convolution inside subgraphs. The subgraph consists of users with similar interests and their interacted items. To form the subgraphs, we design an unsupervised subgraph generation module, which can effectively identify users with common interests by exploiting both user feature and graph structure. To this end, our model can avoid propagating negative information from high-order neighbors into embedding learning. Experimental results on three large-scale benchmark datasets show that our model can gain performance improvement by stacking more layers and outperform the state-of-the-art GCN-based recommendation models significantly.

1 INTRODUCTION

GCN-based recommendation improves user and item representation learning by aggregating graph-neighbor information, but deeper propagation can cause over-smoothing. IMP-GCN addresses this by restricting high-order message passing to interest-based subgraphs and reports stronger deep-layer performance.

  • GCN-based recommendation models learn representations by iteratively aggregating information from local and high-order graph neighbors, helping exploit graph structure and alleviate sparsity.
  • Over-smoothing occurs because graph convolution acts as graph Laplacian smoothing, making node embeddings increasingly similar as more layers are stacked.
  • IMP-GCN groups users and their interacted items into subgraphs and performs high-order graph convolution only within each subgraph.
  • Its subgraph generation module identifies users with similar interests using both user features and graph structure.
  • Experiments on three large-scale real-world datasets show that IMP-GCN outperforms state-of-the-art methods, benefits from deeper layers up to 7 layers, and filters negative high-order information propagation.

2 METHODOLOGY

IMP-GCN addresses over-smoothing by restricting high-order message passing to interest-based user-item subgraphs while retaining whole-graph first-order propagation. Its unsupervised grouping module combines graph structure and user features to form these subgraphs.

  • 2.1 Recap: GCN recommendation models aggregate progressively higher-order neighbors, but indiscriminate propagation can mix users with unrelated or contradictory interests.This mixing makes embeddings increasingly similar and can neglect user uniqueness in deep structures.
  • 2.1 Recap: After 6–7 graph-convolution layers, a node can receive information from nearly all other graph nodes, intensifying embedding homogenization.The observed broad coverage helps explain why deeper conventional GCN structures suffer from over-smoothing.
  • 2.2 IMP-GCN MODEL: IMP-GCN performs first-order propagation on the whole graph but restricts high-order propagation to subgraphs containing users with similar interests and their interacted items.Users belong to one subgraph, while an item may belong to multiple subgraphs; item representations combine embeddings learned across those subgraphs.
  • 2.2.1 Interest-aware Message-passing Strategy: Within each subgraph, nodes use only neighbor information from that subgraph, preventing embeddings learned from unrelated nodes from contributing to one another.Users still receive information from all their linked items, whereas items can maintain separate embeddings for the subgraphs they join.
  • 2.2.2 Optimization: IMP-GCN targets top-n recommendation and optimizes observed-versus-unobserved user-item triplets with pairwise learning.The objective ranks an interacted positive item above an unobserved negative item for each user.
  • 2.3 Subgraph Generation Module: The subgraph-generation module treats user grouping as an unsupervised classification task using features that fuse graph structure with user ID embeddings.A two-layer network maps the fused feature to a prediction vector, whose maximum-valued position determines the user’s group.

3.1 Experimental Setup

IMP-GCN was evaluated on three benchmark datasets using 10-core filtering, an 80:10:10 per-user split, and top-20 Recall and NDCG.

  • The evaluation used Amazon-Kindle Store, Amazon-Home&Kitchen, and Gowalla datasets filtered to retain users and items with at least 10 interactions.
  • Interactions were randomly divided per user into training, validation, and testing sets with an 80:10:10 ratio.
  • Performance was measured using Recall and Normalized Discounted Cumulative Gain over each user's top 20 recommendations.
  • All models used 64-dimensional embeddings initialized with Xavier; optimization used Adam with a default learning rate of 0.001.
  • Figure 3 compares IMP-GCN and LightGCN across layers on Kindle Store and Gowalla, with IMP-GCN2, IMP-GCN3, and IMP-GCN4 denoting two, three, and four subgraphs.

3.2 Study of IMP-GCN

The layer and subgraph studies show that IMP-GCN benefits from deeper propagation than LightGCN, while performance depends on balancing interest separation against graph connectivity.

  • Effect of Layer Numbers: IMP-GCN consistently outperforms LightGCN beyond two or three layers on Kindle Store and Gowalla.
  • Effect of Layer Numbers: LightGCN peaks at three or four layers and then degrades sharply, whereas IMP-GCN continues improving with deeper structures through seven layers.
  • Effect of Subgraph Numbers: With no more than three layers, IMP-GCN2 performs best because its subgraphs let nodes reach more neighbors over short distances.
  • Effect of Subgraph Numbers: Beyond three layers, IMP-GCN3 performs best, while coarse two-subgraph grouping retains noisy propagation and finer four-subgraph grouping cuts valuable connections.
  • Effect of Subgraph Numbers: On Kindle Store, higher coverage ratios correspond to higher IMP-GCN performance, and six-layer propagation can approach whole-graph coverage without over-smoothing.

3.3 Comparison with SOTA Methods

IMP-GCN is compared with competitive recommendation baselines across three datasets, showing consistent gains over existing methods and LightGCN.

  • Baselines: The comparison covers NeuMF, HOP-Rec, CSE, GCMC, NGCF, and LightGCN alongside IMP-GCN.All methods use the same pairwise learning strategy and are tuned using validation data.
  • Overall comparison: NGCF outperforms earlier baselines by directly exploiting high-order graph connectivities in embedding propagation.The cited comparison contrasts direct high-order embedding use with methods that only enrich training data or use first-order neighbors.
  • Overall comparison: LightGCN substantially improves over NGCF by removing feature transformation and nonlinear activation modules.Its simplified formulation retains neighborhood aggregation for collaborative filtering.
  • Overall comparison: IMP-GCN consistently outperforms all baselines across the three datasets.The evaluated datasets are Amazon-Kindle Store, Amazon-Home&Kitchen, and Gowalla.
  • Overall comparison: 7.85%, 7.19%, and 3.66% relative NDCG@20 improvements over LightGCN are reported on KindleStore, Home&Kitchen, and Gowalla, respectively.The values are reported for the strongest baseline comparison on each dataset.

3.4 Ablation Study

The ablation study evaluates graph-structure information in subgraph generation and first-order propagation, finding both design choices beneficial to IMP-GCN.

  • Ablation setup: The study compares IMP-GCN with IMP-GCN_s and IMP-GCN_f to assess individual model components.IMP-GCN_s removes graph-structure information from subgraph generation, while IMP-GCN_f also replaces first-order propagation.
  • Subgraph generation: IMP-GCN outperforms IMP-GCN_s across all datasets, supporting graph-structure information in subgraph generation.The variant removes the graph-structure embedding component from the subgraph generation module.
  • Propagation components: IMP-GCN_s performs much better than IMP-GCN_f, highlighting the contribution of first-order user-item propagation.The paper attributes this to direct interaction information in collaborative filtering.

4 RELATED WORK

Related recommendation research spans collaborative filtering, deep learning, graph-based methods, and GCNs that exploit high-order user-item proximity.

  • Collaborative filtering: Model-based collaborative filtering learns user and item embeddings from interactions and has become a mainstream recommendation approach.Related variants incorporate additional information such as reviews, images, and knowledge graphs for different tasks.
  • Deep learning: Deep-learning recommenders enhance embedding learning or introduce nonlinear interaction functions, including NeuMF and Wide&Deep.These models represent a separate development line within recommendation research.
  • Graph-based recommendation: Graph-based methods exploit high-order proximity through random walks, interaction enrichment, or user-user and item-item relations.HOP-Rec enriches interactions with random walks, while related methods explore additional collaborative relations.
  • GCN-based recommendation: GCN-based recommenders learn from non-Euclidean structure; GCMC uses one convolution layer, while later models exploit broader connectivity.The paper positions IMP-GCN as restricting message passing to subgraphs containing users with similar interests.

5 CONCLUSION

The conclusion attributes IMP-GCN’s performance to interest-aware high-order convolution within subgraphs, which limits noisy propagation and supports deeper models.

  • Contribution: IMP-GCN groups users with similar interests and their interacted items into subgraphs for high-order graph convolution.A designed subgraph-generation algorithm uses user similarity to form these groups.
  • Contribution: Restricting embedding contributions to nodes within each subgraph avoids incorporating noisy high-order information.The conclusion links this design to preserving user uniqueness during propagation.
  • Conclusion: Experiments on large-scale real-world datasets report improved performance with more layers and state-of-the-art results.The paper presents distinguishing high-order neighbors as important for addressing over-smoothing in GCNs.
Loading 2102.10044v2…