Source-linked AI summary
Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach
Lei Chen, Le Wu, Richang Hong, Kun Zhang, Meng Wang
TL;DR
Graph-based CF models face complexity from nonlinear transformations and over-smoothing in deeper layers, particularly with sparse user-item interactions. The paper removes nonlinearities and adds residual preference learning, yielding a linear model that is easier to train and experimentally effective and efficient.
Problem
GCN-based CF models face training difficulty from nonlinear activations and over-smoothing that limits deeper graph convolution on sparse user-item data.
Method
LR-GCCF uses linear embedding propagation and residual user-item preference learning at each graph convolution layer.
Results
LR-GCCF is a wide linear model that is easier to train and more time efficient than nonlinear GCN-based models.
Takeaways & Limitations
Removing nonlinear transformations and learning residual preferences are effective design choices for graph-based recommendation.
Takeaways & Limitations
Residual learning is presented as a hypothesis about optimizing residual ratings and alleviating over-smoothing, rather than as an established guarantee.
Abstract
from arXiv · showhide
Graph Convolutional Networks (GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering (CF) based Recommender Systems (RS), by treating the user-item interaction behavior as a bipartite graph, some researchers model higher-layer collaborative signals with GCNs. These GCN based recommender models show superior performance compared to traditional works. However, these models suffer from training difficulty with non-linear activations for large user-item graphs. Besides, most GCN based models could not model deeper layers due to the over smoothing effect with the graph convolution operation. In this paper, we revisit GCN based CF models from two aspects. First, we empirically show that removing non-linearities would enhance recommendation performance, which is consistent with the theories in simple graph convolutional networks. Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data. The proposed model is a linear model and it is easy to train, scale to large datasets, and yield better efficiency and effectiveness on two real datasets. We publish the source code at https://github.com/newlei/LRGCCF.
Introduction
The paper identifies two unresolved issues in graph-based collaborative filtering: unnecessary nonlinear transformations and over-smoothing in deeper GCNs. It revisits these models with linear propagation and residual preference learning.
- GCN-based CF models represent user-item behavior as a bipartite graph to learn higher-order collaborative signals.
- Nonlinear transformations add complexity to GCN-based CF, raising whether they are necessary for recommendation.
- Deeper graph convolutions over-smooth embeddings, making higher-layer neighbors less distinguishable, especially with sparse user-item interactions.
- The proposed approach removes nonlinearities and learns residual user-item preferences at each layer to preserve lower-layer uniqueness.
- Linear residual graph convolution is presented as easier to train and suitable for scaling to large recommendation datasets.
Preliminaries and Related Work
The preliminaries define graph convolution as repeated neighborhood propagation followed by nonlinear transformation, then review its simplification and use in recommendation. CF models apply these operations to learned user and item embeddings on a bipartite graph.
- A graph is represented by nodes V and adjacency matrix A, with neighbor sets defined by nonzero adjacency entries.
- Graph Convolutional Networks: GCNs recursively propagate node features through neighbors and then apply a nonlinear transformation at each layer.
- Graph Convolutional Networks: Feature propagation can be interpreted as Laplacian smoothing of the previous layer’s features.
- Graph Convolutional Networks: Simple Graph Convolution removes nonlinear transformations, reducing GCNs to iterative feature propagation with few parameters and improved scalability.
- Graph Convolutional based Recommendation: In CF, users and items form two entity sets with implicit-feedback interactions represented by a rating matrix.
- Graph Convolutional based Recommendation: GCN-based recommendation feeds free user and item embeddings into propagation and transformation layers to inject up to K-th-order user-item connections.
- Graph Convolutional based Recommendation: GC-MC is a special case of NGCF with K = 1, modeling only first-order user-item connectivity.
- Although deeper neural networks can approximate complex functions, increasing depth often complicates training and can degrade performance.
Collaborative Filtering
LR-GCCF applies linear embedding propagation and residual preference learning to user-item bipartite graphs, targeting nonlinear training complexity and over-smoothing in graph-based collaborative filtering.
- LR-GCCF is a general graph convolutional collaborative-filtering model for recommendation.
- At each propagation layer, LR-GCCF uses linear embedding propagation without nonlinear transformations.The embedding matrix is trained rather than treated as fixed node features; propagation uses the normalized adjacency matrix with self-loops and a linear transformation.
- Linear Embedding Propagation: For a predefined depth K, recursive propagation stops at layer K, whose embeddings represent up to K-th-order bipartite graph similarity.User and item latent vectors are combined through an inner product to predict preference.
- Residual Preference Prediction: On Amazon Books, increasing GCN depth beyond the best shallow setting rapidly reduces performance while embedding similarity variance decreases with depth.The paper reports that performance rises from K=0 to K=1 or 2, then drops quickly; increasing K from 0 to 2 improves performance by less than 10%.
- Residual Preference Prediction: The authors conclude that BPR with K=0 can already approximate user preference to a large extent.This observation motivates learning residual preferences rather than directly fitting each layer’s full preference.
- Residual Preference Prediction: Residual preference learning adds each layer’s user-item preference contribution instead of directly approximating the full preference at every layer.The paper motivates residual learning as easier to optimize and as a way to alleviate over-smoothing with deeper layers.
- Model Discussion: LR-GCCF is a wide linear model that can use stochastic gradient descent instead of backpropagation, making it easier and more time-efficient to train than classical nonlinear GCN models.The paper also connects the linearization to simple graph convolution and a graph low-pass filter interpretation.
- Model Discussion: The paper compares LR-GCCF with GC-MC, PinSage, and NGCF as closely related graph-based recommendation models.
Experiments
Experiments evaluate LR-GCCF on Amazon Books and Gowalla using ranking metrics, baseline comparisons, depth studies, and embedding-similarity analysis. Results support the contributions of linear propagation and residual preference learning.
- Experimental Setup: Experiments use Amazon Books and Gowalla, filtering users and items with fewer than 10 interactions and splitting records into 80% training, 10% validation, and 10% test.
- Experimental Setup: The evaluation compares LR-GCCF with BPR, GC-MC, PinSage, NGCF, and linear or residual GC-MC variants using HR@N and NDCG@N.
- Overall Comparison: LR-GCCF consistently outperforms NGCF on HR@N and NDCG@N, supporting residual preference prediction and linear embedding propagation.
- Detailed Model Analysis: Linear-GC-MC outperforms GC-MC by a large margin, while comparisons across residual variants examine the contribution of residual learning.
- Efficiency: 20s is LR-GCCF's reported per-iteration runtime at K=4, compared with 30s for GC-MC at K=1 and 38s for PinSage and NGCF at K=2.
- Depth Analysis: The best propagation depth is K=4 on Amazon Books and K=3 on Gowalla, while K=0 removes propagation and degenerates to BPR.
- Detailed Model Analysis: Residual learning produces larger variance in user-user cosine similarities than L-GCCF, empirically indicating partial alleviation of over-smoothing.
Conclusions
The paper concludes that LR-GCCF combines linear embedding propagation with residual preference learning to address over-smoothing in graph-based collaborative filtering. Experiments show the model is effective and efficient, while Figure 3 compares residual and non-residual structures across layer depths on Amazon Books.
- Figure 3 compares models with and without residual preference prediction under different layer depths K on Amazon Books.
- LR-GCCF removes nonlinear transformations and uses linear embedding propagations.
- Its residual preference prediction structure learns residual user-item preferences at each layer to reduce over-smoothing from higher-layer graph convolutions.
- Extensive experiments show that LR-GCCF is effective and efficient for collaborative filtering recommendation.