Source-linked AI summary
STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems
Jiani Zhang, Xingjian Shi, Shenglin Zhao, Irwin King
TL;DR
Recommender systems need scalable representations that support both ordinary rating prediction and unseen users or items. STAR-GCN addresses these settings with stacked encoder-decoders, masked-embedding reconstruction, and leakage-aware training, achieving strong benchmark performance including cold-start gains.
Problem
GC-MC uses one-hot node inputs that scale with graph size and cannot represent users or items unseen during training, limiting cold-start prediction.
Method
STAR-GCN learns low-dimensional node embeddings, reconstructs masked embeddings through stacked GCN encoder-decoders with intermediate supervision, and uses sample-and-remove training to avoid label leakage.
Results
STAR-GCN achieves state-of-the-art performance in four of five real-world datasets and significantly outperforms baselines in inductive rating prediction.
Takeaways & Limitations
STAR-GCN learns latent factors for both transductive and inductive recommendation and can use graph structure to support cold-start prediction without available content information.
Abstract
from arXiv · showhide
We propose a new STAcked and Reconstructed Graph Convolutional Networks (STAR-GCN) architecture to learn node representations for boosting the performance in recommender systems, especially in the cold start scenario. STAR-GCN employs a stack of GCN encoder-decoders combined with intermediate supervision to improve the final prediction performance. Unlike the graph convolutional matrix completion model with one-hot encoding node inputs, our STAR-GCN learns low-dimensional user and item latent factors as the input to restrain the model space complexity. Moreover, our STAR-GCN can produce node embeddings for new nodes by reconstructing masked input node embeddings, which essentially tackles the cold start problem. Furthermore, we discover a label leakage issue when training GCN-based models for link prediction tasks and propose a training strategy to avoid the issue. Empirical results on multiple rating prediction benchmarks demonstrate our model achieves state-of-the-art performance in four out of five real-world datasets and significant improvements in predicting ratings in the cold start scenario. The code implementation is available in https://github.com/jennyzhang0215/STAR-GCN.
1 Introduction
The paper motivates STAR-GCN by limitations in GCN-based recommendation and introduces a stacked, reconstructed architecture for scalable transductive and cold-start prediction.
- 1 Introduction: Matrix completion fills missing entries in a user–item rating matrix from observed ratings.Classical matrix factorization generates ratings from user and item latent factors.
- 1 Introduction: GCN methods learn node representations by aggregating transformed information from local graph neighborhoods.Stacked aggregators and nonlinearities extract features across increasingly distant parts of a graph.
- 1 Introduction: GC-MC is limited by one-hot node inputs whose dimensionality grows with graph size and cannot represent unseen users or items.The latter limitation is the cold start problem.
- 1 Introduction: STAR-GCN learns low-dimensional user and item embeddings, reconstructs masked embeddings, and stacks encoder-decoder blocks with intermediate task supervision.These mechanisms target scalable inputs, unseen-node embeddings, and improved prediction performance.
- 1 Introduction: Experiments cover transductive and inductive rating prediction, with state-of-the-art performance on four of five datasets and significant inductive gains over baselines.The inductive task uses a few ratings connected to new users or items before prediction.
2 Preliminary
The paper distinguishes transductive recommendation from inductive cold-start prediction and reviews bipartite-graph GCN matrix completion using rating-specific message aggregation.
- 2.1 Rating Prediction Tasks: Rating prediction models users and items as nodes in an undirected bipartite graph, with observed ratings represented as typed edges.The task predicts ratings for unobserved user–item pairs from a small subset of observed rating pairs.
- 2.1 Rating Prediction Tasks: Transductive prediction assumes testing users and items were observed during training, matching the setting targeted by traditional matrix factorization.Figure 1a illustrates this setting.
- 2.1 Rating Prediction Tasks: Inductive prediction introduces users or items unseen during training and uses a few testing-time edges connected to them before making predictions.Traditional collaborative filtering requires retraining for this setting, while content-based methods use node attributes.
- 2.1 Rating Prediction Tasks: STAR-GCN combines node content and graph structure to learn representations for both transductive and inductive prediction.Its structural information can support cold-start prediction when content information is unavailable.
- 2.2 Graph Convolutional Matrix Completion: GC-MC assigns each rating level a transformation, aggregates messages from neighboring nodes, and applies nonlinear transformations to produce node vectors for rating prediction.User-to-item messages are processed analogously with separate parameters.
3 Our Models
STAR-GCN uses stacked graph encoder-decoder blocks with masked embedding reconstruction, low-dimensional node inputs, and intermediate supervision to support scalable rating prediction and cold-start inference. It also addresses GCN label leakage by removing sampled training edges during aggregation.
- Architecture: Each STAR-GCN block combines a graph encoder, a decoder that reconstructs input node embeddings, task-specific supervision, and reconstruction loss.The encoder aggregates graph structure and content features, while the decoder recovers the input node vectors.
- Architecture: STAR-GCN can stack blocks with separate parameters or recur a shared encoder-decoder block to reduce memory usage.Stacking uses distinct parameter sets; recurrence unfolds one shared block across multiple stages.
- Input Node Representations: Low-dimensional embedding lookup vectors replace one-hot node inputs, reducing input dimensionality but requiring masked reconstruction to represent unseen nodes.The embedding dimension de is much smaller than the total number of users and items, while naive lookup alone cannot initialize nodes absent from training.
- Input Node Representations: STAR-GCN randomly masks node embeddings and reconstructs them, enabling iterative cold-start embedding refinement and acting as a multi-task regularizer for rating prediction.New nodes start with zero embeddings; successive encoder-decoder blocks use neighborhood data or attributes to refine embeddings alongside rating predictions.
- Input Node Representations: External node features are transformed to fixed-size vectors and concatenated with learned node embeddings before graph encoding.The resulting input dimension is din = de + df when content features are available.
- Training by Avoiding a Leakage Issue: GCN rating prediction suffers label leakage because target rating edges enter neighborhood aggregation, so training samples are removed from the graph before model updates.The leakage changes the effective training input to include the target label and causes overfitting; removing sampled edges substantially boosts performance.
4 Experiments
Experiments evaluate STAR-GCN on transductive and inductive rating prediction using recommendation benchmarks, ablations, and cold-start settings. Results show strong transductive performance, benefits from leakage prevention and reconstruction, and improved inductive prediction.
- Experimental Setup: Five recommendation benchmarks are used to evaluate transductive and inductive rating prediction, with RMSE measuring prediction accuracy.Flixster, Douban, and three MovieLens datasets are included; inductive experiments use Douban, ML-100K, and ML-1M.
- Transductive Rating Prediction: STAR-GCN achieves state-of-the-art transductive results on four of five datasets.Table 2 reports the best results among STAR-GCN variants against baseline scores from prior work.
- Ablation Analysis: Removing sampled training edges significantly decreases testing RMSE, supporting the sample-remove strategy for avoiding training data leakage.Sampled user-item pairs are removed from the bipartite graph before neighborhood aggregation in each training batch.
- Ablation Analysis: Reconstruction consistently improves prediction over models without reconstruction using the same total number of graph encoders.The reconstruction module masks part or all of node embeddings and reconstructs them through graph encoder-decoder blocks.
- Ablation Analysis: The recurrent structure achieves competitive results with fewer parameters, while external node features do not always improve performance.Recurrence shares parameters across encoder-decoder blocks; feature usefulness varies across compared models.
- Inductive Rating Prediction: In inductive prediction, STAR-GCN significantly outperforms two baselines, and reconstruction is crucial when predicting ratings for unseen nodes.Performance tends to worsen when fewer neighboring edges are available for new users or items; content information is not always beneficial.
5 Conclusion and Future Work
The paper presents STAR-GCN for transductive and inductive rating prediction and reports state-of-the-art results in both tasks. It also identifies training label leakage in GCN-based rating prediction and outlines future extensions to heterogeneous graphs and ranking.
- Conclusion: STAR-GCN is introduced as a GCN-based architecture for transductive and inductive rating prediction.The framework is described as generic and potentially applicable to other graph-based prediction applications.
- Conclusion: STAR-GCN achieves state-of-the-art results in both transductive and inductive rating prediction tasks.
- Conclusion: The paper identifies training label leakage in GCN-based rating prediction and proposes the discovery as a reminder for later research.
- Future Work: Future work targets heterogeneous graphs with diverse node types and integration of ranking algorithms for other recommendation tasks.