Source-linked AI summary
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, Jure Leskovec
TL;DR
Web-scale recommendation requires scalable training and inference for graph convolutional networks on graphs with billions of nodes and edges. PinSage addresses this with localized, random-walk-based convolutions and achieves higher-quality recommendations in offline metrics, human evaluations, and A/B tests.
Problem
Scaling GCN training and inference to graphs with billions of nodes and tens of billions of edges remains difficult because full-graph Laplacian operations are infeasible.
Method
PinSage generates item embeddings with localized graph convolutions that sample neighborhoods via random walks and aggregate graph and feature information using shared parameters.
Results
More than 40% improvement over the best baseline in offline ranking metrics, about 60% preference in human evaluations, and 30% to 100% user-engagement gains in A/B tests.
Takeaways & Limitations
PinSage demonstrates that graph convolutional embeddings can support production recommendation on web-scale graphs with billions of objects.
Takeaways & Limitations
Pixie is excluded from offline comparisons because it generated the historical labeled pairs, though it is compared using human evaluations.
Abstract
from arXiv · showhide
Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.
1 INTRODUCTION
The introduction presents PinSage, a random-walk-based GCN designed to make graph-based recommendation practical at web scale. It addresses billion-node training and inference challenges through localized convolutions and new sampling and training techniques, achieving strong recommendation results in offline and online evaluations.
- Background: GCNs learn node representations by iteratively aggregating feature information from local graph neighborhoods.A convolution aggregates information from a node’s one-hop neighborhood, and stacked convolutions propagate information across larger neighborhoods.
- Challenge: Scaling GCN embeddings to graphs with billions of nodes and tens of billions of edges is difficult because existing methods require the full graph Laplacian during training.This full-graph assumption is infeasible for web-scale recommendation graphs.
- Present work: PinSage is a production-deployed, random-walk-based GCN operating on 3 billion nodes and 18 billion edges, a graph 10,000× larger than typical GCN applications.The framework was developed and deployed at Pinterest for highly scalable recommendation.
- Scalability: PinSage improves scalability through on-the-fly localized convolutions that sample neighborhoods and dynamically construct computation graphs instead of using the full graph Laplacian.These sampled computation graphs specify localized convolutions around particular nodes.
- Algorithmic innovations: Short random walks replace suboptimal random sampling for constructing computation graphs, while importance scores support pooling and aggregation.The introduction also highlights training and algorithmic innovations intended to improve learned representations and downstream recommendation quality.
- Results: Offline ranking metrics improve over the best baseline by more than 40%, human evaluators prefer the recommendations about 60% of the time, and A/B tests show 30% to 100% higher user engagement.These evaluations cover related-pin and homefeed recommendation tasks and include offline metrics, controlled user studies, and A/B tests.
2 RELATED WORK
Prior graph neural-network methods achieved strong benchmark results but remained difficult to scale because training required expensive message passing or access to the entire graph Laplacian. PinSage addresses these production-scale limitations through sampled graph neighborhoods, improved training techniques, and scalable inference.
- Graph neural networks: Initial graph neural networks required expensive message-passing procedures to converge, making them prohibitively expensive on large graphs.Gated Graph Sequence Neural Networks addressed some limitations with recurrent architectures, but the passage does not state that they solved scalability.
- Graph convolutional networks: Graph convolutional networks emerged from spectral graph convolutions and produced state-of-the-art results on benchmark datasets.Subsequent work proposed improvements, extensions, and approximations of spectral convolutions.
- Production-scale scalability: No prior GCN method had been applied to production-scale graphs with billions of nodes and edges because traditional training operated on the entire graph Laplacian.The paper positions PinSage as filling this scalability gap in recommender systems.
- Relation to GraphSAGE: Relative to GraphSAGE, PinSage avoids storing the whole graph in GPU memory by sampling graph neighborhoods with low-latency random walks in a producer-consumer architecture.The approach is described as an inductive GCN variant modified to avoid operating on the entire graph Laplacian.
- Scalable training and inference: PinSage adds training techniques for improved performance and a MapReduce inference pipeline for graphs containing billions of nodes.These additions complement the sampled-neighborhood design for web-scale deployment.
3 METHOD
PinSage generates node embeddings by stacking localized graph convolutions that aggregate feature and structural information from importance-weighted neighborhoods. It trains these embeddings with a supervised max-margin ranking objective and addresses scalable embedding generation through an inference pipeline.
- Localized graph convolutions: PinSage applies stacked localized graph convolutions to aggregate node features from small graph neighborhoods and capture local network topology.Each module aggregates information from a node’s local neighborhood, while stacking modules expands the structural context.
- Embedding objective: The model combines bipartite graph structure with real-valued item attributes, including rich text and image features, to produce recommendation embeddings.Embeddings support nearest-neighbor related-item recommendation and downstream candidate re-ranking.
- Localized graph convolutions: Random walks define each node’s neighborhood as the top T nodes by L1-normalized visit counts, enabling importance-weighted aggregation.The aggregator uses a weighted mean whose weights are the normalized visit counts; in the infinite-simulation limit, these approximate Personalized PageRank scores.
- Training: PinSage uses supervised max-margin ranking to pull embeddings of related item pairs together while separating query items from unrelated negatives.The loss maximizes positive-pair inner products and constrains negative-pair inner products to remain smaller by a margin.
- Inference: Overlapping K-hop neighborhoods make naive embedding generation computationally repetitive after training, motivating an efficient inference procedure.The paper identifies repeated computations across layers and target nodes as the central inference challenge.
4 EXPERIMENTS
Experiments evaluate PinSage on related-pin and homefeed recommendation tasks using offline metrics, user studies, and production A/B tests. PinSage achieves strong recommendation quality while supporting graph-scale training and inference through efficient sampling and MapReduce pipelines.
- Experimental setup and scalability: Training on a sampled subgraph containing 20% of boards and 70% of labeled examples reduces runtime by a factor of 6 versus training on the full graph.The remaining labeled examples are reserved for tuning and testing, while inference runs over the entire graph.
- Related-pin recommendation: PinSage achieves 67% hit-rate and 0.59 MRR on related-pin recommendation, outperforming the top baseline by 40% absolute hit-rate and 22% absolute MRR.The corresponding relative improvements are 150% for hit-rate and 60% for MRR.
- Related-pin recommendation: Around 60% of preferred items in the head-to-head user study are recommended by PinSage.The comparison covers items for which users expressed an opinion about which recommendation was more related.
- Homefeed recommendation: PinSage recommendations improve repin rate by 10-30% over Annotation and Visual embedding recommendations in production homefeed A/B tests.Repin rate measures the percentage of homefeed recommendations saved by users.
- Experimental setup and scalability: A two-layer GCN with neighborhood size 50 best captures node neighborhood information while remaining computationally efficient.This configuration is selected during the experiments’ model and hyperparameter evaluation.
- Experimental setup and scalability: MapReduce inference generates embeddings for 3 billion items in less than 24 hours.The inference pipeline is applied after training completes.
5 CONCLUSION
The paper proposes PinSage, a random-walk graph convolutional network designed to learn node embeddings in web-scale graphs with billions of objects. Its scalability techniques, importance pooling, and curriculum training improved embedding performance, and the system was deployed and evaluated at Pinterest.
- Conclusion: PinSage is a highly scalable random-walk GCN for learning embeddings of nodes in web-scale graphs containing billions of objects.The method combines graph convolution with random walks to address web-scale embedding.
- Conclusion: Importance pooling and curriculum training drastically improved embedding performance.These techniques were introduced alongside methods designed to ensure scalability.
- Conclusion: PinSage was deployed at Pinterest and comprehensively evaluated on the quality of its learned embeddings.The evaluation covered a number of recommendation-related settings, though the supplied passage is truncated before specifying them.