Source-linked AI summary
AliGraph: A Comprehensive Graph Neural Network Platform
Rong Zhu, Kun Zhao, Hongxia Yang, Wei Lin, Chang Zhou, Baole Ai, Yong Li, Jingren Zhou
TL;DR
Large, heterogeneous, attributed, and dynamic commercial graphs make scalable GNN storage, computation, and algorithm development difficult. AliGraph addresses this gap with a comprehensive platform combining distributed storage, optimized sampling and runtime, and supported GNN models. Its in-house models report 4.12%–17.19% lifts in normalized evaluation metrics, alongside system-level speedups described in the paper context.
Problem
Real-world commercial graphs are large-scale, heterogeneous, attributed, and dynamic, while existing GNN methods face time, space, integration, and incremental-learning challenges.
Method
AliGraph combines distributed graph storage, optimized sampling operators and runtime, and existing and in-house-developed GNN algorithms in one platform.
Results
4.12%–17.19% lift in normalized evaluation metrics is reported for AliGraph’s in-house-developed GNN models.
Takeaways & Limitations
AliGraph provides a deployed platform for supporting GNN methods and applications on practical commercial graph problems.
Takeaways & Limitations
The evaluation does not compare each proposed GNN algorithm with all competitors because each algorithm targets a different focus.
Abstract
from arXiv · showhide
An increasing number of machine learning tasks require dealing with large graph datasets, which capture rich and complex relationship among potentially billions of elements. Graph Neural Network (GNN) becomes an effective way to address the graph learning problem by converting the graph data into a low dimensional space while keeping both the structural and property information to the maximum extent and constructing a neural network for training and referencing. However, it is challenging to provide an efficient graph storage and computation capabilities to facilitate GNN training and enable development of new GNN algorithms. In this paper, we present a comprehensive graph neural network system, namely AliGraph, which consists of distributed graph storage, optimized sampling operators and runtime to efficiently support not only existing popular GNNs but also a series of in-house developed ones for different scenarios. The system is currently deployed at Alibaba to support a variety of business scenarios, including product recommendation and personalized search at Alibaba's E-Commerce platform. By conducting extensive experiments on a real-world dataset with 492.90 million vertices, 6.82 billion edges and rich attributes, AliGraph performs an order of magnitude faster in terms of graph building (5 minutes vs hours reported from the state-of-the-art PowerGraph platform). At training, AliGraph runs 40%-50% faster with the novel caching strategy and demonstrates around 12 times speed up with the improved runtime. In addition, our in-house developed GNN models all showcase their statistically significant superiorities in terms of both effectiveness and efficiency (e.g., 4.12%-17.19% lift by F1 scores).
1. INTRODUCTION
Graph Neural Networks address graph learning by producing low-dimensional embeddings while preserving structural and property information, but real-world commercial graphs create major scale, heterogeneity, attribute, and dynamism challenges. AliGraph responds with a comprehensive platform combining system components and algorithms, with reported effectiveness gains for its in-house models.
- Challenges: Real-world commercial graphs are large-scale, heterogeneous, attributed, and dynamic, creating challenges for graph embedding and GNN methods.These properties include billions of vertices and edges, multiple object types, rich attributes, and rapid evolution.
- Challenges: Existing GNN methods struggle to scale efficiently on irregular, exceedingly large graphs.The paper identifies improving time and space efficiency as the first practical problem.
- Challenges: Heterogeneous information, attributes, and graph updates require unified embeddings and efficient incremental GNN methods.The paper separately frames integration of multiple object types, topology with unstructured attributes, and dynamic updates as open problems.
- Contributions: AliGraph provides a comprehensive platform with system and algorithm components to support GNN methods and applications across these practical problems.Its architecture includes storage, sampling, and operator layers, while also supporting existing and in-house-developed GNNs.
- Evaluation: 4.12%–17.19% lift in normalized evaluation metrics is reported for AliGraph’s in-house-developed GNN models.The models are evaluated on data collected from Alibaba’s Taobao e-commerce platform.
2. PRELIMINARIES
The preliminaries define graph embeddings, attributed heterogeneous graphs, dynamic graphs, and the vertex-level embedding problem. These concepts formalize the structures and information types that AliGraph is designed to represent.
- Graph Definitions: A simple graph G = (V, E, W) consists of vertices, edges, and edge weights indicating relationship strength.Graphs may be directed or undirected, and Nb(u) denotes the neighbors of vertex u.
- Graph Definitions: An attributed heterogeneous graph includes multiple vertex or edge types together with feature vectors for vertex and edge attributes.Its mappings assign types to vertices and edges, while attribute functions assign feature vectors.
- Graph Definitions: Figure 2 illustrates an attributed heterogeneous graph containing user and item vertices connected by four edge types.The example visualizes multiple node and edge types with rich attributes.
- Graph Definitions: A dynamic graph is a time-indexed series of graphs G(1), G(2), . . . , G(T), where each state may be simple or attributed heterogeneous.Superscript (t) identifies the state of graph objects at timestamp t.
- Embedding Problem: The embedding problem converts an input graph into a d-dimensional space while preserving graph properties as much as possible.GNN is defined as a graph embedding method that applies neural networks to produce a d-dimensional vector for each vertex.
3. SYSTEM
AliGraph provides a layered platform for GNN algorithms, combining distributed graph storage, neighborhood sampling, operators, caching, and runtime support. Its system design targets large attributed graphs by reducing storage, communication, traversal, and computation costs.
- GNN Framework: GNN processing initializes vertex embeddings from attributes, samples neighbors, aggregates sampled embeddings, combines them, normalizes results, and returns final vertex embeddings.The output is a d-dimensional vector for each vertex for downstream tasks such as classification and link prediction.
- System Architecture: AliGraph organizes GNN support into storage, algorithm, and application layers, with underlying system layers serving high-level algorithms and applications.The platform includes five layers overall, with three underlying layers forming the system.
- Storage: AliGraph partitions graphs across workers and separately stores structural and attribute information using adjacency tables plus vertex and edge attribute indices.The separate-storage strategy reduces space cost from O(nNDNL) to O(nND + NANL).
- Storage: Neighbor caching stores 1- to k-hop out-neighbors only for vertices whose importance exceeds user-specified thresholds, trading storage cost against traversal and communication cost.The system uses a small cache depth, usually 2, and caches only a small number of important vertices under power-law degree distributions.
- Operators: Materializing intermediate embedding vectors reduces computation costs for the AGGREGATE and COMBINE operators, with operator time costs speeding up by an order of magnitude.The reduction comes from eliminating redundant computation of intermediate embedding vectors.
4. METHODOLOGY
AliGraph provides a platform where existing GNNs can be implemented alongside new algorithms targeting heterogeneous, attributed, multiplex, hierarchical, dynamic, and sampled graph settings.
- Platform and algorithm layer: AliGraph supports existing GNN implementations and adds new algorithm plugins addressing four challenges of real-world graph embedding.The platform's algorithm layer includes in-house methods for practical requirements.
- Sampling: AHEP samples important neighbors separately by node type using distributions designed to minimize sampling variance.It targets the computation and storage costs of embedding propagation on heterogeneous networks.
- Sampling: AHEP runs much faster than HEP while achieving comparable accuracies.Its task loss combines supervised learning, sampled embedding propagation, and parameter regularization.
- Heterogeneous and multiplex graphs: GATNE combines general, specific, and attribute embeddings to represent structural, heterogeneous, and attribute information in multiplex heterogeneous graphs.Type-specific embeddings are combined into a final vertex representation, with adjustable coefficients and trainable transformations.
- Heterogeneous and multiplex graphs: Mixture GNN extends skip-gram modeling to heterogeneous graphs in which each node has multiple senses.Because negative sampling is difficult to apply directly, training maximizes a lower bound approximated through modified sampling.
- Hierarchical and dynamic graphs: Hierarchical GNN learns layer-specific embeddings, clusters vertices, and updates adjacency matrices across graph layers.Evolving GNN instead learns vertex representations over sequential graphs while distinguishing normal evolution from burst links.
5. EXPERIMENTS
AliGraph is evaluated as a deployed platform for graph storage, sampling, operators, and several GNN algorithms. Experiments report fast, scalable system components and improved effectiveness or efficiency across proposed models.
- Experimental scope: AliGraph evaluates storage, caching, sampling, operators, and proposed GNN algorithms on Taobao and Amazon-related datasets.The system experiments cover both platform and algorithm performance.
- System evaluation: 5 minutes: AliGraph builds the large Taobao graph in minutes, with graph-building time decreasing as worker count increases.The Taobao-large dataset is the larger of the two system-evaluation datasets.
- System evaluation: 40%–50% less time than random caching and 50%–60% less than LRU: importance-based neighbor caching reduces cost while caching around 20% of extra vertices.The threshold is set to 0.2 to balance cache cost and benefit.
- System evaluation: Sampling completes in a few milliseconds to no more than 60ms, and its time grows slowly despite a sixfold dataset-size difference.These observations support efficient and scalable sampling implementations.
- System evaluation: An order of magnitude: optimized AGGREGATE and COMBINE operators reduce time by eliminating redundant intermediate-embedding computation.The improvement is attributed to the caching strategy.
- Algorithm evaluation: AHEP runs about 2–3 times faster than HEP with less memory and comparable ROC-AUC and F1-score on Taobao-small.Both methods are reported as producing results within reasonable time and space limits.
- Algorithm evaluation: GATNE outperforms existing methods on all metrics, while converging in less than 2 hours with 150 distributed workers.On Taobao-small, reported lifts include 4.6% ROC-AUC, 1.84% PR-AUC, and 5.08% F1-score.
- Algorithm evaluation: The Mixture, Hierarchical, Evolving, and Bayesian GNNs improve reported recommendation or link-prediction metrics over their comparison methods.Reported improvements include around 2% hit recall, around 7.5% F1-score or hit recall, 4.2% and 3.6% F1-score, and 1%–3% hit recall, respectively.
6. RELATED WORK
The related-work review organizes graph embedding and GNN methods around homogeneous, heterogeneous, attributed, and dynamic graph settings. It positions AliGraph’s proposed models against representative approaches across these categories.
- Reported comparisons: GATNE outperforms all competitors on Amazon and Taobao-small, with a 16.43% F1-score lift on Amazon.The comparison covers all reported metrics.
- Reported comparisons: Mixture GNN improves hit recall by around 2% on Taobao-small relative to its compared methods.The comparison is against DAE and β∗-VAE.
- Method categories: Existing graph embedding and GNN methods are reviewed by graph-data properties, including homogeneous, heterogeneous, attributed, and dynamic settings.The review connects these categories to the practical challenges motivating the platform.
- Representative methods: The review describes representative methods such as DeepWalk, LINE, Node2Vec, GCN, GraphSAGE, PMNE, MNE, and dynamic-network approaches.These methods address different structural, attribute, heterogeneity, or temporal settings.
7. CONCLUSIONS AND FUTURE WORK
AliGraph addresses large-scale, heterogeneous, attributed, and dynamic graph challenges through a platform combining system support with algorithms. Future work targets broader embeddings, execution optimization, early stopping, and Auto-ML.
- Conclusion: AliGraph provides both system and algorithm components for four practical graph-data challenges: large-scale, heterogeneous, attributed, and dynamic graphs.The conclusion frames the platform as a comprehensive solution to these challenges.
- Future work: Future work includes edge-level and subgraph-level embeddings, more execution optimizations, early stopping, and Auto-ML.Listed optimizations include reducing cross-network traffic, distributed gradient improvements, and better multi-GPU worker assignment.
Appendix
The appendix derives the distribution of k-hop neighbor counts and vertex importance under power-law assumptions. It concludes that importance is power-law distributed, motivating selective caching of highly important vertices.
- Theorem proof: The appendix derives k-hop in- and out-neighbor distributions by induction from power-law degree assumptions at k=1.The derivation treats neighbor counts as random variables for a randomly chosen vertex.
- Theorem proof: The k-hop in- and out-neighbor counts obey power-law distributions for each k ≥1.This conclusion follows from the summarized induction steps.
- Caching implication: Vertex importance also follows a power-law distribution, so most vertices have low importance and only a small number require caching.Highly important vertices are characterized by many in-neighbors and few out-neighbors.