Source-linked AI summary
Computing Graph Neural Networks: A Survey from Algorithms to Accelerators
Sergi Abadal, Akshay Jain, Robert Guirado, Jorge López-Alonso, Eduard Alarcón
TL;DR
GNN processing remains an open question because the field is still nascent and combines graph-dependent, dense, and sparse computation. This paper examines GNN algorithms, software frameworks, and hardware accelerators, concluding that no single design fits all workloads and that hardware proposals lack a clear architectural trend.
Problem
GNN processing remains an open question because the field is in its nascent stages and combines graph-dependent computation with dense and sparse operations.
Method
The paper examines relational-learning algorithms through message passing and differential pooling, alongside software frameworks and acceleration approaches.
Results
The review finds that one approach does not fit all GNNs, software frameworks and hardware accelerators are increasingly prevalent, and hardware proposals show no clear architectural trend.
Takeaways & Limitations
A hardware-software co-design should exploit graph awareness and address workload variability through a reconfigurable interconnect.
Takeaways & Limitations
Quantitative comparison among systems remains difficult because there is no common baseline system, and training remains an open question because most proposals target inference.
Abstract
from arXiv · showhide
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data is inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of groundbreaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage of research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this paper aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.
1 INTRODUCTION
GNNs model graph-structured data by adapting computation to input connectivity, supporting relational applications that conventional architectures handle inefficiently. This survey examines the field from a computing perspective, including algorithm fundamentals, evolution, software, hardware accelerators, and architectural opportunities.
- GNN motivation: Conventional CNNs and RNNs exploit locality or temporal sequentiality, making them less suited to data with arbitrary structures.
- Applications: GNNs are used across relational domains including communication networks, synthetic chemistry, social networks, neuroscience, and other application areas.
- GNN motivation: GNNs adapt their structure to an input graph and iteratively aggregate information across vertices to capture underlying system dependencies.This supports predictions for nodes, connections, entire graphs, and unseen graphs.
- Computing challenges: Efficient GNN processing remains underexplored and must support dense and extremely sparse operations, graph-dependent computation, and very large graphs.
- Computing challenges: Existing software and hardware efforts adapt GNN operations to CPUs or GPUs and design custom processors, but recent reviews lack comprehensive analysis of these advances.
- Survey contributions: The survey unifies GNN fundamentals, charts field evolution through a Knowledge Graph approach, analyzes computation and accelerators, and outlines research directions.
2 FUNDAMENTALS OF GRAPH NEURAL NETWORKS
GNNs learn graph-structured relationships by repeatedly aggregating neighboring edge and vertex features and transforming them through combination functions. Their computation typically proceeds through preprocessing, iterative updates, and readout, with sampling or pooling used to manage complexity.
- GNNs use graph connectivity to learn and model relationships between nodes.
- GNN execution commonly includes optional preprocessing, iterative edge and vertex updates, and a final readout producing node-, edge-, or graph-level outputs.Preprocessing may sample or reorder the graph and encode features; readout aggregates final representations when a graph-level output is required.
- Stacking layers propagates information across increasingly distant nodes and edges, although excessive depth can cause oversmoothing, vanishing gradients, or overfitting.Most widespread GNN algorithms have 1–5 layers; some proposals reach up to 100 layers, but those techniques remain in their infancy.
- Each layer aggregates neighboring edge or vertex features, then applies a combination function to produce updated representations.Aggregation operates over graph neighborhoods, while combination functions may use learnable matrices or neural networks.
- Aggregation and combination choices affect expressive power and may range from averaging and weighted sums to neural networks such as MLPs or LSTMs.Operations can vary across layers and between edges, vertices, and global updates, although implementations often share operations across layers or simplify updates.
- Sampling prunes graphs or neighborhoods, while pooling coarsens graphs across layers to reduce aggregation and combination work.Dynamic graphs can additionally use recurrent units to adapt weight matrices over time.
3 THE EVOLUTION OF THE GNN FIELD
The GNN field has expanded rapidly across algorithms and applications, while computing research remains nascent and comparatively small. The paper traces this evolution and analyzes the literature to motivate deeper, graph-aware acceleration work.
- The paper distills existing acceleration studies into a hardware-software, graph-aware, and communication-centric perspective for emerging GNN accelerator design.
- The literature review uses an annotated-paper repository, manual discipline tags, extracted references, and a Neo4j knowledge graph to study the field.
- Computing-related categories comprise 10.99% for accelerators, 8.79% for HW/SW requirements, and 3.30% for dataflow.
- GNN computing research is in its nascent stages, with the first specific acceleration paper appearing in 2017.
- The knowledge graph shows computing categories are small but connected to theory, whereas algorithms and applications are large, with applications sparsely connected internally.
- Algorithm and application categories are strongly interconnected, but application-to-computing connections remain weak, potentially reflecting computing’s relative immaturity.
- GNN research grew slowly until around 2016, then expanded dramatically alongside GCN’s introduction; computing research began growing similarly from 2017.
4 THE REVOLUTION OF GNN ACCELERATION
GNN acceleration is difficult because variants combine different operations, depend strongly on graph structure, and alternate dense and irregular sparse computation. The surveyed systems therefore use specialized software and hardware techniques, but comparisons remain difficult and most accelerators target inference.
- Challenges: GNN variants require different combinations of acceleration techniques because their operations, computational complexities, and ordering constraints vary.The paper notes that no single general technique is sufficient for every GNN variant.
- Challenges: Graph characteristics can require changes to logic type, dataflow, partitioning, mapping, and scheduling within and across graphs.Relevant characteristics include graph size, sparsity, clustering, connectivity, and feature-vector length.
- Challenges: Alternating dense combination and irregular aggregation phases demand architectures that accelerate both phases and their runtime interplay.Sparse GEMM can optimize aggregation for some cases, but it is not generalizable to all graphs or GNNs.
- Software acceleration: Software frameworks report substantial improvements, including up to 100X speedup for G3 over PyG and TF in GNN training on a high-end GPU.Other surveyed software techniques report 3×-4× speedup over DGL, PyG, and NeuGraph, while PCGCN reaches up to 8.8× in highly clustered graphs.
- Hardware acceleration: Hardware accelerators are generally reported as two to three orders of magnitude faster and more energy efficient than GPU and CPU platforms, respectively, often below 10 mm2.The comparison is qualified by the lack of a common baseline and by most proposals targeting inference.
- Discussion: The survey concludes that distinct design approaches are probably required for different applications rather than one universal accelerator.The paper also identifies dynamic graphs, training support, common benchmarks, and performance-generalization balance as open challenges.
5 GNN ACCELERATION: THE VISION
The proposed vision addresses GNN diversity and graph dependence through software-hardware co-design, graph awareness, and communication-centric architecture. It combines software control and graph-informed decisions with adaptable hardware data movement to support varied execution stages.
- Motivation: GNN acceleration must balance performance and efficiency with flexibility across different graph sizes, characteristics, and algorithms.Existing approaches trade highly efficient specialization against broader but less efficient flexibility.
- Architectural vision: The vision has three pillars: software-hardware co-design, graph awareness, and communication-centric design.Figure 9 summarizes these pillars through control and data planes, guided mapping and scheduling, and a reconfigurable interconnect.
- Co-design examples: Software preprocessing can complement specialized hardware by eliminating redundant operations before optimized aggregation and combination execution.The paper gives graph reordering and redundancy elimination as examples of complementary software and hardware roles.
- Software-hardware co-design: A software control plane can partition computation, map vertices and edges, schedule executions, and drive preprocessing, while custom hardware implements the data plane.The planes are conceptually separated but tightly coupled.
- Communication-centric design: A homogeneous processing-element array, specialized memory hierarchy, and reconfigurable interconnect can adapt dataflow across multiple GNN execution stages.The interconnect is intended to support changing communication patterns among processing elements.
- Graph awareness: Graph awareness uses graph features to adapt partitioning, phase ordering, pipelining, and scheduling, but its preprocessing overhead must remain limited.Community detection is cited as a way to identify densely connected graphlets relevant to pooling, redundancy elimination, and scheduling.
6 CONCLUSION
GNN computing is gaining importance as applications expand, while efficient processing remains challenging because workloads vary with algorithms and graph characteristics. The survey observes growing software and hardware support and envisions co-designed, graph-aware accelerators using reconfigurable communication.
- 6 CONCLUSION: The survey anticipates rapid growth in software and hardware support for GNNs, continuing the upward trend observed from 2018 onward.The paper attributes this expectation to field maturation and application-oriented development, alongside strong demand from disruptive applications.
- 6 CONCLUSION: GNN processing faces challenges from diverse algorithm variants, graph-dependent workloads, and massive scale in some applications.These factors motivate discussion of both processing challenges and possible solutions.
- 6 CONCLUSION: Software frameworks and hardware accelerators for GNNs have recently become increasingly popular.Software efforts include libraries extending widespread frameworks and acceleration through graph analysis, pre-coding, and distributed computation.
- 6 CONCLUSION: Existing hardware proposals show no clear architectural trend, balancing specialization against support for multiple GNN variants and unified against hierarchical tiled designs.The survey characterizes these as ongoing architectural debates rather than a settled direction.
- 6 CONCLUSION: Future GNN accelerators are envisioned to use hardware-software co-design, graph awareness, and reconfigurable interconnects to address workload variability.The proposed vision links co-design with performance, graph awareness with optimization opportunities, and reconfigurable interconnects with variable workloads.