PaperlayerView exact revision on arXiv

Source-linked AI summary

FrontGI: Architecture-Aware Parallel GNN Inference on Chiplet-Based Multicore Systems

Binglin Ji, Chenfeng Zhao, Roger Chamberlain

arXiv:2610.34911v1cs.DC

TL;DR

High-core-count multicore systems can scale poorly for GNN inference, despite the memory capacity and broad availability of multicore platforms. FrontGI adapts inference parallelization to graph properties and multicore architecture; across the evaluated graphs, it achieves up to 3.35× performance gains over commonly used libraries, with the preferred strategy varying by graph degree.

  • Problem

    Standard-library GNN inference can scale poorly on high-core-count multicore systems, where irregular access patterns can cause inter-chiplet communication overhead.

    Source: §1 · p. 2, §2.1 · p. 4

  • Method

    FrontGI uses graph-property- and architecture-aware parallelization, including hierarchical node-level and mixed node–tensor strategies for Aggregate and a parallel Update strategy.

    Source: §1 · pp. 2–3

  • Results

    FrontGI achieves up to 3.35× performance gains relative to the higher-performing of two commonly used standard libraries, with the better strategy depending largely on graph topology.

    Source: §6 · p. 21

  • Takeaways & Limitations

    Hierarchical node parallelism scaled better for graphs with average degree below 20, mixed tensor/node parallelism for degrees above 50, and either strategy for degrees 20 to 50.

    Source: §6 · p. 21

    Before production-library incorporation, FrontGI requires evaluation across a broader range of GNN models and graph datasets.

    Source: §6 · p. 21

Abstract

from arXiv · show

The realm of high-performance computing has expanded to include machine learning tasks of many forms. Graph Neural Networks (GNNs) are becoming increasingly popular, with their applications expanding across diverse domains. As the scale of graph data continues to grow, including larger numbers of nodes, edges, and higher embedding dimensions, standardized libraries such as DGL and PyG have been developed to facilitate GNN computation. These libraries allow the computation to be described a a high level of abstraction, providing high productivity on the part of the developer. However, with the rapid increase in the number of processor cores and the evolution of multicore architectures, these libraries often show poor scalability and fail to execute GNN inference efficiently on the latest multicore systems, particularly those with upwards of a hundred cores. To address this limitation, we introduce FrontGI, a fast and scalable set of techniques that advances the frontier of GNN inference on large-scale graphs (over 100,000 vertices) by leveraging graph-property- and architecture-aware design. FrontGI employs different parallelization strategies optimized for diverse graph structures, maximizing the utilization of multi-level cache hierarchies in multicore systems. We evaluate a number of GNN models with FrontGI on a 128-core AMD EPYC system. Compared to state-of-the-art libraries, FrontGI achieves up to 3.35$\times$, 3.20$\times$, 2.89$\times$ and 3.10$\times$ speedups over DGL for GCN, GraphSAGE, GIN, and SGC models, respectively, and even greater improvements over PyG, across a range of large-scale, high-dimensional graph datasets with diverse structural properties.

1 Introduction

FrontGI targets scalable GNN inference on modern multicore systems by adapting parallel execution to graph properties and architecture. Its evaluation reports gains over standard libraries and examines four GNN models across twelve large-scale graph datasets.

1 Introduction

  • FrontGI addresses multicore GNN inference, where irregular graph access can drive inter-chiplet communication overhead and limit scalability.Multicore systems can support terabytes of memory, mitigating the memory-capacity constraint posed by large graphs on accelerators.Source: §1 · p. 2
  • Existing multicore approaches often rely on offline preprocessing, co-design assumptions, or training-specific optimization, leaving inference without preprocessing an open challenge.The introduction asks how to accelerate inference across graph topologies while avoiding offline preprocessing.Source: §1 · p. 2
  • The evaluation covers twelve large-scale datasets with varied graph sizes, average degrees, and degree distributions to assess strategy selection across graph structures.The techniques are designed for incorporation into standard libraries while retaining high-level model expression.Source: §1 · p. 3
  • FrontGI combines graph-property- and architecture-aware techniques, including hierarchical node-level and mixed node–tensor parallelism for Aggregate and a GEMM-based Update strategy.The design addresses distributed last-level caches and evaluates GCN, GraphSAGE, GIN, and SGC.Source: §1 · pp. 2–3
  • 3.35× speedup over DGL is achieved by FrontGI inference techniques.The techniques are intended for incorporation into high-productivity libraries.Source: §1 · p. 3

2 Motivation and Background

The motivation is to adapt GNN execution to chiplet-based multicore caches, where node-level parallelism can incur inter-chiplet communication. The background introduces GNN operations and models, plus tensor parallelism as a way to divide high-dimensional feature computations.

2.1 Motivation

  • DGL’s GCN speedup wanes at high core counts and declines beyond about 90 cores for OGBG-Molhiv and OGBN-Proteins, and beyond 64 cores for Reddit.The discussion asks whether computation can be partitioned to localize memory accesses within chiplet LLCs and reduce inter-chiplet communication.Source: §2.1 · p. 4
  • Chiplet-based systems group cores with a dedicated LLC slice; irregular GNN access patterns can make node-level parallelism incur substantial inter-chiplet communication.Software cannot directly control the hardware-determined LLC placement, motivating parallel execution adapted to cache organization.Source: §2.1 · p. 4

2.2 Background

  • The background establishes notation used throughout the paper and introduces GNN operations, models, and neural-network tensor-program optimizations.The supplied notation-table passage provides its caption only.Source: §2.2 · p. 5, Table 1 · p. 5

2.2.1 Graph Neural Network Operations

  • A GNN layer first aggregates neighbor embeddings, then updates the resulting embedding with a neural-network function.The Update may be a single layer with linear and nonlinear transformations or a multilayer perceptron.Source: §2.2.1 · p. 5

2.2.2 Graph Convolutional Network

  • GCN sums neighboring embeddings during aggregation, then updates them with a single neural-network layer.The described example applies ReLU activation.Source: §2.2.2 · pp. 5–6

2.2.3 GraphSAGE

  • GraphSAGE aggregates neighbor embeddings, concatenates them with the input node features, and applies a learnable transformation and nonlinear activation.Its weight matrix is decomposed into separate matrices applied to input-node and aggregated features to facilitate parallelization and reduce memory footprint.Source: §2.2.3 · p. 6

2.2.4 Graph Isomorphism Network

  • GIN sums neighbor and self embeddings before applying a learnable transformation with an MLP and nonlinear activation.Its sum aggregator is described as more expressive for distinguishing graph structures; ε is a learnable or fixed scalar.Source: §2.2.4 · p. 6

2.2.5 Simple Graph Convolution

  • SGC performs K rounds of normalized neighborhood propagation, then applies a weight matrix, with intermediate nonlinearities removed and weights collapsed into one linear layer.The four models represent distinct dataflows: GCN and GraphSAGE sparse–dense, GIN sparse-to-multiple-dense, and SGC multiple-sparse then dense computations.Source: §2.2.5 · pp. 6–7

3 FrontGI Design

FrontGI combines graph- and architecture-aware techniques for multicore GNN inference, separating aggregation from update and tailoring parallel execution to chiplet systems.

3 FrontGI Design

  • FrontGI targets scalable GNN inference on large graphs using techniques designed around GNN computation and multicore architecture.The design particularly targets high-core-count systems built with chiplets.Source: §3 · p. 8
  • A pipeline sends graph data from Aggregate to Update through a cache-sized buffer, avoiding writes of intermediate results to main memory.Separating the phases also replaces a large SpMM with per-node vector queries and dense vector-matrix multiplies.Source: §3 · p. 8
  • The Aggregate phase uses mixed tensor/node parallelism for dense graphs and hierarchical node parallelism for sparse graphs, while Update uses tailored BLAS GEMM parallelization.A feature-matrix buffer supports use of matrix libraries during Update.Source: §3 · p. 8

3.1 Graph Processing Systems for GNNs

General-purpose graph systems are not well suited to GNNs because aggregation combines irregular memory access with an update phase that is computationally intensive. The section describes separating and tuning these phases, with aggregation performed per node using CSR graph data.

3.1 Graph Processing Systems for GNNs

  • GNN processing combines irregular memory access in Aggregate with computationally intensive vector- or matrix-multiplication operations in Update.This differs from graph-processing algorithms whose performance is described as primarily dominated by irregular memory access.Source: §3.1 · p. 8
  • Executing Aggregate and Update separately in a pipeline allows the two phases to be tuned independently for their different performance concerns.For CSR-formatted graph data, the described Aggregate operation computes a node's feature vector; GCN uses a sum, which can be replaced with the relevant operation for other GNN models.Source: §3.1 · p. 9
  • The CSR aggregation procedure is applied to an individual node, with the aggregation operation changed to match the GNN model.The sum operation is identified as the GCN Aggregate operation.Source: §3.1 · p. 9

3.2 Parallelization of Aggregate Phase

Irregular aggregation accesses can load many unused node features into higher-level caches and cause frequent cache misses. Embedding dimension changes both feature-vector locality and the amount of unused feature data occupying cache.

3.2 Parallelization of Aggregate Phase

  • Irregular aggregation accesses load substantial amounts of feature data that will not be used into higher-level caches, contributing to frequent cache misses.Node features are stored as rows of a matrix, one feature vector per node.Source: §3.2.1 · p. 9
  • Higher embedding dimensions improve locality within an individual core's feature-vector accesses, but increase the proportion of cached node features that will not be accessed.Source: §3.2.1 · p. 9
  • The cache cost of irregular accesses is tied to feature-matrix loading: much of the data brought into cache may not be accessed during aggregation.Source: §3.2.1 · p. 9

CSR CSR

FrontGI selects Aggregate-phase parallelism according to graph structure and chiplet cache organization. Dense graphs use feature-dimension partitioning across chiplets, while sparse graphs use hierarchical node partitioning.

CSR CSR

  • For dense graphs, chiplets split feature dimensions and process partial aggregations with node-level parallelism; the partial results are then concatenated.Treating each chiplet's shared L3 cache as a device reduces feature dimensionality per chiplet, allowing more feature vectors to fit in its cache and reducing LLC-miss likelihood.Source: CSR CSR · p. 9, Figure 5 · p. 9

3.2.2 Hierarchical Node Parallelism

  • For low-degree graphs, tensor parallelism's partitioning, merging, and synchronization overhead can consume a large share of aggregation cost and reduce overall performance.For highly irregular graphs, greater LLC capacity does not ensure that needed, dispersed features are loaded into cache.Source: §3.2.2 · p. 11
  • For sparse graphs, hierarchical node parallelism partitions nodes across chiplets and processes each partition with node parallelism, avoiding tensor parallelism's feature splitting.The strategy uses an outer node partition and inner chiplet-level parallel execution.Source: §3.2.2 · pp. 11–12
  • On sparse graphs, tensor parallelism can increase the number of feature vectors loaded into LLC without increasing the number cores can retrieve, and reduced feature dimensions can degrade locality.Figure 6 contrasts memory accesses on dense and sparse graphs.Source: Figure 6 · p. 10, §3.2.2 · p. 11
  • For regular graphs, predictable neighbors support LLC reuse or prefetching; for uneven-degree graphs, fixed node ranges reduce Aggregate execution variability relative to single-level node parallelism.Source: §3.2.2 · p. 11
  • Hierarchical node assignment gives feature vectors a more deterministic inter-chiplet distribution than relying solely on OpenMP scheduling, though that distribution may not be optimal.Source: §3.2.2 · p. 12

3.3 Parallelization of Update Phase

FrontGI adapts the Update phase to chiplet-based cache topology, where classical BLAS cache assumptions do not capture the differing costs of intra- and inter-chiplet access. Its approach executes BLAS independently within compute units to reduce inter-chiplet communication while retaining optimized lower-cache kernels.

3.3 Parallelization of Update Phase

  • Classical BLAS assumes a unified cache model, but chiplet-based CPUs have costly cross-LLC communication and nonuniform private-cache-to-LLC latency.These differences can make matrix-multiplication cache reuse patterns effective under the classical abstraction degrade performance on chiplet-based systems.Source: §3.3 · p. 12
  • FrontGI runs high-performance BLAS independently within each compute unit, preserving low-level cache optimization while avoiding excessive cache-to-cache traffic.The design leaves BLAS itself unchanged; each unit calls cblas sgemm on its assigned block and writes to the corresponding output region.Source: §3.3 · pp. 12–13
  • Figure 8 compares OpenBLAS SGEMM with FrontGI’s matrix multiplication for GCN’s Update phase across four graphs at feature dimension 1024.The graphs are OGBN-Proteins, Reddit, Yelp, and OGBG-Molhiv; the passage does not state a winner or quantitative outcome.Source: §3.3 · p. 13
  • The Update strategy is particularly effective when embedding dimensionality is small relative to node count, improving weight-matrix reuse across row blocks.Preserving block locality and thread affinity also mitigates cross-chiplet data movement.Source: §3.3 · p. 13

3.4 Fusion of Linear Operators

When a linear operator lies between Aggregate and Update, or is embedded in Aggregate, FrontGI fuses it into the aggregation parallelism. The implementation follows the chosen parallel decomposition, with synchronization to merge partial results in the mixed-parallelism case.

3.4 Fusion of Linear Operators

  • FrontGI fuses linear operators into aggregation parallelism when they occur between Aggregate and Update or within Aggregate, as in GIN’s weighted-sum AXPY operation.The method uses the decomposability and composability of linear operators.Source: §3.4 · p. 13
  • Under hierarchical node parallelism, each chiplet performs Aggregate-Linear for approximately N/D nodes.Source: §3.4 · p. 13
  • Under mixed parallelism, chiplets process assigned feature dimensions with Aggregate-Linear, then synchronize to merge partial results.Source: §3.4 · p. 13

3.5 Memory Access Threads Placement

FrontGI localizes memory-intensive tensor partitioning and concatenation, along with simple activation functions such as ReLU, within as few chiplets as possible. It uses the available threads within those chiplets to maintain parallelism.

3.5 Memory Access Threads Placement

  • FrontGI confines dimension-wise partitioning, concatenation, and simple activations such as ReLU to as few chiplets as possible.The stated aim is to reduce inter-chiplet communication and maximize data locality.Source: §3.5 · p. 14
  • Within the selected chiplets, FrontGI uses all available threads to exploit parallelism and improve computational efficiency.Source: §3.5 · p. 14
  • The placement strategy targets memory-intensive operations and simple activation functions, rather than specifying a blanket restriction on other operations.Source: §3.5 · p. 14

4 Experiments

FrontGI was evaluated on 12 diverse graph datasets using a 128-core AMD EPYC system, with comparisons against DGL and PyG. Results show that the better-performing parallelization strategy varies with graph degree, while FrontGI scales to 128 cores and changes LLC behavior.

4.1.1 Datasets

  • The evaluation covers 12 datasets with average node degrees from 2.1 to 597.0, varied degree distributions, and 1024-dimensional, 32-bit floating-point features.Source: §4.1.1 · p. 14

4.1.2 Evaluation Methods

  • Tests use a 128-core AMD EPYC 9754 system and compare FrontGI with DGL 2.4.0 and PyG 2.6.1; OpenBLAS uses all 128 cores for the Update phase.The system has 16 chiplets, each with a shared 16 MiB L3 cache; the evaluation assigns 64 feature dimensions per chiplet for mixed parallelism and N/16 nodes per chiplet for hierarchical parallelism.Source: §4.1.2 · p. 14

4.2.1 Performance

  • For high-degree graphs, mixed tensor/node parallelism reaches approximately 3.35× DGL speed for GCN, 3.20× for GraphSAGE, 2.89× for GIN, and 3.10× for SGC.For lower-degree graphs, hierarchical node parallelism achieves speedups from 1.35× to 1.82× for GCN, 1.72× to 2.07× for GraphSAGE, 1.42× to 2.17× for GIN, and 1.17× to 2.59× for SGC.Source: Figure 9 · p. 16, Figure 10 · p. 16, Figure 11 · p. 16
  • On PC, FrontGI achieves up to 7.42×, 8.10×, 6.94×, and 15.76× PyG’s speed with different models; PyG also frequently runs out of memory on large, high-degree graphs.Source: §4.2.1 · p. 16
  • The performance analysis profiles GCN execution time by phase and calculates ROTG to relate mixed-parallelism overhead to its gain over hierarchical parallelism.Mixed parallelism’s Aggregate time begins decreasing above average degree 19; at approximately 46, its reduction offsets tensor-parallelism overhead.Source: Table 7 · p. 17, §4.2.1 · p. 17

4.2.2 Impact of Graph Properties

  • FrontGI’s speedup over DGL varies with graph degree: hierarchical node parallelism performs best below degree 20, strategies are similar around 20–50, and mixed tensor/node parallelism wins above 50.The best strategy is associated with average degree rather than degree-distribution uniformity in the reported examples.Source: §4.2.2 · p. 18

4.2.3 Scalability

  • FrontGI continues improving through 128 cores, while DGL’s speedup begins to plateau at 32 cores and its performance declines beyond 96 cores.The scalability experiment adds chiplets progressively and normalizes each graph’s 8-core execution to 1.Source: Figure 14 · p. 18, §4.2.3 · p. 18

4.2.4 LLC Hit/Miss

  • Mixed parallelism reduces LLC misses on high-degree graphs by 49.83%–87.54%, while hierarchical node parallelism reduces LLC accesses by 66.10%–83.17% on low-degree graphs.The reported LLC miss reductions for low-degree graphs range from 62.46% to 76.36%.Source: Table 8 · p. 19, §4.2.4 · p. 19

5 Related Work

Prior GNN acceleration work has concentrated on GPUs and FPGAs, using hardware- and dataflow-oriented techniques to address irregular memory access and computation.

5 Related Work

  • Earlier GNN acceleration research largely targets GPUs and FPGAs, applying kernel fusion, memory-access optimization, and customized accelerators to irregular workloads.These efforts seek to make GNN training and inference better suited to specialized hardware.Source: §5 · pp. 19–20

5.1 GNNs on GPUs

  • GPU methods include sampling pipelines such as GraphTensor and approaches that adapt sparse representations, workload balance, and computation to graph structure and tensor cores.GNNAdvisor also optimizes hierarchical GPU memory using input-graph and GNN-architecture properties.Source: §5.1 · p. 20

5.2 GNNs on FPGAs

  • FPGA work spans GCN acceleration, real-time frameworks, and HLS-based support for varied GNN models, while design-space exploration and long compilation times remain non-trivial.The cited passage reports that optimized and naive implementations of the same kernel can differ by two to three orders of magnitude.Source: §5.2 · p. 20

6 Conclusions and Future Work

FrontGI proposes two parallelization techniques for multicore GNN inference and reports gains of up to 3.35× relative to the higher-performing of two commonly used standard libraries. Which technique scales better depends in large part on graph average degree.

6 Conclusions and Future Work

  • FrontGI proposes two parallelization techniques motivated by modern processors’ memory hierarchies and designed for incorporation into high-productivity GNN environments.The authors assessed the techniques empirically across several GNN models and twelve representative graphs.Source: §6 · p. 21
  • 3.35× maximum performance gain relative to the higher-performing of two commonly used standard libraries.The evaluation used several GNN models and twelve representative graphs.Source: §6 · p. 21
  • Graphs with average degree below 20 scaled better with hierarchical node parallelism, while those above 50 scaled better with mixed tensor/node parallelism.For average degrees from 20 to 50, either FrontGI strategy proved effective.Source: §6 · p. 21
  • The evaluation covers GCN, GraphSAGE, GIN, and SGC; broader model and dataset coverage is needed before production-library adoption.The authors identify GAT graph attention and GNN training as possible extensions.Source: §6 · p. 21
Loading 2610.34911v1…