Source-linked AI summary
TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information
Dain Kwon, Kanghyun Choi, Hyeyoon Lee, Sunjong Park, Seoyong Lee, Sukjin Kim, Jinho Lee
TL;DR
Existing GNN quantization methods can incur substantial overhead, limiting practical deployment. TopGQ is a topology-aware post-training framework that combines dual-axis scale absorption with TopPIN-based node grouping. Across experiments, it reduces quantization time by an order of magnitude or more while maintaining accuracy.
Problem
Existing GNN quantization methods incur substantial quantization overhead, limiting their practical use in real-world scenarios.
Method
TopGQ combines dual-axis scale absorption for activation quantization with TopPIN, a topology-based index for grouping nodes and assigning quantization parameters.
Results
TopGQ achieves QAT-level accuracy while reducing quantization time by an order of magnitude compared with prior work.
Takeaways & Limitations
TopGQ provides a practical accuracy-speed balance for small- and large-scale GNN quantization.
Takeaways & Limitations
On-the-fly parameter computation can add runtime overhead that offsets quantization efficiency gains.
Abstract
from arXiv · showhide
Existing GNN quantization methods suffer from considerable quantization overhead, which severely limits their practical usage in real-world scenarios. To this end, we present TopGQ, an accurate post-training GNN quantization framework, alleviating redundant quantization overhead. We propose dual-axis scale absorption, which enables activation quantization along both the outer and inner dimensions by merging one into the adjacency matrix. On top of that, we introduce TopPIN, a proxy for nodes' local structure, and use it to group nodes with similar topology during quantization. Experimental results show that TopGQ reduces quantization time by an order of magnitude while preserving accuracy.
1 Introduction
GNN quantization can reduce memory and computation, but existing approaches impose prohibitive time overhead. TopGQ addresses this with topology-aware post-training quantization while preserving task performance.
- Low-bit quantization reduces GNN memory and computational costs for processing large real-world graphs.
- Existing GNN quantization can take hours on Reddit and days on MAG240M, making deployment infeasible.The overhead arises from extensive outlier tuning, retraining, or gradient-based iterations on quantization parameters.
- Frequent model updates in personalization and recommendation require quantization on minute-to-hour timescales, making excessive overhead a deployment barrier.
- TopGQ combines dual-axis scale absorption with the topology-aware TopPIN index in a post-training quantization framework.Scale absorption merges node scaling into the adjacency matrix, while TopPIN assigns parameters to unseen nodes using local topology.
2 Background
The background defines GNN message passing, graph settings, and uniform quantization. It explains why inductive inference and fine-grained quantization are especially relevant to practical GNN acceleration.
- Graph neural networks: A GNN represents a directed graph with an adjacency matrix and updates each node using its closed in-neighborhood and degree.
- Graph neural networks: GNN message passing combines node features with layer weights and aggregates neighboring information through permutation-invariant operations.
- Graph neural networks: Matrix-form GNN computation uses an architecture-specific adjacency operator, including normalized adjacency for GCN and binary adjacency for GIN.
- Transductive and inductive settings: Inductive GNN inference introduces unseen nodes or graphs at test time, requiring fresh node-embedding computation and making quantization especially valuable.This setting reflects evolving graphs and new users, molecules, or other unseen entities.
- Quantization: Uniform quantization uses scale and zero-point parameters to replace high-precision operations with low-bit integers.Quantization granularity may be per-tensor, per-row, or per-column; finer granularity can reduce outlier-induced error, but scale placement constrains these choices.
- Quantization: PTQ calibrates scale and zero-point without modifying pretrained weights, whereas QAT updates weights through backpropagation under simulated quantization.The paper uses PTQ because this calibration-based process is substantially faster in practice.
3 Motivations & Challenges
GNN message passing creates node-dependent activation ranges, favoring node-wise quantization but complicating aggregation and unseen-node inference. TopGQ addresses these challenges with scale absorption and topology-based parameter mapping.
- Node-wise activation ranges are more concentrated than feature-wise ranges, supporting node-wise quantization for GNN activations.The 5th–95th percentile ranges lie close to the min–max ranges for nodes.
- Node-wise quantization is straightforward for the combination operation because node-wise activation scales and feature-wise weight scales preserve standard integer GEMM.
- Aggregation makes node-wise quantization difficult because an inserted diagonal scale matrix cannot be computed by integer matrix multiplication units.Column-wise quantization preserves acceleration but may lose activation precision.
- Dual-axis scale absorption merges node-wise scales into the static adjacency matrix to retain node-wise precision while enabling integer aggregation.
- On-the-fly parameter computation scans every unseen node’s activation row for minimum and maximum values, but its runtime overhead can offset quantization gains.
- TopPIN groups nodes with similar expected activation patterns using local topology, enabling low-overhead assignment of quantization parameters to unseen nodes.
4 TopGQ Methodology
TopGQ combines dual-axis scale absorption with TopPIN-based node grouping to support efficient quantization across aggregation and inductive inference. It calibrates layer-wise quantization choices and reuses group-specific parameters for unseen nodes.
- Selective Dual-axis Scale Absorption: Dual-axis scale absorption merges node-wise scaling into the static adjacency matrix, enabling integer matrix multiplication during aggregation.The node-wise factor S_N scales X_c before being merged into the normalized adjacency matrix.
- Selective Dual-axis Scale Absorption: TopGQ selects dual-axis or feature-wise quantization per layer by comparing mean squared error against floating-point activations.The lower-MSE configuration is saved for inference.
- TopPIN: A Fast Index for Unseen Nodes: TopPIN supports inductive quantization by mapping unseen nodes to calibrated groups and computing group parameters from aggregated node statistics.For nodes sharing a TopPIN value, calibration uses global minimum and maximum statistics; inference then uses the corresponding group parameters.
- TopPIN: A Fast Index for Unseen Nodes: TopPIN indexes nodes by local topology so nodes with similar neighborhood aggregation characteristics can share similar quantization parameters.The index approximates accumulated activation variance using graph-structure information, including node indegree and degree-normalization effects.
5 Experimental Results
Across node- and graph-level evaluations, TopGQ substantially reduces quantization time while matching or improving task accuracy across GNN architectures and graph sizes.
- Evaluation Setup: TopGQ was evaluated on node- and graph-level tasks using GCN, GraphSAGE, GIN, and GAT at 4-bit and 8-bit precision.Datasets included Cora, CiteSeer, Reddit, ogbn-products, MAG240M, IMDB-BINARY, and COLLAB.
- Node-level Results: 16.35h versus less than a minute: TopGQ reduced Reddit GraphSAGE quantization time while matching or exceeding baseline accuracy.Across graph sizes from Cora and CiteSeer to Reddit and ogbn-products, TopGQ was the fastest method.
- Node-level Results: 28.87%p: TopGQ improved INT4 GCN accuracy over the strongest baseline on node classification.The reported maximum gain occurred across the evaluated node-classification settings.
- Architecture Generalization: 32.92%p: TopGQ achieved at most this 4-bit accuracy gain over baselines on GAT while keeping Cora quantization under half a second.TopGQ also preserved the original model’s accuracy across GNN structures with learnable edge weights.
- Graph-level Results: 2.40 hours versus 13.85 seconds: On COLLAB with GCN, TopGQ cut quantization overhead while achieving superior accuracy to Degree-Quant.Across graph-classification datasets, TopGQ often delivered the best accuracy with minimal degradation compared with FP32.
- Inference Efficiency: TopGQ avoids repeated row-wise scans by storing quantization parameters and retrieving them through TopPIN-based mapping during inference.This design targets practical deployment on GPUs and edge devices, where A2Q and on-the-fly PTQ incur expensive parameter derivation.
- TopPIN and Ablation: TopPIN indexing groups nodes by local topology, while dual-axis scale absorption preserves node-wise quantization effects across layers.The ablation shows that TopPIN alone can be insufficient as graph size increases, whereas adding scale absorption recovers additional accuracy.
6 Related Work
Prior GNN quantization methods incur substantial overhead through retraining, mixed-precision handling, or gradient optimization, while topology-based methods have not related topology to node feature patterns.
- GNN Quantization: Existing GNN quantization methods require significant overhead from costly retraining, mixed-precision assignment, or gradient-based optimization.Degree-Quant, SGQuant, and A2Q use QAT, while DRA optimizes quantization parameters with gradients.
- GNN Quantization: Degree-Quant excludes high-degree node activations during calibration and compresses them later at inference.
- GNN Quantization: SGQuant and A2Q use mixed precision to assign higher bitwidths to high-magnitude features.
- Topology in GNNs: Prior topology-aware methods use structural measures such as degree or betweenness centrality, but do not relate topology to node feature patterns.
7 Conclusion
TopGQ is a topology-aware PTQ framework that removes retraining costs while preserving task accuracy. It combines TopPIN with dual-axis scale absorption to support fast, precise quantization and reports QAT-level accuracy with an order-of-magnitude reduction in quantization time.
- Conclusion: TopGQ eliminates retraining costs while preserving task accuracy through topology-aware post-training quantization.
- Conclusion: TopPIN and dual-axis scale absorption handle unseen node features with differing magnitudes while preserving integer-operation benefits.
- Conclusion: An order of magnitude reduction in quantization time accompanies QAT-level accuracy across various GNN architectures and datasets.