TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information

1 Introduction

• Hours for Reddit and days for MAG240M make existing GNN quantization impractical, with outlier handling requiring extensive tuning or lengthy training.QAT methods require costly retraining [7] [25]. Existing PTQ methods still use gradient-based quantization-parameter iterations, negating PTQ’s expected speed advantage.Source: §1 · p. 1
• Minute-to-hour model updates [23] [24] make excessive quantization time a deployment barrier for large-scale personalization and recommendation applications [6] [9] [29] [33].Low-bit representations reduce memory and computational costs [2] [20] [22], but excessive quantization time can cancel these benefits.Source: §1 · pp. 1–2
• Orders of magnitude faster quantization with comparable or better task performance is reported for TopGQ relative to current state-of-the-art baselines.The reported improvements preserve inference speed as well as accuracy.Source: §1 · p. 2
• Dual-axis scale absorption merges node scaling factors into the adjacency matrix, enabling fast, accurate integer matrix multiplication during aggregation.Outlier nodes make node-wise quantization preferable, but applying it directly during aggregation prevents fast integer arithmetic.Source: §1 · p. 2
• TopPIN is a quickly computed topology-based node index that enables rapid assignment of quantization parameters to unseen nodes, supporting fast inference.Source: §1 · p. 2

Graph neural networks

• GNN message passing combines node features with layer weights, then aggregates neighboring information using a permutation-invariant operator and an update function.A node’s closed in-neighborhood includes its incoming neighbors and itself; its size defines the degree used in the diagonal degree matrix D. Matrix formulations use node-feature, weight, and adjacency matrices.Source: §2 · p. 2
• GCN [18] uses D^-1/2(A + I_n)D^-1/2, GIN [31] uses A + I_n, and GraphSAGE [10] samples neighbors rather than aggregating the entire neighborhood.The adjacency operator therefore depends on the GNN architecture.Source: §2 · p. 2

Transductive and inductive settings

• Quantization is especially valuable in inductive settings, where unseen nodes or graphs require embedding computation during inference and reduced computation and memory directly accelerate inference.Transductive inference can use precomputed embeddings because training exposes test-node features and topology, but not labels, leaving limited acceleration opportunities. Inductive settings reflect evolving social networks, recommendation systems with new users, and prediction for unseen molecules.Source: §2 · p. 2

Quantization

• Finer quantization granularity can reduce outlier-induced error, but dequantization requires scale factors along GEMM’s outer dimension, restricting per-row or per-column choices.Uniform quantization replaces floating-point operations with low-bit integers using a scale and zero-point, reducing computation and memory. Granularities include per-tensor, per-row, and per-column.Source: §2 · p. 2
• PTQ calibrates scales and zero-points without modifying pretrained weights, making it substantially faster than QAT; TopGQ adopts this approach.QAT instead incorporates simulated quantization during training and iteratively updates model weights using backpropagated gradients.Source: §2 · p. 2

Motivation

• Node-wise activation quantization better accommodates message-passing-induced diversity, while feature-wise ranges contain outliers that waste quantization bins and increase error.Within nodes, the 5th–95th percentile range lies close to the min–max range; within features, most values occupy a much narrower interval than the full range.Source: §3 · p. 3

Combination-phase quantization

• Existing methods [7] already support efficient node-wise combination quantization by pairing node-wise activation scales with feature-wise weight scales.Multiplication remains standard integer GEMM, with scales applied afterward for dequantization, maintaining high throughput.Source: §3 · p. 3

Challenge 1: Quantization along inner dimensions

• TopGQ proposes dual-axis scale absorption because node-wise aggregation scales interrupt standard integer GEMM, creating a trade-off between activation precision and computational efficiency.Node-wise scales introduce a diagonal matrix inside aggregation multiplication. Existing methods [7] instead use column-wise activation quantization, enabling acceleration but potentially sacrificing precision.Source: §3 · p. 3

Challenge 2: Generalization on unseen nodes

• TopPIN uses local topology to map unseen nodes to precomputed quantization parameters with low inference overhead.Parameters are calibrated on training nodes; the mapping requires a low-complexity index whose similar values indicate similar feature statistics. Inference uses parameter lookup rather than per-node min–max scanning, which provides low quantization error but can offset efficiency gains.Source: §3 · p. 3

4.1 Selective Dual-axis Scale Absorption

• TopGQ selects dual-axis or feature-wise quantization independently for each layer by choosing the configuration with lower activation mean squared error during calibration.The selected configuration is saved for inference, and node-wise scales are calibrated when dual-axis absorption is selected. At inference, multiplying node-wise and feature-wise scales provides element-wise quantization parameters.Source: §4.1 · pp. 3–4
• Dual-axis scale absorption merges node-wise scaling into the static adjacency matrix, retaining node-wise scaling benefits while using feature-wise quantization compatible with integer matrix multiplication.Each node’s scale is its maximum feature value, used to normalize its features before absorption into the adjacency matrix.Source: §4.1 · pp. 3–4

4.2 TopPIN: A Fast Index for Unseen Nodes

• TopPIN groups nodes with similar local aggregation characteristics so they can share quantization parameters suited to their expected activation distributions.For unnormalized GNNs, activation-variance accumulation is proportional to the number of paths reaching a node; denser neighborhoods accumulate more variance.Source: §4.2 · p. 4
• TopPIN uses first-order approximations of aggregation patterns, replacing higher-order terms with constants and accounting for degree-normalization factors in normalized GNNs.The derivation uses node indegree; the authors report that these approximations retain most benefits with minimal computational overhead across GNN architectures.Source: §4.2 · p. 4
• TopPIN calibration pools extrema within each index group; inference interpolates parameters from the k-nearest TopPIN groups for each unseen node.Pooling each group’s global maximum and minimum ensures its quantization parameters cover the full observed range.Source: §4.2 · p. 4

5.1 Experimental Settings

• 4-bit and 8-bit evaluations compare TopGQ against three QAT baselines and one PTQ baseline on node- and graph-classification tasks.Baselines are SGQuant [7], Degree-Quant [25], A2Q, and DRA, with fixed bitwidths across layers. GCN [18], GraphSAGE [10], GIN [31], and GAT [26] cover five node datasets and two graph datasets; evaluation is inductive except transductive MAG240M, and graph accuracy uses 10-fold cross-validation.Source: §5.1 · p. 4

5.2 Evaluation Results of Node-level Tasks

• Up to 28.87%p higher INT4 GCN accuracy than the strongest baseline accompanies TopGQ’s fastest quantization across tested graph sizes.Across small, medium, and large node-classification graphs, TopGQ matches or exceeds baseline accuracy. Baselines require up to 16.35 hours on Reddit with GraphSAGE, whereas TopGQ takes less than a minute.Source: Table 1 · p. 5, §5.2 · p. 5
• Up to 32.92%p accuracy gains over baselines occur with 4-bit GAT quantization, demonstrating TopGQ’s applicability to dynamic edge weights.TopGQ preserves original-model accuracy on GAT. On Cora, it achieves the best accuracy while quantizing in under half a second.Source: Table 2 · p. 5, §5.2 · p. 5

5.3 Evaluation Results of Graph-level Tasks

• 13.85 seconds replaces Degree-Quant’s 2.40 hours for GCN quantization on COLLAB, while TopGQ achieves superior accuracy.Across IMDB-BINARY and COLLAB, TopGQ quantizes fastest and often achieves the best accuracy with minimal degradation relative to FP32. The authors attribute this performance to topology-aware design using TopPIN.Source: Table 3 · p. 5, §5.3 · p. 5

5.2 Evaluation Results of Node-level Tasks

• At least 50× faster quantization brings MAG240M runtime to 58.8 minutes, versus 2.06–5.50 days for other methods, with negligible accuracy differences from FP32.MAG240M contains 240 million nodes. Degree-Quant accuracy is omitted because its 37.5-day training time exceeds the computational budget.Source: §5.2 · p. 5

5.4 Evaluation Results of Inference Latency

• TopGQ enables efficient inference through negligible-overhead TopPIN retrieval, avoiding the activation scans that slow A2Q and on-the-fly PTQ.Minibatched ogbn-products measurements cover GPU and edge devices. A2Q and on-the-fly PTQ scan intermediate activations row-wise to derive quantization parameters, motivating stored parameters with efficient retrieval.Source: Table 4 · p. 6, §5.4 · p. 6

5.5 Analysis on TopPIN and Ablation Study

• TopPIN outperforms naive PTQ and centrality-based indexing while reducing computational overhead through dependence on only 1-hop neighborhood information.Naive global quantization loses accuracy because node magnitudes vary substantially. Betweenness, closeness, and Katz centrality can mitigate this degradation but require costly graph traversal per node.Source: §5.5 · p. 6, Table 5 · p. 6
• Dual-axis scale absorption supplements TopPIN by preserving node-wise quantization effects across layers, recovering accuracy where topology-based indexing alone becomes insufficient.The incremental ablation adds TopPIN and selective dual-axis scale absorption to naive PTQ. TopPIN improves quantization-bin utilization, but alone is insufficient as graph size increases, exemplified by GCN on ogbn-products.Source: Table 6 · p. 6, §5.5 · p. 6

6 Related Work

• Orders of magnitude shorter quantization time distinguishes TopGQ from prior methods with significant, redundant quantization overheads.DRA optimizes quantization parameters using gradients to reconstruct FP32 distributions.Source: §6 · p. 6
• Degree-Quant [25] uses QAT, while SGQuant [7] and 𝐴2𝑄 use mixed-precision QAT to assign higher bitwidths to high-magnitude features.Degree-Quant [25], the first GNN QAT method, excludes high-degree node activations during calibration for robust quantization parameters, then compresses them at inference.Source: §6 · p. 6
• Topology-aware GNN training uses degree centrality to identify central nodes and betweenness centrality to weight nodes during aggregation.These approaches integrate graph structure to support representation learning.Source: §6 · p. 6
• Prior topology-based GNN binarization methods [3] [17] do not incorporate topology in relation to node feature patterns for GNN quantization.Source: §6 · p. 6

• An order-of-magnitude reduction in quantization time versus prior works accompanies TopGQ’s QAT-level accuracy across various GNN architectures and datasets.Source: §7 · p. 6
• TopGQ combines the TopPIN node index with dual-axis scale absorption to handle unseen node features with differing magnitudes.Source: §7 · p. 6
• TopGQ eliminates retraining costs, while its node-level strategies enable fast, precise quantization that preserves the computational benefits of integer operations.Source: §7 · p. 6