Source-linked AI summary
Engineering Low-Latency, High-Throughput Quantum Decoders on NVIDIA GPUs
Stefano Markidis, Luca Pennati, Ivy Peng
TL;DR
QEC decoders must process measurement data at least as fast as it is generated, with growing logical-qubit counts increasing aggregate decoding demand. This paper maps Relay-BP with OSD-0 to NVIDIA GPUs using execution and precision optimizations. Across workloads, RTX 5080 can beat A100 configurations under fixed response-time limits, while GH200 reaches the highest measured throughput.
Problem
Fault-tolerant quantum computing requires classical decoders to process measurement data at least as fast as it is generated, while growing logical-qubit counts increase aggregate decoding throughput requirements.
Source: §I · p. 1Method
The paper implements Relay-BP with OSD-0 on NVIDIA GPUs, combining GPU convergence control and sample queues with optimized sparse kernels, warp-level binary elimination, and reduced-precision formats.
Source: §I · pp. 1–2, §II · p. 2Results
Across all three workloads, RTX 5080 configurations exceed every A100 frontier point in throughput with shorter response, while GH200 reaches the highest measured throughput.
Source: §V · p. 7Takeaways & Limitations
The results support commodity GPU clusters as programmable HPC infrastructure for QEC, combining competitive decoding throughput with low-latency responses.
Source: §VII · p. 8
Abstract
from arXiv · showhide
We design and optimize a quantum error correction (QEC) decoder for NVIDIA GPUs, combining Relay belief propagation, a state-of-the-art message-passing algorithm, with order-zero ordered statistics post-processing. Our implementation uses CUDA Graphs to control GPU execution, stops updates for converged samples, and collects unresolved samples in queues for post-processing. We optimize sparse traversal through coalesced message accesses, prefetching, and register reuse, and accelerate binary post-processing with warp-level collectives. In addition, mixed FP16-FP32 arithmetic and INT8 quantization reduce message storage and traffic while retaining accuracy using accumulators at higher precision. We evaluate surface, hypergraph product, and bivariate bicycle code workloads on NVIDIA A100 and GH200 data-center GPUs and a commodity RTX 5080. On the bivariate bicycle code at error probability 0.003, INT8 halves amortized decoding time relative to independently tuned FP32 with a negligible logical error rate increase on RTX 5080. With independent syndrome batches submitted back-to-back to concurrent decoder instances, RTX 5080 delivers 91% of GH200's highest measured throughput with an 8.9x shorter median submission-to-correction latency. The combination of performance optimization, mixed precision, INT8 quantization, and concurrent decoder instances opens a path toward QEC decoder clusters built from commodity GPUs.
I. INTRODUCTION
QEC decoding must keep pace with growing syndrome-data rates, and independent requests provide parallel work for the classical system. This work maps Relay-BP with OSD-0 onto NVIDIA GPUs and evaluates decoder performance, accuracy, and response-time tradeoffs.
I. INTRODUCTION
- The work evaluates latency, throughput, and logical accuracy to assess GPUs as programmable devices for QEC workloads.The introduction frames QEC decoding as sparse graph processing with irregular neighborhoods and low-arithmetic edge updates.Source: §I · p. 1
- The decoder combines parallel graph updates within each instance with concurrency across independent syndrome batches, returning corrections for the controller’s Pauli frame.Samples share graph connectivity but retain independent messages and convergence state.Source: §I · p. 1
- The decoder adds GPU convergence control, sample retirement, compacted fallback queues, and ordered multi-warp binary elimination.These mechanisms support Relay-BP updates and OSD-0 post-processing.Source: §I · p. 2
- The optimized decoder studies mixed FP16-FP32 arithmetic, FP16 arithmetic, and INT8 quantization for their effects on memory, convergence, and full-decoder performance.The design targets high throughput and short correction latency while quantifying precision effects on accuracy.Source: §I · pp. 1–2
- The evaluation spans three circuit-level code families and three NVIDIA GPU architectures, profiling kernels and measuring latency-throughput tradeoffs.It examines precision, batch size, and stream configurations for different response-time requirements.Source: §I · p. 2
II. A QUANTUM ERROR CORRECTION WORKFLOW ON NVIDIA GPUS
The section establishes the sparse graph and decoding algorithms underlying the GPU decoder. It describes circuit-level syndrome generation, Relay-BP message updates, and OSD-0 fallback for samples that do not converge.
QEC Sparse Matrices and Decoding Graphs
- The sparse binary check matrix H maps error variables in columns to parity checks in rows, with each nonzero defining an edge in its bipartite Tanner graph.Row and column traversals access check and variable neighborhoods, respectively.Source: §II · p. 2, Figure 2 · p. 2
- Circuit-level noise assigns error probabilities to gates, initialization, idle intervals, and measurements; a detector error model records each mechanism’s probability and flipped detectors.Stim can generate syndrome samples and derive the detector error model used to supply H and error probabilities.Source: §II · p. 2
- For each syndrome sample, the decoder seeks a binary error pattern satisfying H e = s (mod 2), with error probabilities guiding the choice among compatible patterns.A batch supplies multiple independent right-hand sides for the same sparse matrix.Source: §II · p. 2
Message-Passing QEC Decoders
- Relay-BP updates check-to-variable and variable-to-check reliability messages, retires samples whose hard decisions reproduce the syndrome, and sends unresolved samples to OSD-0.Relay memory carries the preceding leg’s final posterior into the next leg; OSD-0 orders variables by posterior confidence and uses Gaussian elimination to construct one candidate correction.Source: §II · pp. 2–3
III. PERFORMANCE OPTIMIZATION ON NVIDIA GPUS
The section optimizes sparse traversal, convergence scheduling, binary elimination, and message precision for GPU execution. It also describes exploratory kernel variants that were evaluated but not retained.
Data Layout and Thread Mapping
- CSR check neighborhoods and CSC variable-edge indices provide direct access, while sample-contiguous edge-major messages coalesce accesses across the batch.The sparse traversal prefetches messages into registers and reuses cached incoming values for variables with at most four neighbors.Source: §III · p. 3
Convergence and Work Scheduling
- A CUDA Graph controls Relay iteration and GPU-side convergence, retires converged samples, and compacts unresolved IDs into a fallback queue without moving payloads.Chunked OSD-0 execution bounds workspace because per-sample basis storage grows approximately quadratically with the number of checks; decoder instances use separate streams, buffers, and graphs.Source: §III · p. 3, Figure 3 · p. 4
Warp-Level Collectives and Ordered Multi-Warp OSD-0
- Warp-level shuffles support neighborhood reductions and packed binary operations; eight candidate reductions run concurrently while reliability-ordered basis selection is preserved.For BB90, each candidate column uses thirteen 64-bit words, limiting parallelism within a single candidate.Source: §III · pp. 3–4
Reduced Precision, Mixed Precision, and Quantization
- For a batch of 128 BB90 samples, the two directed message arrays occupy about 178 MiB in FP32 and 45 MiB in INT8.The fourfold reduction also lowers the message-memory cost of concurrent decoder instances, which keep private message arrays.Source: §III · p. 4
- INT8 messages use scaled reliabilities clipped to magnitude 20, while variable updates accumulate in INT32 and retain posterior beliefs and Relay memory in INT16.The wider retained state preserves posterior magnitudes beyond edge-message clipping for OSD-0 reliability ordering; Relay blending uses fixed-point coefficients and INT64 intermediate products.Source: §III · p. 4, Figure 4 · p. 4
Additional Optimization Experiments
- Additional experiments tested packed half2 and uchar4 accesses, 32-bit packed binary vectors, and Tensor Core 1-bit operations, but these variants were not retained.The packed-access variants targeted load/store instruction overhead; Tensor Core instructions were tested on dense binary tile products.Source: §III · p. 5
IV. EXPERIMENTAL SETUP
The experiments compare complete Relay-BP/OSD-0 decoding across three code workloads, noise levels, GPU platforms, and numerical precisions. They also measure burst-load response and throughput, and profile BB90 kernels.
Workloads and Accuracy
- The evaluation covers rotated surface, hypergraph product, and bivariate bicycle codes at p ∈{0.001, 0.003, 0.005}, with logical error rate measured after complete decoding.Identical samples are used across precisions and GPUs; logical verification compares a predicted logical Z flip with Stim’s sampled value.Source: §IV · p. 5
Decoder Configuration and Timing
- Amortized decoding time includes graph launch, Relay-BP, compaction, OSD-0, and internal synchronization, but excludes setup, transfers, and verification.Each GPU/workload/noise/precision configuration is independently tuned over batch sizes 32–2048 using 2048 samples.Source: §IV · p. 5
Decoding under Burst Load
- Burst-load tests submit 32 independent batches to 1–4 A100/GH200 or 1–8 RTX 5080 decoder instances, measuring response time and burst throughput.Requests include pinned-host upload, graph execution, and correction download; response time excludes waiting for an available instance.Source: §IV · p. 5
Kernel Profiling
- BB90 INT8 profiling at p = 0.003 records compute and memory utilization, occupancy, issue activity, memory stalls, cache hit rates, and registers per thread.SM and DRAM SOL express throughput relative to sustained peak.Source: §IV · p. 5
V. RESULTS
The results show that tuned batch and stream configurations let RTX 5080 approach GH200's peak burst throughput with shorter responses, while precision choices affect complete-decoder time and logical error rate. Kernel-level findings identify memory dependencies and resource use as factors in GPU performance.
Finding 1
- For BB90 mixed precision on A100 at p = 0.003, increasing batch size from 32 to 2048 reduces amortized decoding time from 1466 to 439 µs/sample, a 3.34× speedup.The selected batch depends on noise: at p = 0.001, the selected size is 1024, and doubling it to 2048 increases amortized time by 10.3%.Source: §V · pp. 5–6
Finding 2
- On A100 for BB90 in FP32 with B = 256 at p = 0.003, conditional CUDA Graph execution takes approximately 614 µs/sample versus 607 µs/sample for static execution.At p = 0.001, conditional execution reduces time from 229 to 191 µs/sample; the main contribution is a unified workflow with device-side control.Source: §V · p. 6
Finding 3
- For RTX 5080 BB90 at p = 0.003, INT8 reduces amortized time from 416 to 207 µs/sample, while observed LER changes from 0.62% to 0.65%.At p = 0.005, the INT8 time reduction reaches 55.6%, but observed LER rises from 5.19% to 5.51% and OSD fallback reaches 30.85%, versus 4.73% at p = 0.003.Source: §V · p. 6, Table III · p. 6
Finding 4
- On A100, packed reduction is 20–21× faster than a one-thread-per-sample reference, and eight concurrent candidate reductions further speed the solver kernel by 2.25×.For BB90 at p = 0.005 in FP32 with B = 256, complete-decoder time decreases by 8.8% with unchanged logical-error counts.Source: §V · p. 6
Finding 6
- At the selected INT8 batch sizes for BB90, Belief/Relay L2 hit rates are 74% on A100 and 65% on GH200, versus 36% on RTX 5080.RTX 5080 reaches 55% DRAM SOL at 0.52 TB/s, while GH200 transfers 1.13 TB/s at 28% SOL.Source: §V · p. 7, Table IV · p. 7
Findings 7 and 5
- On A100, Parity/min-sum has 51% issue activity and 1.32 eligible warps per scheduler at 97% occupancy, while Belief/Relay has 65% and 2.05 at 73% occupancy.For 65,536 packed binary products with 8×8 output tiles and inner dimension 128, Tensor Core time falls from 30.77 to 27.43 µs, a 1.12× speedup.Source: §V · pp. 6–7
Finding 8
- At p = 0.003, Surface on GH200 reaches 14.30 kdecodes/s at 8.94 ms with B = 32 and S = 4, the best measured configuration within a 10 ms response requirement.For B = 16, raising streams from one to four increases throughput from 3.01 to 9.02 kdecodes/s while median response rises from 5.93 to 7.78 ms.Source: §V · p. 7, Figure 5 · p. 8
Finding 9
- RTX 5080 delivers 91% of GH200’s BB90 throughput with an 8.9-times shorter median response, while GH200 reaches the highest measured throughput across code families.For BB90, RTX delivers 5.95 kdecodes/s at 17.6 ms; GH200 delivers 6.50 kdecodes/s at 156 ms.Source: §V · p. 7, Figure 5 · p. 8
VI. RELATED WORK
The related work situates BP with OSD among established QLDPC decoders and describes prior algorithmic, hardware, GPU, and scheduling approaches.
Related work
- Prior work covers BP with OSD for quantum LDPC codes, Relay-BP’s memory and relay ensembling [4], localized statistics decoding, and FPGA/ASIC implementations.The passage also identifies commodity-GPU BP decoding, CUDA-Q QEC decoders, parallel retries and streaming [12], and provisioning for bursty demand.Source: §VI · p. 8
- Knapen et al. develop qLDPC hardware predecoders that reduce OSD invocation and add Relay-BP legs [14].Source: §VI · p. 8
VII. CONCLUSION
The paper implements and evaluates an all-GPU Relay-BP/OSD-0 decoder across three code families and three NVIDIA GPU architectures. Its conclusion is that small concurrent batches let a commodity RTX GPU approach GH200’s peak burst throughput with substantially shorter responses.
Conclusion
- The evaluation covers decoding time, logical recovery, and latency-throughput tradeoffs for concurrent requests across three circuit-level code families and three GPU architectures.Source: §VII · p. 8
- The authors argue that the results support commodity GPU clusters as programmable QEC infrastructure, potentially alongside quantum controllers in fault-tolerant computers.Source: §VII · p. 8