Source-linked AI summary
Python in the front, party in the Backline: compiling quantum workloads across CPUs, GPUs, and FPGAs
Joseph K. L. Lee, Mehrdad Malekmohammadi, Hong-Sheng Zheng, Shuli Shu, Cheick Doumbia, Kalman Szenes, Mehran Zamani Abnili, Thomas Ainsworth, Matthew Seymour, Thomas Germain, Leonhard Neuhaus, Josh Izaac, Lee J. O'Riordan
TL;DR
Fault-tolerant quantum workloads combine demanding QEC timing requirements with increasingly heterogeneous hardware and complex software coordination. Backline adds a Python-facing heterogeneous compilation and runtime layer to PennyLane and Catalyst. In hardware-handshake measurements over RoCE v2, steady-state median round trips were 2.305 µs to a CPU and 4.5 µs to a GPU.
Problem
Fault-tolerant quantum computing requires low-latency QEC across increasingly complex and heterogeneous systems, while connecting high-level algorithm development with low-latency execution is challenging.
Source: §I · pp. 1–2, §II · p. 3Method
Backline adds a heterogeneous compilation and runtime layer to PennyLane and Catalyst, with placement-defined device interactions and data movement.
Source: 1) Data placement abstraction · p. 6, §VI · p. 14Results
2.305 µs to a CPU and 4.5 µs to a GPU were the measured median steady-state round-trip latencies over RoCE v2 using an FPGA controller and hardware-handshake engine.
Source: §VI · p. 14Takeaways & Limitations
Backline’s placement model makes device interactions and transfers implicit across heterogeneous workload participants.
Source: 1) Data placement abstraction · p. 6Reported steady-state measurements omit round 0, which includes one-time endpoint initialization.
Source: C. FPGA experiment results · p. 20
Abstract
from arXiv · showhide
Moving from quantum research and development to production-grade, fault-tolerant quantum workload execution remains one of the most significant challenges facing quantum platform builders. While Python frameworks have enabled an easy entry point for quantum algorithm design, the low-latency requirements for real-time quantum error correction (QEC) demand performance that traditional interpreted environments cannot provide. FPGAs and ASICs play a central role at these layers, but their specialized programming models make development rigid and time-consuming. CPUs, GPUs, and other accelerators introduce a different challenge: as infrastructure becomes increasingly heterogeneous, programming across different devices and their associated abstractions becomes more complex. Allowing researchers to write workloads in high-level languages that map to low-latency execution across diverse distributed target platforms will enable the development of key infrastructure for utility-scale quantum systems. For this, we introduce $\textit{Backline}$, a heterogeneous compilation and runtime framework built within PennyLane and Catalyst. Backline allows us to design and build quantum-classical workloads for high-performance and low-latency devices, with compilation directly from a Python interface through MLIR. We demonstrate the compilation and execution of several quantum workloads with low-latency data movement across a mix of CPUs, GPUs, and FPGAs, for both local and distributed remote hardware targets, all from a vendor-agnostic Python frontend. With an AMD VPK120 FPGA board as the controller, issuing each round from its hardware-handshake engine, we measured median steady-state round-trip latencies over RoCE v2 of $2.305~\mu$s to an AMD Ryzen Threadripper PRO CPU and $4.5~\mu$s to an AMD Instinct MI210 GPU across $10^6-1$ rounds per path, demonstrating microsecond-scale synchronous co-processing.
I. INTRODUCTION
Fault-tolerant quantum workloads require fast, programmable coordination across specialized and heterogeneous hardware, yet both low-level device programming and distributed software complexity impede development. The paper motivates layered Python-to-hardware workflows and situates QEC workloads within distinct latency regimes.
I. INTRODUCTION
- QEC requires tight communication and feedback loops, while specialized FPGA and ASIC programming remains rigid and time-consuming.Researchers also face workloads spanning compute-bound and communication-bound behavior.Source: §I · p. 1
- As quantum systems scale, software spanning quantum and distributed computing grows more complex, while premature standardization can entrench suboptimal solutions.The authors favor an evolving, best-effort compatibility approach because QEC platforms are bespoke and target-platform variety remains limited.Source: §I · pp. 1–2
- Python-fronted frameworks and MLIR/LLVM compilation can abstract device execution while supporting compiled workloads across local and remote hardware.Efficient DMA and RDMA data movement is also relevant when workloads span devices and nodes.Source: §I · p. 2
- Backline addresses the gap between high-level algorithm design and low-latency execution across device types with a heterogeneous compilation and runtime layer.The broader R&D need is to progressively compile Python-prototyped workloads onto local and distributed heterogeneous devices.Source: §I · pp. 1–2
II. FTQC PLATFORMS AND QEC
- FTQC latency tiers distinguish hardware capability from modality requirements, ranging from real-time control to distributed workloads.The required tier depends on QEC cycle rate; tighter modalities are around 1 µs, while looser ones are closer to 1 ms.Source: §II · p. 3
- Hard-deadline quantum workloads cannot benefit from latency hiding when a late result can be fatal, and probabilistic decoders may stall while awaiting convergence.This makes timely error-correction subroutines essential to device operation.Source: §II · p. 3
- An auxiliary CPU, GPU, or FPGA decoder in Tier 2 could handle rare complex tail-errors that a fixed FPGA decoder cannot correct in time.The proposed role is to prevent decoding backlog from building to unresolvable amounts and allow execution to continue.Source: §II · p. 3
III. PROGRAMMING AND COMPILING QUANTUM PROGRAMS
The paper describes a Python-first compilation path and a low-latency runtime approach for heterogeneous workloads. Backline connects device placement, compilation, and runtime execution, with implicit data movement intended to ease local-to-HPC workflow transitions.
III. PROGRAMMING AND COMPILING QUANTUM PROGRAMS
- Catalyst lowers PennyLane programs from Python through MLIR and LLVM to device-native operations, while a dual-mode frontend combines DSL tracing with AST rewriting for structured control flow.The frontend treats Python as the expression of user intent rather than the interpreted execution environment.Source: §III · p. 4
IV. LOW-LATENCY COMPUTING WITH HPC-GRADE HARDWARE
- Low-latency heterogeneous execution uses direct communication paths and RDMA/RoCE to reduce system mediation and unnecessary data movement.The design focuses on Tier 2 programmability, with Tier 3 and Tier 4 also supported.Source: §IV · pp. 4–5
A. Backline
- Backline combines a declarative PennyLane placement interface, Catalyst compilation and coordination, and a runtime for target-specific execution and data movement.The modular setup supports adding runtime devices through a unified, portable user interface.Source: A. Backline · p. 5
B. User interface for workload expression
- Users can prototype heterogeneous workloads on a laptop or workstation and move them to RDMA-capable HPC hardware with minimal program changes.The interface is exposed through PennyLane in Python.Source: B. User interface for workload… · p. 5
1) Data placement abstraction:
- A placement declares a controller, coprocessors, and transport; transfers follow from node declarations rather than requiring users to write them explicitly.The placement acts as the workload topology’s control plane, with interactions managed through an implicit memory model.Source: 1) Data placement abstraction · pp. 5–6
2) Coprocessor functions:
Backline lets users supply coprocessor decoders as precompiled symbols, Python-written Triton kernels, or generated BP-OSD decoders from CSS-code Tanner graphs. These alternatives can be selected through a common placement-level interface, and coprocessors can dispatch among registered functions using a decoder_id.
2) Coprocessor functions:
- Precompiled CPU or GPU functions give direct control over a decoder’s inner loop and memory-access patterns.The function can be written and compiled in a library, then referenced by symbol.Source: 2) Coprocessor functions · p. 6
- A payload’s decoder_id can select among multiple registered coprocessor functions, such as separate decoders for X- and Z-type errors.The identifier occupies a section of the message payload.Source: 2) Coprocessor functions · p. 6
- Precompiled symbols, Python Triton kernels, and Tanner-graph-generated BP-OSD decoders all produce interchangeable CoprocessorFunction objects.The helper compiles Triton functions or Tanner-graph decoders ahead of time as GPU kernels.Source: 2) Coprocessor functions · p. 6
3) Transport mode and local prototyping:
Backline supports local prototyping and remote execution through configurable placements and transports, while exposing runtime calls for operations that need finer control. Its compiler infrastructure separates code movement from data exchange and lowers each through dedicated dialects.
3) Transport mode and local prototyping:
- Choosing memcpy exercises Backline’s placement and request/reply path locally; changing the transport to rdma replaces local copies with one-sided network writes without changing the program.The local path requires only a Linux machine, with an optional GPU.Source: 3) Transport mode and local prototyping · p. 6
4) Granular control and runtime call:
- qp.runtime_call embeds a call to a precompiled, linked dynamic library at the expression point for local or remote execution.It can transfer messages or trigger a handshake, initialization, or device setup, with data available at that instruction.Source: 4) Granular control and runtime call · pp. 6–7
C. Compiler infrastructure
- A Placement assigns program components to devices, while the executor dialect moves code and the transport dialect specifies runtime data exchanges.Target-specific compilation options are configured through PennyLane’s qjit interface.Source: C. Compiler infrastructure · p. 7
- The executor dialect represents transfer and invocation of separately compiled components, including opening sessions, sending binaries, launching entry points, and closing sessions.A pre-existing target library can instead be called directly by symbol.Source: 1) The executor dialect · p. 7
- The transport dialect models a connection-oriented request/reply session, from connection setup and data-path configuration through message rounds and teardown.Its operations cover both endpoint roles and the round-level request and reply.Source: 2) The transport dialect · p. 7
3) Compilation flow and the QEC pipeline:
Backline’s compilation flow inserts transport setup and replaces decoded QEC operations with request/reply rounds, then compiles and dispatches components to their selected targets. The runtime connects the compiled program to hardware through a shared C ABI.
3) Compilation flow and the QEC pipeline:
- inject-transport-session adds session setup and teardown, while lower-decode-to-transport replaces each bufferized decode with a session lookup and a stage_payload, post, and collect round.Where available, the reply is received through the transport-owned reply_slot buffer.Source: 3) Compilation flow and the QEC pipeline · p. 8
- The compiler cross-compiles each target component and replaces calls assigned to executors with open, send_binary, and launch operations before lowering dialects to runtime C-interface calls.When implicit QEC is enabled, remaining physical QEC operations are lowered afterward.Source: 3) Compilation flow and the QEC pipeline · p. 8
- With implicit QEC enabled, the Steane path lowers logical circuits through QEC dialects into encoded physical circuits with error-correction rounds; explicit qp.backline.decode supports other codes and circuit-specific round placement.The implicit path currently supports qec_code="steane"; the qLDPC demonstrations use the explicit path.Source: 3) Compilation flow and the QEC pipeline · pp. 7–8
D. The runtime layer
- A shared library interface with a well-defined C ABI decouples compiled programs from runtime implementations, allowing custom runtimes for bespoke hardware.Source: D. The runtime layer · p. 8
1) Session life cycle and memory:
Backline’s transport sessions manage connection setup, data movement, and teardown, with memory provisioned by each target. The system composes transport, controller, coprocessor, locality, and decoder choices so workloads can move from local prototypes to heterogeneous remote configurations.
1) Session life cycle and memory:
- An RDMA session proceeds through connection, metadata exchange, channel setup, role-specific configuration, start, and eventual stop and destroy.The target manages memory regions, which may use host RAM, GPU HBM, or FPGA DDR or BRAM, organized as ring buffers with fixed slot counts.Source: 1) Session life cycle and memory · p. 8
2) Network transport and fabric selection:
- Direct libibverbs targeting is chosen to avoid potential overhead from higher-level networking abstractions, while latency-hiding communication overlap is of limited use for workloads that require fast feedback.The authors identify tooling for other networking abstractions, including UCX, OFI, and ef_vi, as follow-up work.Source: 2) Network transport and fabric… · pp. 8–9
3) Composing a system:
- Transport, controller, coprocessor, locality, and coprocessor function are independent placement choices, so moving from a laptop prototype to HPC hardware need not change the workload.The demonstrations include local and remote endpoints, multiple architectures, and an FPGA controller.Source: 3) Composing a system · p. 9
V. DEMONSTRATIONS AND EVALUATION
- The demonstrations span configurations from a laptop without specialized hardware to an FPGA controller driving a GPU decoder across a fabric.The configurations require progressively more setup, from a single Linux CPU machine to a system with RDMA networking, a GPU, and an FPGA board.Source: §V · p. 10, A. Hardware setup · pp. 10–11
A. Hardware setup
- The FPGA’s hardware-handshake engine performs per-round work-queue posting, doorbell ringing, reply detection, and latency measurement in programmable logic rather than through repeated host-software operations.Session setup remains a software control-plane operation.Source: A. Hardware setup · p. 11
B. Demonstrations
The demonstrations vary placement, transport, how error correction is expressed, and the source of the decoder. They cover local and remote CPU, GPU, and FPGA configurations, from high-level placement declarations to expanded runtime calls.
B. Demonstrations
- Four demonstration choices vary where controller and coprocessor run, their processor types, the transport, how the round is expressed, and the decoder source.The demonstrations include local and remote configurations.Source: B. Demonstrations · p. 11, 1) Placement · p. 11
1) Placement:
- The showcased controller may be a local CPU, remote CPU, or remote FPGA, while the coprocessor may be a local CPU or remote GPU.Listing 4 presents four placements, including examples that change transport, machine and device class, or controller.Source: 1) Placement · pp. 11–12
- A node configuration sets whether code is shipped remotely, network access options, an RDMA endpoint, target setup arguments, and the hardware library resolved with the transport.The placement carries the two nodes and the transport between them.Source: 1) Placement · p. 12
- Demo 1 uses local memcpy without a verbs device; Demo 1a keeps the same placement but uses a verbs path over Soft-RoCE.With a real RDMA NIC, the same placement can run over loopback or between machines.Source: 1) Placement · p. 12
- The same round can be shown at three levels: a decode call, which expands into four runtime calls, and those calls written out individually.In the explicit version, the user extracts the syndrome, selects a decoder with decoder_id, and applies the correction.Source: 1) Placement · p. 12
2) Implicit and explicit decoding:
Backline supports compiler-inserted error correction, user-written decoding, and direct runtime calls. These routes differ in how much of the round the compiler handles.
2) Implicit and explicit decoding:
- With qec_code="steane", the user writes only a logical circuit; compile-time QEC passes insert encoding and a correction round around each logical gate.The inserted round extracts stabilizers, decodes the syndrome on the coprocessor, and applies the correction.Source: 2) Implicit and explicit decoding · p. 12
- Users can instead encode the circuit and explicitly measure syndromes, decode them, and apply corrections; qp.backline.decode sends the syndrome and blocks for the reply.decoder_id selects among registered decoders.Source: 2) Implicit and explicit decoding · p. 12
- The decode call expands to get_session, stage_payload, post, and collect runtime calls; the three routes differ in how much is left to the compiler.Demo 2a reproduces demo 2’s result with those operations written out directly.Source: 2) Implicit and explicit decoding · p. 12
3) The decoder function:
The demonstrations exercise multiple decoder implementations, including pre-compiled, Triton, and belief-propagation decoders. Registered decoders can be selected per round using decoder_id.
3) The decoder function:
- The demonstrations use a pre-compiled Steane-code decoder on both CPU and GPU coprocessors, a Steane decoder written as a Triton kernel, and a BP-OSD decoder for a [[13, 1, 3]] hypergraph-product code.The BP-OSD decoder is generated from parity-check matrices and run for 10 iterations.Source: 3) The decoder function · pp. 12–13
- Several decoders can be registered on one coprocessor and selected per round with decoder_id, allowing separate decoders to serve the X and Z stabilizer types.Table II lists demonstrations by decode route and decoder source, including implicit, explicit, and runtime routes.Source: 3) The decoder function · p. 13, Table II · p. 13
C. Measurements
The baseline measures FPGA-initiated CPU and GPU round trips, with the CPU path showing lower median latency and a tighter spread. Both paths fall within the synchronous co-processing tier.
1) Baseline:
- The FPGA handshake engine posts a 16-byte one-sided RDMA write and detects the reply without host involvement in the round trip.The target polls the expected buffer; the CPU echoes directly, while the GPU path signals the CPU to write the reply.Source: 1) Baseline · p. 13
- 4.64 µs for the first CPU round trip and 9.27 µs for the first GPU round trip; both warm-up samples were excluded from the steady-state plots.The first sample is outside the subsequent steady-state distributions.Source: 1) Baseline · p. 13, Figure 6 · p. 13
- Both baseline paths fall within the synchronous co-processing tier, and the measurements suggest the framework is suitable for that tier with room to spare.The CPU path is stronger on spread, which the passage says matters for a budget that must be met on every round.Source: 1) Baseline · p. 13
- 2.305 µs CPU median versus 4.5 µs GPU median; the CPU path was 2.2 µs faster in the median and over seven times tighter in spread.The CPU standard deviation was 21.4 ns and the GPU standard deviation was 0.159 µs; Fig. 6 shows the steady-state round-trip distributions.Source: Figure 6 · p. 13, 1) Baseline · p. 13
2) Decoder runtime:
Backline supports decoder experimentation across heterogeneous targets, with measured FPGA-to-coprocessor round trips in the synchronous co-processing tier. Its placement-based abstractions support moving workloads across fabrics and hardware configurations, while variable payload sizes remain unsupported.
2) Decoder runtime:
- Simple decoders add virtually no time relative to communication, while complex decoders can exceed transport overhead; optimized CPU functions or GPU kernels are the performance path.Python through Triton offers the least resistance for prototyping, and Backline supports moving from that mode to tuned implementations.Source: 2) Decoder runtime · pp. 13–14
VI. CONCLUSIONS AND OUTLOOK
- The same GHZ program ran across local memcpy, local RDMA, remote CPU/GPU, and FPGA-to-GPU configurations, with configuration changes confined to placement.Backline adds heterogeneous compilation and runtime layers to PennyLane and Catalyst, including a unified ABI for target devices and transport modes.Source: §VI · p. 14
- 2.305 µs to a CPU and 4.5 µs to a GPU were the median steady-state FPGA-controller round-trip latencies, both within the synchronous co-processing tier.Measurements used the FPGA hardware-handshake engine over RoCE v2.Source: §VI · p. 14
A. Choosing a coprocessor
- The CPU path was faster and more predictable than the GPU path, while the preferred coprocessor depends on whether decoding or transport dominates.A GPU may matter for larger, batched, or neural-network decoders; when decoding is cheap relative to the round trip, its transport penalty can outweigh unneeded parallelism.Source: A. Choosing a coprocessor · p. 14
B. Fabric is a moving target
- Backline exposes new fabrics through the same runtime interface, but its current transport requires fixed message sizes and cannot support variable payloads on the pre-registered fast path.The fixed-size design suits compile-time-known syndrome widths; variable-length corrections and runtime-sized batches remain future work.Source: B. Fabric is a moving target · p. 14
C. Scaling up
- Backline’s placement model can target rack-scale systems as multiple coprocessors behind an RDMA-capable NIC.The paper presents this as likely compatibility and a potential basis for multi-device targeting and full-system emulation.Source: C. Scaling up · p. 16
D. Scaling down
- The same workload can move from laptop or software-RDMA development to commodity-RNIC or FPGA hardware by changing placement rather than rewriting the workload.Backline is open-source, Python-native, and extensible across CPU, GPU, and FPGA vendors.Source: D. Scaling down · p. 16
E. Down to silicon
- Backline currently supports application and compilation layers, orchestration, runtime, and real-time QEC workloads, while qubit control and data acquisition remain beyond its current reach.The paper locates bounded submicrosecond control, data acquisition, and primary decoding on FPGAs and ultimately ASICs.Source: E. Down to silicon · p. 16
APPENDIX A TEST ENVIRONMENT AND FPGA EXPERIMENTS
The experiments use a NUMA-aware CPU/GPU server and a VPK120 FPGA controller. The FPGA-side ERNIC handles RDMA transport, while a hardware-handshake state machine manages rounds and profiling.
1) CPU/GPU server:
- The benchmark server places the GPU and RNIC in one NUMA domain and pins the executor allocations and real-time polling thread; CPU and GPU responders measure transport floors or run decoder implementations.The GPU echo responder polls its memory and returns messages through host memory; the GPU QLDPC responder runs a Triton-generated BP-OSD decoder.Source: 1) CPU/GPU server · p. 19
2) FPGA controller:
- Both the FPGA’s ARM processing system and programmable logic participate in session setup, with responsibilities separated in the controller design.The controller is the AMD VPK120 described for the demonstrations and benchmarks.Source: 2) FPGA controller · p. 19, Table V · p. 19
1) ERNIC software and hardware split:
- The ERNIC implements FPGA-side RoCE v2 transport; hardware performs zero-copy data movement and packet handling, while the HWHS orchestrator handles timing and dispatch.The processing system allocates memory, establishes queue-pair contexts, and handles exceptions.Source: 1) ERNIC software and hardware split · p. 19
2) HWHS state machine:
- The HWHS state machine pre-posts work descriptors, validates replies by sequence number, and uses a timeout path; its JOIN state synchronizes reply handling with preparation of the next descriptor.The design uses 16% of LUTs and 9% of flip-flops; its 128 URAM-block trace space stores 1,048,576 entries for cycle-accurate RTT profiling.Source: 2) HWHS state machine · pp. 19–20
3) Controller modes and cadences:
The hardware-loop benchmarks vary how the FPGA controller issues rounds and how frequently it does so, then measure steady-state latency across CPU and GPU responders. Random-delayed cadence shifts CPU and GPU latency distributions in different directions while widening their extreme tails.
3) Controller modes and cadences:
- hw-handshake moves the handshake into PL, while hw-loop also schedules rounds in PL and includes payload movement between engines.The hw-loop pacer uses a fixed-seed Fibonacci-LFSR interval generator; b2b issues the next syndrome as soon as correction returns, while random-delayed inserts a quiet period excluded from reported RTT.Source: 3) Controller modes and cadences · p. 20, Table VIII · p. 21, Table IX · p. 21
- The interval generator's 10,000-interval replay averaged 677.847 ms, against a uniform expectation of 676.089 ms, with 20 bins flat within sampling error.The replay used a 200 MHz clock, freq = 1,000,000, span = 0x0FFFFFFF, and seed 1.Source: 3) Controller modes and cadences · p. 20
C. FPGA experiment results
- Round 0 takes 4.640 µs for CPU echo and 9.270 µs for GPU echo, versus steady-state medians of 2.305 µs and 4.500 µs.Round 0 includes one-time endpoint initialization; subsequent reported experiments omit it and show the N −1 steady-state rounds.Source: C. FPGA experiment results · p. 20, Figure 9 · p. 21
1) Hardware-loop experiments:
- CPU Steane adds almost nothing over CPU echo, GPU Steane is 0.005 µs faster than GPU echo, and the qLDPC kernel dominates its own end-to-end RTT.CPU and GPU echo isolate transport, Steane adds a pre-compiled decoder, and qLDPC uses a generated GPU decoder.Source: 1) Hardware-loop experiments · p. 21
- In b2b mode, the five responders' median RTTs were 2.470, 2.465, 4.575, 4.570, and 32.635 µs, respectively.The responders comprise CPU and GPU echo, CPU and GPU Steane, and qLDPC; the hardware-loop table excludes Round 0.Source: Table X · p. 21, 1) Hardware-loop experiments · p. 21
2) Cadence effect on latency:
- CPU median RTT rises 20 ns for both echo and Steane under random-delayed cadence; P99.9 rises 130 ns for echo and 160 ns for Steane.The identical median shift places the cost on the long-gap path rather than on decoding.Source: 2) Cadence effect on latency · p. 22
- GPU median RTT changes by -90 ns for echo, 0 ns for Steane, and -65 ns for qLDPC; P99.9 rises by 65 ns, 45 ns, and 35 ns, respectively.These distribution shifts remain small relative to round trips of several microseconds.Source: 2) Cadence effect on latency · p. 22