PaperlayerView exact revision on arXiv

Source-linked AI summary

cuQuantum SDK: A High-Performance Library for Accelerating Quantum Science

Harun Bayraktar, Ali Charara, David Clark, Saul Cohen, Timothy Costa, Yao-Lung L. Fang, Yang Gao, Jack Guan, John Gunnels, Azzam Haidar, Andreas Hehn, Markus Hohnerbach, Matthew Jones, Tom Lubowe, Dmitry Lyakh, Shinya Morino, Paul Springer, Sam Stanwyck, Igor Terentyev, Satya Varadhan, Jonathan Wong, Takuma Yamaguchi

arXiv:2308.01999v1quant-phcs.PFcs.SE

TL;DR

Quantum circuit simulation becomes increasingly difficult as quantum devices grow, motivating fast and scalable classical simulators. The cuQuantum SDK supplies composable GPU-optimized primitives for state-vector and tensor-network simulation, with distributed execution and Python and C interfaces. Adopted simulators demonstrate substantial GPU acceleration and expanded simulation scale across cloud and HPC platforms.

  • Problem

    Quantum circuit simulation is increasingly difficult as quantum devices grow, while simulators remain important for algorithm development and quantum-hardware validation.

  • Method

    The cuQuantum SDK provides composable GPU-optimized primitives for state-vector, tensor-network, and approximate tensor-network simulators, including distributed execution and Python APIs.

  • Results

    cuQuantum-adopting simulators demonstrate significant GPU acceleration and distributed scaling for state-vector and tensor-network simulations, including a 93% accurate 5000-qubit MaxCut solution.

  • Takeaways & Limitations

    The SDK enables simulator developers to study larger quantum simulations on NVIDIA GPUs, cloud systems, and HPC platforms.

Abstract

from arXiv · show

We present the NVIDIA cuQuantum SDK, a state-of-the-art library of composable primitives for GPU-accelerated quantum circuit simulations. As the size of quantum devices continues to increase, making their classical simulation progressively more difficult, the availability of fast and scalable quantum circuit simulators becomes vital for quantum algorithm developers, as well as quantum hardware engineers focused on the validation and optimization of quantum devices. The cuQuantum SDK was created to accelerate and scale up quantum circuit simulators developed by the quantum information science community by enabling them to utilize efficient scalable software building blocks optimized for NVIDIA GPU platforms. The functional building blocks provided cover the needs of both state vector- and tensor network- based simulators, including approximate tensor network simulation methods based on matrix product state, projected entangled pair state, and other factorized tensor representations. By leveraging the enormous computing power of the latest NVIDIA GPU architectures, quantum circuit simulators that have adopted the cuQuantum SDK demonstrate significant acceleration, compared to CPU-only execution, for both the state vector and tensor network simulation methods. Furthermore, by utilizing the parallel primitives available in the cuQuantum SDK, one can easily transition to distributed GPU-accelerated platforms, including those furnished by cloud service providers and high-performance computing systems deployed by supercomputing centers, extending the scale of possible quantum circuit simulations. The rich capabilities provided by the SDK are conveniently made available via both Python and C application programming interfaces, where the former is directly targeting a broad Python quantum community and the latter allows tight integration with simulators written in any programming language.

I. INTRODUCTION

The cuQuantum SDK provides GPU-optimized building blocks to accelerate and scale quantum circuit simulators based on state-vector and tensor-network methods. Its components support distributed execution and broad adoption across quantum science workflows.

  • Quantum circuit simulators support algorithm development and hardware validation when quantum computers remain small, error-prone, capacity-constrained, and expensive.
  • cuQuantum combines cuStateVec and cuTensorNet to improve GPU kernels, performance, and access to advanced simulation techniques.
  • The SDK raises abstraction above basic linear and tensor algebra libraries through flexible building blocks targeted at quantum circuit simulator developers.
  • cuQuantum also supports generic quantum-science libraries beyond its primary target of quantum circuit simulators.
  • 93% accurate solution to a 10,000-vertex MaxCut graph using 5000 qubits demonstrated large tensor-network simulation capability.
  • The SDK encapsulates state-vector, tensor-network, and approximate tensor-network methods for simulations spanning wide qubit counts and circuit depths.
  • cuQuantum has accelerated quantum-information-science research across industry and academia, including optimization, quantum chemistry, security, and privacy applications.

A. cuStateVec API Design

cuStateVec is designed as a flexible, memory-conscious primitive layer for state-vector simulators. Its APIs support core operations and distributed execution, while gate application and fusion target simulation performance.

  • A. cuStateVec API Design: cuStateVec provides GPU-accelerated APIs for gate application, measurement, expectation values, and sampling under a relaxed memory model.
  • A. cuStateVec API Design: In-place operations and small internal temporary buffers reduce additional memory requirements for state-vector simulations.
  • A. cuStateVec API Design: Multi-GPU and multi-node APIs support state-vector index-bit swapping and qubit reordering for distributed simulations.
  • B. Gate Application Performance and Gate Fusion: Gate application is typically the most time-consuming state-vector operation.
  • B. Gate Application Performance and Gate Fusion: 3.0 TB/s peak memory bandwidth was measured for gate application on an NVIDIA H100, corresponding to 90% of peak bandwidth.
  • B. Gate Application Performance and Gate Fusion: Gate fusion combines multiple small gate matrices into one multi-qubit matrix, reducing repeated applications and potentially accelerating simulation.

C. Distributed State Vector Simulation

Distributed state-vector simulation slices the state vector across devices and reorders qubits so gates can operate locally, supporting multi-GPU and multi-node simulators.

  • Distributed state-vector simulation equally slices the state vector across multiple computing devices, separating local index bits from global index bits.
  • Gates on local index bits execute independently on each slice, whereas gates on global index bits require access to multiple slices.
  • Qubit reordering swaps global and local index-bit positions so remapped target qubits remain local to state-vector slices.
  • The distributed approach underlies multi-GPU and multi-node backends, including qsim integration with Cirq.

1) Multi-GPU qsim backend:

The qsim-mgpu backend combines qubit reordering and cuStateVec gate primitives to simulate circuits across GPUs, with substantial measured speedups on DGX systems.

  • qsim-mgpu avoids device-to-device communication during each gate application by reordering qubits and using multi-device index-bit swaps when needed.
  • Dense and diagonal gate fusion uses separate limits and corresponding cuStateVec APIs for fused gate application.
  • The evaluation targets QFT, QAOA with p = 2, and Quantum Volume circuits on DGX A100 and DGX H100 nodes.
  • 297-fold speedup over the CPU backend was achieved for the QFT simulation with 8 GPUs, compared with 78.6 s on CPUs.

2) Qiskit/Qiskit Aer Multi-node Simulator:

The cusvaer extension enables Qiskit Aer multi-node state-vector simulation on NVIDIA GPU clusters, where scaling is shaped by communication bandwidth during qubit reordering.

  • cusvaer extends Qiskit Aer with a multi-node simulator optimized for NVIDIA GPU-based HPC clusters.
  • Distributed state-vector performance is limited by data transfers among state-vector slices required for qubit reordering.
  • The Selene evaluation varies Quantum Volume and quantum phase estimation simulations from 32 qubits on 1 GPU and 1 node to 40 qubits on 256 GPUs and 32 nodes.
  • Simulation-time growth is steeper across nodes than within a node, reflecting lower inter-node IB bandwidth than NVLink/NVSwitch bandwidth.
  • The tensor-network module provides GPU-accelerated contraction and exact or approximate tensor decomposition through cuTensorNet.

A. Tensor Network Contraction Module

cuTensorNet contracts tensor networks using optimized paths, slicing, and GPU tensor operations to fit memory limits and expose distributed parallelism, while slicing can increase FLOPs.

  • 1) Path Finding: Tensor-network contraction uses a sequence of pairwise contractions whose path strongly determines computational and memory cost.
  • 1) Path Finding: cuTensorNet simplifies networks, then applies bubbling and hyper-optimization to search quickly for high-quality contraction paths.
  • 2) Slicing: Slicing splits a contraction into independent smaller contractions so the network fits device memory or gains distributed parallelism.
  • 2) Slicing: Sliced contractions can be parallelized across distributed settings and can accelerate tensor-network contractions even when memory is sufficient.
  • 2) Slicing: Slicing often increases total FLOP count, with overhead depending heavily on the contraction path and selected modes.
  • cuTensorNet uses NVIDIA cuTENSOR as the GPU backend for pairwise tensor contractions after a contraction path is generated or supplied.

3) Planning and Workspace:

cuTensorNet combines contraction-path planning, slicing, and GPU-optimized contraction to reduce workspace demands and accelerate tensor-network simulations. Its pathfinder finds high-quality paths quickly, while network contraction achieves substantial speedups over CPU and GPU baselines.

  • Planning and Workspace: For Sycamore-53 circuits with over 3,000 tensors, cuTensorNet averages about 8 seconds per path versus 730 seconds for Cotengra.The comparison concerns 1000 hyper-optimizer samples across circuits with different depths.
  • Planning and Workspace: cuTensorNet contracts networks on GPUs through cuTENSOR kernels and internal optimizations that reduce memory use and improve kernel performance.These optimizations include reordering intermediate tensor modes and reducing the memory footprint of pairwise contractions.

B. Distributed Multi-GPU Multi-Node Execution

cuTensorNet extends tensor-network simulation from single GPUs to distributed multi-GPU, multi-node platforms. Its parallel pathfinding and slice execution are designed to scale across many GPUs, including cloud and HPC systems.

  • Distributed Multi-GPU Multi-Node Execution: Automatic multi-GPU and multi-node parallelization enables larger quantum circuit simulations.The distributed capability was introduced in the 22.11 release.
  • Distributed Multi-GPU Multi-Node Execution: Existing single-GPU cuTensorNet applications can transition to cloud and HPC GPU platforms through an MPI library and a single API call.The implementation supports standard-conforming CUDA-enabled MPI implementations through a runtime-loaded interface library.
  • Distributed Multi-GPU Multi-Node Execution: Distributed pathfinding assigns hyper-sampling of contraction candidates across GPUs, while execution distributes generated tensor-network slices.Both procedures are described as embarrassingly parallel, supporting strong scaling to many GPUs.
  • Distributed Multi-GPU Multi-Node Execution: Figure 9 evaluates strong scaling for a Sycamore-53 random quantum circuit at depth 14 by computing a single bit-string probability amplitude.The circuit was part of validation experiments on Google’s Sycamore quantum chip.

C. Intermediate Tensor Caching

cuTensorNet accelerates repeated tensor-network contractions by caching constant intermediate tensors when only a small subset of inputs changes. This reuse can substantially improve performance across repeated evaluations.

  • Intermediate Tensor Caching: Repeated tensor-network evaluations may reuse the same network structure when only a small subset of input tensors changes.This pattern occurs when validating quantum processors by repeatedly computing individual bit-string probability amplitudes.
  • Intermediate Tensor Caching: cuTensorNet separates workspace into scratch memory for temporary computation data and cache memory for constant intermediate tensors.Cached intermediates are stored after the first contraction for reuse in subsequent calls.
  • Intermediate Tensor Caching: Intermediate tensor reuse can produce drastic speedups when mutable inputs change across many contractions.Table IV measures the impact on a synthetic network over 1000 repetitions while varying the number of constant input tensors.

D. Approximate Tensor Network Features

cuTensorNet supports approximate tensor-network methods through GPU-accelerated decompositions and compound operations. Its APIs cover QR, SVD, and gate splitting, with benchmarks showing substantial GPU speedups over CPU implementations at large bond dimensions.

  • Approximate Tensor Network Features: Approximate tensor-network methods use QR and SVD to represent quantum states efficiently and control MPS bond dimension and accuracy.QR supports MPS canonicalization, while SVD truncates singular values to limit bond dimension.
  • Approximate Tensor Network Features: cuTensorNet provides hierarchical APIs for tensor-level QR and SVD operations and higher-level approximate tensor-network algorithms.The single-GPU C APIs leverage cuSOLVER kernels for transposition and decomposition.
  • Approximate Tensor Network Features: GateSplit factorizes a gate operand onto two connected tensors, using either direct contraction followed by decomposition or reduced QR-based processing.The operation is exposed through the specialized cutensornetGateSplit C API.
  • Approximate Tensor Network Features: At bond dimension D = 4096, tensor QR reaches 102x speedup on A100 and 230x on H100 over a 64-core CPU implementation.The benchmarks use rank-3 MPS tensors with shape (D, 2, D).
  • Approximate Tensor Network Features: At D = 4096, tensor SVD reaches 6.4x speedup on A100 and 8.8x on H100, while GateSplit reaches 8.6x and 13.8x, respectively.These results compare NVIDIA GPU APIs with an equivalent NumPy implementation using all 64 AMD EPYC 7742 cores.

IV. CUQUANTUM PYTHON

cuQuantum Python provides both direct bindings and higher-level Pythonic interfaces for accessing cuStateVec and cuTensorNet functionality. Its layered design supports convenient integration, customization, quantum-circuit workflows, and distributed tensor-network execution.

  • API design: cuQuantum Python exposes 1:1 bindings for cuStateVec and cuTensorNet C APIs alongside high-level Pythonic APIs.The two-layer approach targets both direct access and easier integration with Python applications.
  • Performance evaluation: The SDK benchmarks tensor QR, SVD, and GateSplit against NumPy using execution-time comparisons with cuTensorNet APIs.The figure identifies NumPy with a dashed black line and cuTensorNet APIs with solid green lines.
  • API design: High-level APIs encapsulate low-level boilerplate, including NumPy-compatible tensor-network contraction, customizable contraction, and QR or SVD decomposition.The interfaces include einsum(), contract(), tensor.decompose(), and experimental contract_decompose() functionality.
  • Resource control: Underlying Python classes provide finer control over tensor-network resource management, path finding, kernel autotuning, and contraction execution.The Network class encapsulates a tensor network while exposing methods for these operations.
  • Distributed execution: cuTensorNet distributed contraction can be enabled through an mpi4py communicator, supporting users across different MPI implementations.The mpi4py abstraction passes an MPI communicator to the distributed configuration helper.
  • Quantum-circuit workflows: CircuitToEinsum converts parameterized Cirq or quantum circuits into contraction inputs for cuQuantum APIs, with optional reverse lightcone simplification for RDMs and expectation values.Users can choose default or customized path optimization and optionally reduce the effective tensor-network size.

V. CONCLUSIONS

The conclusions report GPU-optimized primitives for state-vector and tensor-network simulators, with distributed execution enabling larger simulations across multi-node systems. cuQuantum Python extends these capabilities to Python workloads, while future work broadens support for quantum applications.

  • State-vector simulation: cuStateVec provides memory-efficient state-vector primitives, high-memory-bandwidth gate application, GPU speed-ups over CPU-only execution, and scalable multi-node multi-GPU execution.These capabilities support larger state-vector simulations.
  • Tensor-network simulation: cuTensorNet generates high-quality contraction paths faster than state-of-the-art software and provides GPU acceleration, caching, distributed parallelization, and approximate-contraction primitives.Adopting simulators often shows close-to-ideal scalability on cloud and HPC systems.
  • Python access: cuQuantum Python APIs provide access to the SDK’s functionality from Python frameworks and projects for accelerating Python workloads on NVIDIA GPUs.The APIs are presented as a natural extension for Python users.
  • Future direction: Future work will extend cuQuantum’s functionality and features to cover a broader range of quantum applications.The stated aim is to support ongoing and future quantum research efforts.
Loading 2308.01999v1…