PaperlayerView exact revision on arXiv

Source-linked AI summary

The Shape of Speed: Impacts of Partition Geometry and Rank Density in Distributed Quantum Circuit Simulations

Yikai Mao, Yuan He, Shaowen Li, Masaaki Kondo

arXiv:2609.34369v1cs.ETcs.DCcs.PF

TL;DR

This paper asks how partition geometry and rank density affect communication-bound distributed quantum circuit simulations, where network performance constrains execution. It measures these factors in a controlled Fugaku evaluation and finds geometry dominates runtime across algorithms, while rank density helps only on compact partitions.

  • Problem

    At production scale, distributed quantum circuit simulations are communication-bound, but the effect of scheduler-assigned partition geometry on execution is not routinely controlled.

    Source: §I · p. 2, A. The Placement Lottery of Default… · p. 4

  • Method

    The authors evaluate twelve algorithms across thirteen torus geometries and six rank densities in 764 validated configurations for 39-qubit simulations on 1,024 Fugaku nodes.

    Source: §I · p. 2

  • Results

    Flat partitions run 1.73–2.31× slower than near-cubic partitions across all twelve algorithms despite identical injected traffic, while 16 ranks per node shortens runtime by 11% only on compact partitions.

    Source: §I · p. 2, B. Geometry Dominates and Constrains… · p. 4

  • Takeaways & Limitations

    Requesting near-cubic partitions with 16 ranks per node roughly halves time-to-solution versus flat partitions, which consume 1.82× more energy.

    Source: §I · p. 3

    Configurations were not repeated across allocations, so run-to-run variability across allocations is unquantified.

    Source: A. Limitations · p. 9

Abstract

from arXiv · show

In distributed quantum circuit simulation, a poorly shaped partition can halve performance before computation begins. Evaluation on Fugaku across 764 validated configurations (twelve algorithms, thirteen torus partition geometries, and six rank densities for 39-qubit simulations on 1,024 nodes) shows that partition geometry dominates runtime. All twelve algorithms run 1.73-2.31x slower on flat partitions than on near-cubic ones despite identical data transfer, proving the slowdown stems from network delivery rather than communication volume. This penalty scales with the 3D torus partition aspect ratio (runtime $\propto a^{0.39}$, $r = 0.72$). Rank density is secondary, cutting runtime by 11% at 16 ranks per node only on compact geometries. Ultimately, requesting a near-cubic partition with 16 ranks per node roughly halves time-to-solution relative to flat partitions, which also consume 1.82x more energy. A simulator-free all-to-all microbenchmark confirms a similar geometry penalty for collective-dominated workloads.

I. INTRODUCTION

This paper examines how scheduler-assigned partition geometry and user-selected rank density affect communication-bound quantum circuit simulations. Across controlled Fugaku measurements, geometry is the primary performance factor, and the study proposes a predictor and placement practice for improving time-to-solution.

Motivation

  • Default placement can silently cost network-bound workloads about half their performance because schedulers optimize utilization without application communication information.Eleven identical 1,024-node requests received five distinct partition geometries, eight of them highly elongated.Source: §I · p. 2

Evaluation

  • The controlled evaluation covers twelve algorithms, thirteen torus geometries, and six rank densities across 764 validated configurations for 39-qubit simulations on 1,024 nodes.Explicit shape requests were constrained by scheduler allocation sizes, so the experiments ran on power-of-two subsets of shaped allocations.Source: §I · p. 2

Empirical findings

  • Flat partitions run 1.73–2.31× slower than near-cubic partitions across all twelve algorithms, while rank density shortens runtime by 11% only on compact geometries.Injected network traffic is invariant across geometries, and a simulator-free all-to-all microbenchmark shows a similar geometry penalty for other communication-bound workloads.Source: §I · pp. 2–3

Contributions

  • The paper presents its work as the first controlled characterization of partition-geometry effects on distributed quantum circuit simulation on a production supercomputer.A partial-use allocation method enables controlled experiments on the production scheduler.Source: §I · pp. 2–3
  • A single-parameter aspect-ratio model and near-cubic placement with 16 ranks per node roughly halve time-to-solution versus flat partitions, which consume 1.82× more energy.The model reports runtime ∝ α^0.39, r = 0.72.Source: §I · p. 3

II. BACKGROUND AND RELATED WORK

Distributed state-vector simulation becomes communication-intensive when a state vector exceeds one node’s memory. Prior simulator work largely reduces application-generated data movement, leaving the efficiency of network delivery across different partition geometries open.

A. Distributed State-Vector Simulation and Its Communication

  • When the state vector exceeds a node’s memory, global-qubit gates can require ranks on different nodes to exchange state-vector data over the interconnect.Prior approaches reduce communication or memory use through gate scheduling, cache blocking, fusion, qubit reordering, compression, and communication-avoiding partitioning.Source: A. Distributed State-Vector Simulation… · p. 3
  • Prior simulator work focuses on reducing traffic, leaving open how efficiently the network delivers it across contiguous torus partition geometries.On such systems, geometry is fixed before computation begins and strongly affects delivery efficiency.Source: A. Distributed State-Vector Simulation… · p. 3

B. Topology-Aware Placement in HPC Systems

Prior placement research addresses topology-aware allocation, but controlled application-level measurements of partition geometry on production systems remain scarce. Here, default allocations vary substantially in shape, motivating geometry-aware benchmarking and workload management.

B. Topology-Aware Placement in HPC Systems

  • Prior work studies topology-aware allocation and placement sensitivity, but controlled application-level geometry measurements on production systems remain scarce.The figure contrasts two 1,152-node allocation geometries; both carry identical traffic, while the flat partition has a longer diameter and narrower bisection.Source: B. Topology-Aware Placement in HPC… · p. 3, Figure 1 · p. 4

C. Benchmarking and Workload Management

  • Benchmark suites commonly report qubit, node, and rank scaling but rarely control or record allocation geometry, which can change runtime by about 2× and determine whether rank-density tuning helps.The aspect-ratio surrogate provides a submission-time cost model, and the recommended practice requires no scheduler changes.Source: C. Benchmarking and Workload Management · p. 4

A. The Placement Lottery of Default Allocation

  • Eight of eleven identical 1,024-node requests received flat partitions (α ≥16), which run this workload roughly twice as slowly as near-cubic geometries.The scheduler padded requests to 1,152-node partitions and assigned five geometries; the flattest shape, α = 24, appeared five times.Source: A. The Placement Lottery of Default… · p. 4

B. Geometry Dominates and Constrains Rank Density

Partition geometry is the primary performance determinant, while rank density matters only on compact partitions. Geometry changes network delivery rather than traffic volume, and uncontrolled placement can make rank-density results appear inconsistent.

B. Geometry Dominates and Constrains Rank Density

  • Flat partitions (α ≥16) increase execution time by 1.73–2.31× (mean 2.04×) versus near-cubic partitions (α ≤3) across all twelve algorithms.The pattern holds across gate counts from 91 to 1,404, with identical injected data volume for each algorithm and rank density across geometries.Source: B. Geometry Dominates and Constrains… · pp. 4–5
  • On near-cubic partitions, 16 ranks per node yields an average 11% shorter runtime than one rank per node; on flat partitions, rank-density changes provide no measurable benefit within 2%.Oversubscription performs worse on average than 16 ranks per node, though QAOA and QW achieve optimal runtimes under that configuration.Source: B. Geometry Dominates and Constrains… · p. 5
  • Uncontrolled placement can make rank-density tuning appear inconsistent because measurements mix compact geometries, where density matters, with flat geometries, where it does not.The controlled grid isolates geometry and rank density.Source: B. Geometry Dominates and Constrains… · p. 5

C. Aspect Ratio as a Predictive Surrogate

Partition aspect ratio is the strongest tested geometric predictor of runtime and supports a lightweight submission-time cost estimate. The recommended practice is to request and verify a compact partition, then use rank density as a workload- and geometry-specific tuning choice.

C. Aspect Ratio as a Predictive Surrogate

  • Across 764 configurations, aspect ratio correlates with log runtime at r = 0.72, outperforming inverse bisection width (r = 0.67) and inverse shortest-axis length (r = 0.61).Application gate count has comparatively little predictive weight (r = −0.27).Source: C. Aspect Ratio as a Predictive… · p. 5
  • The fitted power law predicts a 2.6× penalty from α = 2 to α = 24 and a 2.2× near-cubic-to-flat ratio, versus the observed 2.04×.Geometry-specific departures and correlations among geometric metrics prevent separating path length from bisection bandwidth.Source: C. Aspect Ratio as a Predictive… · p. 5
  • The α^0.39 cost model is a submission-time screening heuristic, not an exact model; compact placements also use 1.82× less energy than flat ones.The paper leaves an autonomous allocator outside its scope.Source: C. Aspect Ratio as a Predictive… · p. 5

D. Recommended Execution Practice

  • Request and verify an I/O-aligned near-cubic partition (α ≤3); use 16 ranks per node as the baseline and tune density only when workload and geometry are known.If only flat partitions are available, rank-density adjustments do not improve performance; the suggested options are to defer or run at any density.Source: D. Recommended Execution Practice · p. 5

E. Obtaining Defined Geometries on Production Schedulers

Fugaku’s scheduler padding and I/O-alignment rules conflict with the power-of-two rank count needed by state-vector simulations. Partial use of an explicitly shaped allocation enabled controlled geometry experiments, though some requested placements were not honored.

E. Obtaining Defined Geometries on Production Schedulers

  • Fugaku padded all eleven default 1,024-node submissions to 1,152 nodes, while explicit 1,024-node shapes were overridden by scheduler rules.Explicit geometries were honored only for node counts divisible by 96, which are not powers of two.Source: E. Obtaining Defined Geometries on… · pp. 5–6
  • I/O-aligned node counts and fully used uniform ranks per node cannot jointly yield the power-of-two rank count required to partition the state vector evenly.This incompatibility had made controlled placement studies of the workload impractical on Fugaku.Source: E. Obtaining Defined Geometries on… · p. 6
  • Using a 1,152-node shaped partition for execution on a 1,024-node power-of-two subset incurred 12.5% idle-node overhead, matching default padding.Against a hypothetical zero-padding flat allocation, the 2.04× speedup yielded about 1.8× more completed simulation work per allocated node-hour.Source: E. Obtaining Defined Geometries on… · p. 6
  • The study requested thirteen 1,152-node geometries spanning α = 2 to α = 24, and the scheduler granted each in at least some runs.Of 936 runs, 172 (18%) received a different size and shape and were excluded; 80 axis-order variants were retained.Source: E. Obtaining Defined Geometries on… · p. 6

IV. EVALUATION

  • The evaluation section covers setup, geometry effects, shape–density interaction, predictor comparison, energy scaling, and a simulator-free microbenchmark.These are identified as the evidence supporting the preceding methodology.Source: §IV · p. 6

A. Experimental Setup

The evaluation uses fixed-size 39-qubit workloads on Fugaku and controlled geometry and rank-density configurations. Across the retained runs, near-cubic partitions were fastest for every algorithm, with flat partitions substantially slower despite invariant network traffic volume.

A. Experimental Setup

  • The platform was Fugaku’s 48-core A64FX nodes with Tofu Interconnect D; every workload simulated 39 qubits, an 8 TiB state vector, on 1,024 nodes.The twelve fixed-depth representative kernels span five algorithm categories; the circuits were generated programmatically at full width.Source: A. Experimental Setup · p. 6, Table II · p. 6
  • The factorial grid crossed 12 algorithms, 13 geometries, and 6 rank densities; 764 of 936 runs passed the requested-shape validation and were analyzed.Runs used independent batch jobs and one timed full-circuit update pass; a simulator-free MPI all-to-all benchmark was also conducted.Source: A. Experimental Setup · pp. 6–7

B. Partition Geometry Dominates Performance

  • At 16 ranks per node, DJ injected 668.5 TB and VQC 3,166.7 TB across retained geometries, with each algorithm’s volume invariant to within 10^-6%.The passages attribute the runtime difference to network delivery across longer diameters and narrower bisections, not traffic volume.Source: B. Partition Geometry Dominates… · p. 7
  • 1.73–2.31×: mean execution time on flat partitions relative to near-cubic partitions across all twelve algorithms; the mean ratio was 2.04×.Every algorithm’s fastest retained configuration used a near-cubic geometry.Source: Table III · p. 7, B. Partition Geometry Dominates… · p. 7
  • Normalized runtime trends upward with partition aspect ratio, following a fitted T ∝ α^0.39 power law across the 173 retained algorithm–partition pairs.The reported correlation is r = 0.72, with scatter and geometry-specific departures.Source: Figure 2 · p. 7, B. Partition Geometry Dominates… · p. 7
  • Default placement assigned flat geometries to eight of eleven submissions, which ran 2.04× slower on average than near-cubic partitions.The paper reports that rank-density tuning alone cannot overcome this default-placement penalty.Source: B. Partition Geometry Dominates… · p. 7

C. Interaction Between Shape and Rank Density

Rank density affects runtime differently by partition shape: density changes are visible on near-cubic geometries but largely absent on flat ones. The authors recommend 16 ranks per node as a general default, with tuning conditioned on workload and geometry.

C. Interaction Between Shape and Rank Density

  • On near-cubic geometries, moving from R = 1 to R = 16 reduces mean execution time by 11% (1.06 vs. 0.95); on flat geometries, variation stays within 2%.Oversubscribed R = 32 is slower than R = 16.Source: C. Interaction Between Shape and Rank… · p. 8
  • The density effect is conditional on shape: controlled curves isolate it, whereas an uncontrolled protocol’s runtime swings by 2.15× and reverses direction while confounding placement with node count.The uncontrolled comparison also varied resource group.Source: C. Interaction Between Shape and Rank… · p. 8
  • R = 16 is the general default, but density tuning is recommended only when both workload and target geometry are known.On 8×18×8, four algorithms had their fastest retained cell at R = 1; on 8×9×16, R = 16 beat R = 1 for every algorithm with both runs.Source: C. Interaction Between Shape and Rank… · p. 8

D. Predictive Power of Aspect Ratio

Partition aspect ratio captures the dominant runtime trend, but individual geometries and circuit features reveal variation the scalar model does not explain. The fitted relationship is useful as a first-order predictor, not a complete account of topology sensitivity.

E. Energy Scaling Characteristics

Near-cubic partitions reduce both execution time and energy relative to flat partitions, while energy grows less than proportionally to time. A simulator-free all-to-all benchmark reproduces a similar geometry dependence in bandwidth.

E. Energy Scaling Characteristics

F. Microbenchmark Corroboration

V. DISCUSSION AND LIMITATIONS

The evaluation uses single allocation runs and representative fixed-depth circuits, and infers network-bound behavior without direct link-level measurements. Generalization beyond Fugaku is expected for contiguous torus partitions, while other network topologies require different placement models.

A. Limitations

  • Each configuration used one allocation and timed pass, so run-to-run variability across allocations is unquantified; the fixed-depth kernels do not represent full end-to-end runtimes.Source: A. Limitations · p. 9
  • Without link-level counters, network-bound behavior is inferred from invariant traffic, geometry-dependent runtimes, and the all-to-all microbenchmark rather than measured directly.Source: A. Limitations · p. 9
  • Explicit shape requests may wait longer in job queues, although compact requests showed no such penalty in this campaign; broader load-condition comparisons remain future work.Source: A. Limitations · p. 10

B. Generalization Beyond Fugaku

  • Contiguous torus systems offer a similar low-dimensional placement variable, but dragonfly and fat-tree systems need congestion-aware allocation scoring rather than a static geometry surrogate.Source: B. Generalization Beyond Fugaku · p. 10

C. Implications for Workload Management

The paper recommends treating partition geometry as a scheduling parameter and requesting compact placement. Its conclusion reports roughly twofold faster time-to-solution with near-cubic partitions and 16 ranks per node, while calling for validation and confidence intervals.

C. Implications for Workload Management

VI. CONCLUSION

  • Requesting near-cubic partitions with 16 ranks per node improves time-to-solution by roughly 2× versus mostly default-assigned flat partitions, which consume 1.82× more energy.Source: §VI · p. 10
  • Future work should establish repeated-trial confidence intervals and validate the placement hierarchy and power-law scaling on GPU simulators and alternative torus architectures.Source: §VI · p. 10
Loading 2609.34369v1…