PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, Bo Chen

arXiv:2609.38883v1cs.SE

TL;DR

Quantum software design must navigate enormous search spaces and costly evaluation, while handcrafted heuristics can be suboptimal as hardware evolves. QSA adds quantum-aware verification, execution feedback, and task-specific rewards to LLM-guided evolutionary search. It improves multiprogramming utilization and fidelity and sharply reduces error-mitigation time, with comparable or better fidelity.

  • Problem

    Quantum software design spans enormous search spaces and costly evaluations, yet often relies on handcrafted heuristics that can be suboptimal and need retuning as hardware evolves.

    Source: §1 · p. 1

  • Method

    QSA augments LLM-guided evolutionary search with quantum-specific verification, execution context, and rewards for heterogeneous quantum programs.

    Source: §1 · pp. 1–2

  • Results

    QSA improves QPU utilization by 4.2%–9.5% and fidelity by 15.2%–19.5% over the state of the art, while reducing error-mitigation execution time by at least 96.8% with comparable or better fidelity.

    Source: §1 · p. 2

  • Takeaways & Limitations

    QSA consistently outperforms state-of-the-art baselines across fidelity, QPU utilization, and execution time in the evaluated quantum-software tasks.

    Source: §11 · p. 11

    QSA is semi-automated: engineers provide the task specification and initial implementation, while full automation remains challenging for current LLMs.

    Source: §9 · p. 11

Abstract

from arXiv · show

Quantum software is critical for improving the efficiency and reliability of scarce quantum hardware. However, its design still relies heavily on ad-hoc, handcrafted heuristics that are often suboptimal and quickly become obsolete as quantum hardware evolves. LLM-guided evolutionary search offers a promising way to automatically explore complex software designs, but existing search frameworks lack the quantum-specific support needed for efficient evolution: verification is expensive, feedback is sparse, and heterogeneous quantum programs require different optimization objectives. In this paper, we present QSA, a quantum-aware harness for LLM-guided evolutionary search toward automating quantum software design. QSA equips the search with three forms of quantum-specific guidance: an evolution-hardness-guided coreset and approximate scoring to reduce verification cost, static and snapshot analyses to provide fine-grained execution context, and task-specific rewards for compiler passes and runtime policies. We evaluate QSA on the IBM Quantum platform across three benchmark suites. For multiprogramming, QSA improves QPU utilization by 4.2%-9.5% and Hellinger fidelity by 15.2%-19.5% over the state of the art. For error mitigation, QSA reduces mitigation time by at least 96.8% while achieving comparable or better fidelity. These gains require only $6.9 in LLM API cost over 11.3 hours.

1 Introduction

Quantum software design faces enormous search spaces, costly evaluation, and handcrafted heuristics that can be suboptimal as hardware evolves. QSA adds quantum-specific verification, execution context, and rewards to LLM-guided evolutionary search, improving multiprogramming and error-mitigation outcomes.

1 Introduction

  • Quantum software design combines enormous design spaces with costly simulation or hardware evaluation, encouraging suboptimal handcrafted heuristics that require retuning as hardware evolves.Pairing ten programs already permits 945 groupings; an ad-hoc multiprogramming approach achieves 45.7% lower fidelity than the offline optimum.Source: §1 · p. 1
  • LLM-guided evolution can revise source code, but generic frameworks lack quantum-specific mechanisms for efficient interaction with quantum systems.QSA targets error mitigation as a compiler pass and multiprogramming as a runtime policy.Source: §1 · p. 1
  • QSA improves QPU utilization by 4.2%–9.5% and fidelity by 15.2%–19.5% over the state of the art; for error mitigation, it reduces execution time by at least 96.8% with comparable or better fidelity.The gains cost $6.9 in LLM API usage over 11.3 hours.Source: §1 · p. 2
  • QSA equips LLM-guided evolutionary search with an evolution-hardness-guided coreset and approximate scoring, static and snapshot analyses, and rewards tailored to compiler passes and runtime policies.Runtime-policy rewards combine non-dominated-sort ranks for Pareto quality with rank correlation for generalizability.Source: §1 · p. 2

2 Background and Motivation

The background introduces quantum circuits, hybrid quantum-classical execution, and fidelity, then motivates software for scarce, noisy hardware. Existing design approaches face trade-offs and expensive searches, while a preliminary LLM-guided evolution study shows promise alongside gaps that motivate QSA.

2.1 Quantum Background

  • Quantum circuits encode programs executed on hardware; hybrid quantum software maps circuits to QPU layouts and can mitigate noise before or after execution.Hellinger fidelity measures similarity between ideal and hardware-observed output distributions.Source: §2.1 · pp. 2–3

2.2 Why Quantum Software Design Matters?

  • QOS achieves the highest fidelity among FrozenQubits, CutQC, and QOS, improving relative fidelity by up to 347.7× over Qiskit.Circuit cutting can reduce interference and shorten individual QPU executions.Source: §2.2 · p. 3
  • Scarce QPUs and noise motivate multiprogramming for throughput and error mitigation for reliability, but the design of quantum software remains costly and heuristic-driven.IBM_fez’s 99.9th-percentile queue length reaches 39,985 jobs.Source: §2.2 · p. 3, §2.3 · p. 3

2.3 Pitfalls in Quantum Software Design

  • Multiprogramming’s fixed scoring rules restrict fidelity–utilization trade-offs; at 85%–90% utilization, QOS achieves 45.7% fidelity, below the offline-optimal Oracle.For QAOA-R3 with fragment size fixed at 7, WC reaches the 60-second search limit at 18 qubits, while GV exceeds 10 seconds at 24 qubits.Source: §2.3 · pp. 3–4

2.4 LLM-Guided Evolutionary Search

  • Across 100 iterations, generic LLM-guided evolution found three reward-improving error-mitigation programs that reduced runtime by up to 19.9%, but verification was costly, feedback sparse, and rewards component-specific.The generic loop mutates code, verifies candidates, and feeds observed rewards back into later search.Source: §2.4 · p. 4

3 System Design

QSA augments a generic LLM-guided evolution agent with quantum-aware verification, prompting, and reward design. These components close a feedback loop from lower-cost measurements and execution context to guidance for subsequent mutations.

3 System Design

  • QSA’s harness supplies coreset extraction and proxy scoring, static and snapshot analysis, and task-specific rewards for circuit optimization and runtime policies.The underlying agent retains generic program selection and mutation while the optimized software interfaces with the quantum software stack.Source: §3 · p. 4
  • The verifier, prompter, and reward mechanisms form a closed feedback loop in which measurements and execution context guide subsequent mutations.The verifier reduces circuit verification and replaces costly fidelity calculations with lightweight circuit-property proxies.Source: §3 · p. 4

4 Efficient Quantum Verifier

The verifier reduces evolution-time verification cost by selecting a compact, evolution-hardness-guided coreset and replacing exact fidelity evaluation with lightweight approximate scoring. The coreset targets difficult circuit scales, while the scoring proxies reflect circuit properties and bundle balance.

4.1 Evolution-Hardness-Guided Coreset Extraction

  • Evolutionary hardness measures scale-wise difficulty from post-evolution reward: lower reward after uniform-sample-guided evolution marks a more challenging verification target.Repeated random samples are used to reduce sampling variance.Source: §4.1 · p. 5
  • Evolution-hardness-guided selection identifies 22-qubit circuits as the most challenging scale, and the final coreset samples all circuit types at that scale before full-suite evaluation of the evolved program.Focusing on this hard scale does not sacrifice performance across other circuit types or qubit counts.Source: §4.1 · p. 5, Figure 9 · p. 5

4.2 Evolution-Oriented Approximate Scoring

  • Depth and CNOT count serve as inexpensive fidelity proxies; depth has a moderate negative correlation with fidelity (−0.605).Simulation takes less than 10 seconds for a 12-qubit circuit, while a 24-qubit circuit can be nearly three orders of magnitude slower.Source: §4.2 · p. 5, Figure 11 · p. 6, Figure 10 · p. 6
  • For concurrent circuit bundles, relative balance in depth or CNOT count provides a bundle-level proxy that absolute property values alone miss.The score ranges from 0 to 1, with smaller values indicating greater imbalance.Source: §4.2 · pp. 5–6

5 Quantum-Aware Prompter

The quantum-aware prompter supplements code and reward feedback with static circuit profiles and execution snapshots. Profiles describe circuit characteristics, while snapshots expose intermediate states after operators modify quantum-related metrics.

5.1 Quantum Circuit Static Analysis

  • Static circuit profiles combine Qiskit statistics with SuperMarQ DAG-derived features, including width, depth, and CNOT count.Width captures required qubits, depth approximates the critical path, and CNOT count reflects noise-sensitive two-qubit operations.Source: §5.1 · p. 6
  • The profiles give the LLM a concise description of circuit scale, structure, and noise sensitivity.The GHZ example has width 3, depth 3, and two CNOT gates.Source: §5.1 · p. 6
  • Relating circuit metadata to program rewards and execution behavior can help the LLM identify workload-dependent patterns and develop optimization heuristics.Source: §5.1 · p. 6

5.2 Quantum Software Snapshot Analysis

  • Static profiles describe input circuits, but final rewards do not reveal where execution degrades or which operation causes the change.Source: §5.2 · p. 6
  • Snapshot analysis instruments key operators that modify quantum-related metrics and records program state after each operator completes.Collected snapshots expose intermediate circuit properties and execution outcomes at multiple workflow points.Source: §5.2 · p. 6, Figure 12 · p. 6
  • Step-by-step snapshots let the LLM associate reward changes with program decisions, localize ineffective transformations, and generate targeted mutations.Source: §5.2 · p. 6

6 Task-Specific Reward Design

QSA selects reward formulations according to how programs interact with circuits, distinguishing circuit-modifying programs from policies that optimize hardware execution. Circuit rewards aggregate normalized metrics, while policy rewards combine Pareto quality with ranking agreement.

6 Task-Specific Reward Design

  • Reward formulation is selected according to how the target program interacts with quantum circuits.Source: §6 · p. 6

6.1 Circuit-level Reward

  • Circuit-level rewards normalize system metrics into dimensionless utilities, then aggregate them with weights reflecting each metric’s relative importance.The normalized utilities are direction-aligned so larger values indicate better performance.Source: §6.1 · p. 6

6.2 Policy Reward

  • Policy rewards use Pareto ranks to capture trade-offs among conflicting objectives, avoiding reliance on a fixed weighted score.Non-dominated sorting forms successive Pareto fronts, and lower rank is better.Source: §6.2 · pp. 6–7
  • Policy correlation measures agreement between policy-assigned and ground-truth job rankings, providing a denser signal than rewards based only on selected outcomes.The correlation is computed over all candidate jobs.Source: §6.2 · p. 7
  • The combined policy reward balances average Pareto rank and policy correlation, favoring frontier-near selections while preserving orderings intended to generalize across workload conditions.The parameter α controls the relative importance of Pareto optimality and generalization.Source: §6.2 · p. 7

7 Implementation

QSA’s evolution loop runs on a Linux server, with LLM inference through external APIs and single-threaded verification. The implementation tailors rewards and prompt context to each task.

7 Implementation

  • The evolution engine runs on a 32-core AMD Ryzen 9 Linux server based on OpenEvolve, with external-API LLM inference and single-threaded verification.Each circuit has a 60-second execution deadline; each setting receives three independent 100-iteration trials, with averages and ±1 standard deviation reported.Source: §7 · p. 7
  • For multiprogramming, the implementation uses the policy reward with α=0.5 and depth ratio as a lightweight fidelity proxy; error mitigation uses a circuit reward over depth, CNOT count, and runtime.The error-mitigation metrics are averaged across benchmarks, direction-aligned so larger values are better, and weighted 1:1:1.Source: §7 · p. 7
  • Prompts include eight structural and six communication-centric static features, plus task-specific snapshots of cost traces or top-bundle Pareto ranks and system metrics.The cost trace is for 22-qubit error-mitigation circuits; multiprogramming snapshots cover top 1% circuit bundles.Source: §7 · p. 7

8 Evaluation

QSA improves multiprogramming and error-mitigation outcomes over baselines, with performance examined across LLMs, circuit settings, and harness design choices.

8.1 Experimental Setup

  • Experiments use IBM Heron Torino and Marrakesh QPUs, nine representative NISQ algorithms, and six LLMs, with Gemini 3 Pro used by default.Multiprogramming workloads pair circuits across scales while holding combined qubit count constant across policies.Source: §8.1 · pp. 7–8

8.2 Overall Performance

  • QSA improves QPU utilization by 4.2%–9.5% and fidelity by 15.2%–19.5% over QOS, while reducing mitigation time by at least 96.8% at comparable fidelity.The multiprogramming comparison is shown in Figure 13; the error-mitigation comparison is shown in Figure 14.Source: Figure 13 · p. 8, Figure 14 · p. 8, §8.2 · p. 8

8.3 Robustness Across Settings

  • Across LLMs and circuit settings, QSA generally preserves circuit quality and reduces mitigation time, though GPT-5-mini fails to improve mitigation and Claude worsens circuit properties.Across 12–24-qubit circuits, mitigation time under ratio of means remains below 10% of QOS; QSA also maintains low latency across circuit topologies.Source: §8.3 · p. 9, Figure 17 · p. 9, Figure 18 · p. 9

8.4 Performance Breakdown

  • QSA’s mitigation time grows from 1.10 s to 3.07 s as circuits increase from 12 to 24 qubits, while QPU computation increases by 21.3%.Its evolved mitigation logic bounds fragment-size search, uses binary search and caching, and avoids unnecessary cutting probes; its scheduling policy achieves average Pareto rank 5.59 and correlation 0.60.Source: Figure 21 · p. 9, Figure 22 · p. 9, §8.4 · p. 10

8.5 Ablation Study

  • Depth-based approximate fidelity scoring achieves over 96% utilization with minimal fidelity loss, while simulated fidelity does not improve the final trade-off.Instruction count, depth, and non-local gates form the reported Pareto front; depth outperforms non-local gates and CNOT count at comparable utilization.Source: §8.5 · p. 10, Figure 25 · p. 10
  • The full prompt composition is most stable and strongest, while balancing Pareto rank and policy correlation yields the best outcome at α=0.5.Disabling either static or snapshot analysis lowers fidelity and utilization; disabling policy correlation degrades both metrics, while disabling Pareto rank slightly lowers fidelity.Source: §8.5 · p. 10, Figure 23 · p. 10, Figure 24 · p. 10
  • Under a 60-second deadline, QSA reduces average mitigation time from 22.58 s to 1.06 s while preserving fidelity, outperforming random search, Bayesian optimization, and reinforcement learning.QSA evolves the decision program itself, whereas the comparison methods use predefined search or policy classes.Source: §8.5 · p. 10, Figure 26 · p. 10
  • With equal sampled-set reward of 0.36, the hardness-guided coreset achieves full-suite reward 0.12 versus −0.08 for uniform random selection.Both coresets use the same evolution setting and verification budget.Source: Figure 27 · p. 10, §8.5 · p. 11

9 Discussion

QSA automates iterative optimization after engineers specify the software component, initial implementation, and objectives, with portability depending on measurable metrics and sufficient execution feedback.

9 Discussion

  • QSA remains semi-automated because engineers supply a task specification; applying it across quantum platforms requires measurable objective metrics and sufficient execution feedback.The specification includes the target component, an initial implementation, and optimization objectives.Source: §9 · p. 11

10 Related Work

Prior quantum software and LLM optimization approaches rely on fixed heuristics or constrained search, while QSA adds quantum-specific verification, feedback, and reward mechanisms to evolutionary search.

10 Related Work

  • Existing evolutionary-search frameworks lack quantum-specific mechanisms for efficient verification, informative execution feedback, and reward design across compiler passes and runtime policies.QSA targets reusable quantum software components, unlike prior evolutionary discovery focused on a specific quantum artifact.Source: §10 · p. 11

11 Conclusion

The paper concludes that QSA applies LLM-guided evolutionary search to quantum compiler passes and runtime policies, with evaluation gains across fidelity, QPU utilization, and execution time.

11 Conclusion

  • QSA’s evaluation reports improvements over state-of-the-art baselines across fidelity, QPU utilization, and execution time.The conclusion presents quantum-aware evolutionary search as a potential approach to automated quantum software engineering.Source: §11 · p. 11

A IBM Quantum Platform Analysis

This section analyzes QSA on IBM Quantum, covering backend contention and noise, static features and search setup, evolution behavior, and evolved error-mitigation and multiprogramming programs. It documents implementation changes and reports backend-dependent conditions and reductions in relative circuit depth and CNOT count versus SOTA QOS, helping maintain or improve fidelity.

A IBM Quantum Platform Analysis

  • IBM_fez’s 99.9th-percentile queue reaches 39,985 jobs, and it faces heavier contention than IBM_torino and IBM_marrakesh; QPU error characteristics also vary by backend and over time.IBM_torino has the highest readout error, IBM_fez the lowest 1-qubit gate error, and IBM_marrakesh’s 2-qubit gate errors vary more over time, although rates are broadly comparable.Source: A IBM Quantum Platform Analysis · p. 14, Figure 29 · p. 14, Figure 30 · p. 14

B Static Feature Details

  • QSA’s static analysis extracts 14 features spanning circuit size and complexity, connectivity, and execution shape to profile circuits and prioritize candidates likely to mitigate efficiently while preserving fidelity and runtime.Examples include depth, qubit and gate counts, connected components, parallelism, liveness, measurement intensity, entanglement ratio, and critical depth.Source: B Static Feature Details · p. 14, Table 2 · p. 15

C OpenEvolve Configurations

  • OpenEvolve configurations specify editable formats, reward verification, parent selection balancing exploitation and exploration, and historical programs that guide optimization or provide diverse candidates and reusable patterns.These components are included alongside QSA’s static and snapshot analysis.Source: C OpenEvolve Configurations · p. 14, Table 3 · p. 15

D Evolution Analysis

  • Error mitigation often improves early but can continue improving throughout 100 iterations; multiprogramming improves earlier and more often, with most reward gains occurring early and Gemini 3 Pro converging fastest.Across analyzed runs with at least one improvement, the first improvement typically occurs within 27 iterations and multiprogramming’s within 2; improvements range from 0–8 for multiprogramming and 0–15 for error mitigation. For error mitigation, Gemini models first improve best reward, Claude models second, and GPT models last; Claude reaches the highest reward, while GPT-5-mini can hardly improve it.Source: D Evolution Analysis · pp. 14–15, Figure 32 · p. 14, Figure 34 · p. 15

E.1 Best Case Analysis.

  • Gemini 3 Pro evolves a bounded error-mitigation cost search and a minimalist multiprogramming heuristic using utilization and depth ratio.The mitigation policy uses metadata to set a search range, binary-searches target size, caches probes, prefers gate virtualization, and checks wire cutting only when useful; the multiprogramming policy weights utilization at 0.65 and depth similarity at 0.35, discarding 14+ features.Source: §E.1 · p. 15, §E.3 · p. 16

E.2 Failed Case Analysis.

  • Exploring combinations and permutations of mitigation techniques yields no measurable improvement over the default, indicating that operator ordering has negligible impact on mitigation efficacy.The analysis concludes that successful evolution depends on precise abstraction informed by weakness analysis.Source: §E.2 · p. 16, Figure 35 · p. 16

E.3 Initial and Best Programs

  • The evolved mitigation program uses metadata-informed bounded binary search to find the smallest feasible target size under budget, avoids unnecessary costly probes, and adds caching and final verification to reduce overhead and stabilize runtime.The multiprogramming program condenses 14+ architectural features into a weighted sum of effective utilization (0.65) and depth similarity (0.35), aiming for near-zero scheduling overhead and generalization across hardware backends.Source: §E.3 · p. 16
  • The initial error-mitigation search evaluates both GV and WC costs while adjusting target size, then selects the lower-cost method.Its loops reduce size while either method remains within budget and increase it when both exceed budget.Source: §E.3 · p. 18
  • Gemini-3-Pro’s error-mitigation search minimizes subcircuit size by binary search, prioritizes GV, and skips WC when GV costs at most 0.7 times the budget.It uses 0.2-second search timeouts and a 0.5-second timeout for final checks.Source: §E.3 · p. 19
  • Gemini-3-Pro’s best multiprogramming scorer weights effective utilization at 0.65 and depth ratio at 0.35.The initial scorer instead averages normalized effective utilization with entanglement, measurement, and parallelism comparisons.Source: §E.3 · p. 21

F QSA’s Impact on Circuit Properties

  • By up to 0.06, QSA reduces both relative depth and relative CNOT versus SOTA QOS across circuits of different scales.These reductions help explain QSA’s ability to maintain and sometimes improve fidelity.Source: F QSA’s Impact on Circuit Properties · p. 21

G Additional Ablation Studies

The ablations show that prompt composition, thinking level, and the data used during evolution affect search outcomes. Additional sections explain fidelity proxies, compare optimization strategies, and discuss robustness and possible extension to quantum error correction.

G Additional Ablation Studies

  • Removing prompt components increased mitigation time, while higher thinking levels lowered fidelity and normalized utilization and raised the time ratio.The mitigation-time increase plateaus when both the seed program and per-circuit snapshot are removed; the authors speculate that higher thinking can produce over-complicated solutions.Source: G Additional Ablation Studies · pp. 21–22, Figure 38 · p. 21, Figure 39 · p. 22
  • Using the mean across 8-, 16-, and 24-qubit utilization levels yielded the most robust generalization and balanced fidelity-utilization trade-off.The utilization levels affect both the target optimization score and the artifacts used during evolution.Source: G Additional Ablation Studies · p. 22, Figure 40 · p. 22
  • Top 1% and 5% achieved the best fidelity and normalized utilization, while 1% cost half as much as 5% with Gemini 3 Flash.The authors favor 1% from a monetary-efficiency perspective.Source: G Additional Ablation Studies · p. 22

H Fidelity Approximation Details

  • Because direct fidelity estimation can be computationally infeasible for complex circuits, QSA uses approximation metrics as surrogate optimization objectives.Table 4 summarizes the proxies and their physical rationales.Source: H Fidelity Approximation Details · p. 22, Table 4 · p. 23

I Optimization Baseline Details

  • QSA is compared with random search, Bayesian optimization, and reinforcement learning under the same decision space, workloads, simulator, and 60-second deadline.Unlike those baselines, QSA evolves the error-mitigation program logic rather than only selecting a final configuration.Source: I Optimization Baseline Details · p. 22

J Additional Discussions

  • Fidelity gains vary with hardware noise, whereas mitigation time and normalized utilization are less sensitive to calibration changes.The discussion also reports generalization to existing topologies at varying scales and proposes evolving decoder implementations or schedules for quantum error correction.Source: J Additional Discussions · p. 22
  • For quantum error correction, the proposed extension would target code paths that dominate logical error rate or decoding latency while respecting real-time latency constraints.The discussion suggests combining those quality or latency metrics with resource overhead in the reward.Source: J Additional Discussions · p. 23
Loading 2609.38883v1…