PaperlayerView exact revision on arXiv

Source-linked AI summary

QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RL

Cong Yu, Valter Uotila, Shilong Deng, Qingyuan Wu, Tuo Shi, Songlin Jiang, Lei You, Bo Zhao

arXiv:2510.00967v1cs.AIquant-ph

TL;DR

LLMs can generate low-quality or incorrect quantum circuits, while parameterized gates require precise values that depend on circuit structure and parameters. QUASAR combines tool-augmented agentic RL, external quantum verification, and hierarchical rewards for OpenQASM generation. The authors report improvements across syntax and semantics, including 99.31% Pass@1 syntactical correctness and 100% Pass@10 syntactical correctness.

  • Problem

    Parameterized gates need precise values that depend on circuit structure, while LLMs may lack the quantum-specific knowledge needed to generate high-quality, correct circuits.

    Source: §1 · p. 2

  • Method

    QUASAR post-trains LLMs with agentic RL, external quantum verification, and a hierarchical reward that evaluates syntax, distributional alignment, task-specific expectation values, and optimization usability.

    Source: §1 · p. 3, §6 · p. 9

  • Results

    99.31% Pass@1 syntactical correctness and 100% Pass@10 syntactical correctness; QUASAR also outperforms leading industrial LLMs and SFT-only and RL-only baselines across quantum optimization benchmarks.

    Source: §5.2 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    The results show that tool-augmented RL can bridge general-purpose LLMs and domain-specific quantum code generation, with distributional alignment identified as the key driver of quality.

    Source: §6 · p. 10

    QUASAR remains limited as a warm-start method compared with the rule-based WarmStartQAOAOptimizer, and it also trails that optimizer on the evaluated HUBO problems.

    Source: E LIMITATIONS · p. 22

Abstract

from arXiv · show

Designing and optimizing task-specific quantum circuits are crucial to leverage the advantage of quantum computing. Recent large language model (LLM)-based quantum circuit generation has emerged as a promising automatic solution. However, the fundamental challenges remain unaddressed: (i) parameterized quantum gates require precise numerical values for optimal performance, which also depend on multiple aspects, including the number of quantum gates, their parameters, and the layout/depth of the circuits. (ii) LLMs often generate low-quality or incorrect quantum circuits due to the lack of quantum domain-specific knowledge. We propose QUASAR, an agentic reinforcement learning (RL) framework for quantum circuits generation and optimization based on tool-augmented LLMs. To align the LLM with quantum-specific knowledge and improve the generated quantum circuits, QUASAR designs (i) a quantum circuit verification approach with external quantum simulators and (ii) a sophisticated hierarchical reward mechanism in RL training. Extensive evaluation shows improvements in both syntax and semantic performance of the generated quantum circuits. When augmenting a 4B LLM, QUASAR has achieved the validity of 99.31% in Pass@1 and 100% in Pass@10, outperforming industrial LLMs of GPT-4o, GPT-5 and DeepSeek-V3 and several supervised-fine-tuning (SFT)-only and RL-only baselines.

1 INTRODUCTION

The paper targets OpenQASM 3.0 generation, presenting QUASAR as an agentic RL framework that combines quantum-simulator verification with hierarchical rewards. Its stated contributions include a combined SFT/RL pipeline and evaluation on QAOA and VQE circuits.

1 INTRODUCTION

  • OpenQASM represents circuits in a general, machine-independent low-level language, allowing generated circuits to be deployed across quantum machines.The paper distinguishes it from Python-based frameworks such as Qiskit and Cirq and describes it as closer to QPU hardware.Source: §1 · p. 1
  • Generated code ranges from compilation failure to the desired outcome of compiling with the correct qubit count and near-optimal parameters.The figure distinguishes compilation success alone from correct qubit count and parameter quality.Source: Figure 1 · p. 2
  • QUASAR uses agentic RL post-training to generate OpenQASM 3.0 circuits, with an external quantum-verification tool and a four-level reward mechanism.The reward enforces correctness first, then promotes stronger task alignment in generated circuits.Source: §1 · p. 2
  • Its reward stages assess syntax, distributional alignment, task-specific expectation-value discrepancies, and usability during continued circuit optimization.Syntax failure prevents computation of the other rewards; the final stage combines optimization-step count and final optimized expectation value.Source: §1 · p. 3
  • The end-to-end pipeline combines supervised fine-tuning and RL post-training while interacting with an external quantum-verification tool.This is presented as an efficient tool-interaction pipeline.Source: §1 · p. 3
  • The 4B-Qwen3 model augmented by QUASAR outperforms several leading industrial LLMs and generates practical ansatz patterns and parameter initializations for QAOA and VQE.The evaluation is presented as evidence of QUASAR’s potential in scalable quantum circuit and algorithm design.Source: §1 · p. 3

2 RELATED WORKS

Related work spans LLM-based quantum coding, circuit synthesis, ansatz design, and quantum experiment design. The paper distinguishes its OpenQASM 3.0 target from prior work tied closely to Qiskit and notes limitations in some existing circuit-generation approaches.

2 RELATED WORKS

  • Prior LLM efforts include Qiskit code assistance and RL post-training, multi-agent circuit synthesis using CoT and RAG, ansatz design, quantum experiment design, and guided ansatz design for finance.The paper positions its approach as targeting the platform-independent OpenQASM 3.0 standard, supported by Qiskit, PennyLane, and Cirq.Source: §2 · p. 3
  • Other approaches include transformer-based measurement prediction and circuit optimization, KetGPT’s OpenQASM 2.0 circuits without parameters, and QAOA-GPT for MaxCut.The paper also notes prior use of a quantum-verifiable reward for post-training LLMs to generate Qiskit code.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries introduce quantum optimization and its parameterized circuits, describe OpenQASM as a circuit representation, and outline agentic RL with tool use. Together, these topics frame the paper’s circuit-generation setting and training approach.

3.1 QUANTUM COMPUTING AND QUANTUM OPTIMIZATION

  • Parameterized quantum circuits combine unitary gates such as rotations and entangling operations; quantum optimization seeks states whose Hamiltonian expectation approximates the ground state.QAOA, VQE, and adaptive VQE use different ansatz strategies within hybrid quantum-classical parameter optimization.Source: §3.1 · p. 4

3.2 OPENQASM LANGUAGE

  • OpenQASM is a low-level intermediate representation for quantum algorithms that standardizes circuits and supports exchange across quantum hardware platforms.The paper names Qiskit, Cirq, QDK, and Forest among frameworks supporting it.Source: §3.2 · p. 4

3.3 AGENTIC REINFORCEMENT LEARNING WITH TOOL USE

  • Agentic RL with tool use lets an LLM interact with an environment and learn from verified feedback; GRPO is described as a commonly used algorithm in this setting.The GRPO objective uses rollout trajectories, a token-level importance ratio, and a normalized advantage.Source: §3.3 · p. 4

4 QUASAR DESIGN

QUASAR post-trains an LLM through simulator-based verification and staged rewards for quantum circuits. Its reward design moves from syntax checks to task-aware circuit evaluation and optimization progress.

4.1 RL POST-TRAINING PIPELINE

  • The RL loop sends proposed OpenQASM circuits to an external simulator, returns structured rewards and errors, and uses that feedback for GRPO policy updates.Non-CoT supervised fine-tuning first bootstraps syntactic competence.Source: §4.1 · p. 5

4.2 REWARD DESIGN

  • The four-level reward checks syntax first, then distributional alignment, problem-Hamiltonian expectation values, and progress under local optimization.The staged procedure uses expectation-value evaluation when distributional divergence exceeds a threshold, and runs optimization when either score clears its threshold.Source: §4.2 · pp. 5–6

4.2.1 SYNTACTIC REWARD

  • A syntactic reward checks whether the Qiskit QASM 3.0 parser accepts the OpenQASM code; failed parsing ends reward computation with −1.The other rewards are not computed if parsing fails.Source: §4.2.1 · p. 6

4.2.2 ENTROPY REWARD

  • The entropy reward scores generated-versus-reference distribution similarity with a Jensen–Shannon distance reward from 0 to 1, while accounting for qubit-count mismatches.It compares the first k qubits after marginalization and penalizes active extra qubits and extra-qubit entanglement; idle or reset-only ancillas incur little or no penalty.Source: §4.2.2 · p. 6

4.2.3 EXPECTATION-VALUE REWARD

  • For syntactically valid circuits, the expectation-value reward evaluates the generated circuit’s problem-specific cost Hamiltonian value against its minimum and maximum eigenvalues.The value is min-max normalized; the Hamiltonian is padded with identities to match circuit width, preventing ancillary-qubit reward hacking.Source: §4.2.2 · p. 6, §4.2.3 · p. 7

4.2.4 OPTIMIZATION REWARD

  • The optimization reward favors circuits that reach a lower expectation value with fewer optimization steps, using 1/(1+n) for the step-count component.Here n is the number of steps required to reach an optimized circuit for the task.Source: §4.2.4 · p. 7

5 EXPERIMENTS

The experiments evaluate QUASAR against prompted industrial LLMs and SFT-only and RL-only baselines, then analyze its results and reward components. QUASAR improves syntactic and semantic metrics, while ablations identify distributional alignment as the primary driver of performance.

5.1 EXPERIMENTAL SETUP

  • Setup uses a 12-primitive graph dataset with optimized QAOA/VQE-family circuits; four metrics compare prompted industrial LLMs, SFT-only, and RL-only baselines.The metrics are SCR, SREV, RE, and HQCR, measured at Pass@1 and Pass@10.Source: §5.1 · pp. 7–8

5.2 RESULTS

  • 99.31% Pass@1 SCR; QUASAR reports +12.95% SREV over Cold Start (GRPO) (22.41 vs. 19.41), +8.87% lower RE than SFT (11.61 vs. 12.74), and +13.64% HQCR over SFT.Pass@10 reaches 100% syntactical correctness.Source: Table 1 · p. 8, §5.2 · p. 8
  • QUASAR beats 100 random parameter variants with lower JS-divergence (0.79 vs. 0.95) and expectation values closer to optimum (0.16 vs. 0.36).The randomized parameters were sampled from (−π, π].Source: §5.2 · pp. 8–9

5.3 ABLATION STUDY OF THE HIERARCHICAL REWARD

  • Matched-compute ablations identify RE as the primary driver; EV helps borderline QASMs, Opt boosts HQCR, QMP limits mismatch-related reward noise, and validity-only rewards lag on semantic metrics.Each ablation disabled one component while keeping the rest unchanged.Source: §5.3 · p. 9, Table 2 · p. 9

5.2 RESULTS

  • At best, HQCR gains are +13.65% over SFT, +72.4% over DeepSeek-V3, and up to 1.65×/1.50× over GPT-4o/GPT-5; median and upper-quartile ∆E fall 4.9%/9.7% versus SFT.HQCR thresholds were varied from 0.1 to 0.9; lower ∆E indicates closer expectation values.Source: §5.2 · p. 8, Figure 4 · p. 9, Figure 5 · p. 9

6 CONCLUSION

QUASAR combines external verification, quantum-aware RL, and hierarchical rewards to generate OpenQASM 3.0 circuits. The conclusion identifies distributional alignment as the key quality driver and reports gains over industrial, SFT-only, and RL-only baselines across quantum optimization benchmarks.

6 CONCLUSION

  • QUASAR integrates external verification with quantum-aware RL and hierarchical rewards to generate OpenQASM 3.0 circuits with syntactic validity and semantic fidelity.The reward hierarchy enforces syntax, aligns distributions, reduces expectation-value errors, and promotes optimization efficiency.Source: §6 · p. 9
  • Distributional alignment is the key quality driver, while expectation-value and optimization-progress terms provide complementary gains.The authors report that QUASAR outperforms leading industrial LLMs and SFT-only and RL-only baselines across quantum optimization benchmarks.Source: §6 · p. 10
  • Tool-augmented RL is presented as a bridge between general-purpose LLMs and domain-specific quantum code generation, with broader automated quantum algorithm design as a future direction.The conclusion notes that Appendix E discusses limitations and future directions.Source: §6 · p. 10

A IMPLEMENTATION DETAILS

QUASAR’s implementation uses a 4B SFT model and agentic RL training, while the background introduces parameterized circuits and hybrid quantum optimization. It explains how QAOA and VQE use parameterized circuits to minimize Hamiltonian expectation values.

A IMPLEMENTATION DETAILS

B QUANTUM COMPUTING AND QUANTUM OPTIMIZATION

  • Quantum circuits apply unitary gates to qubits, which can occupy superpositions and are represented in Hilbert spaces.Parameterized rotations combined with standard gates such as Hadamard, CNOT, and CZ form circuits U(θ).Source: B QUANTUM COMPUTING AND QUANTUM… · p. 16
  • Quantum optimization maps binary problems such as QUBO and HUBO to Hamiltonian minimization, seeking states with low measured expectation values.The problems can be rewritten using spin variables and Pauli-Z operators.Source: B QUANTUM COMPUTING AND QUANTUM… · pp. 16–17
  • Parameterized unitary circuits prepare states whose Hamiltonian expectation values can be estimated and minimized using a classical optimizer.For fixed θ, a quantum computer estimates f(θ); minimizing it can yield a measured bitstring that is a solution to the optimization problem.Source: B QUANTUM COMPUTING AND QUANTUM… · p. 17
  • QAOA uses a cost-Hamiltonian-based ansatz and mixer, whereas VQE can use expressive or hardware-efficient ansatzes independent of the cost Hamiltonian.Adaptive VQE selects gates and their circuit positions from a user-defined pool using gradient evaluations.Source: B QUANTUM COMPUTING AND QUANTUM… · p. 17

C.1 OFFLINE DATA COLLECTION

Before training, the authors collect problem-specific ground-truth metadata offline for the quantum agent to evaluate model-proposed OpenQASM circuits.

C.1 OFFLINE DATA COLLECTION

  • For each problem, offline metadata includes its cost Hamiltonian, ground-truth parameterized circuit, and that circuit’s largest and smallest Hamiltonian eigenvalues.The quantum agent uses this metadata to evaluate the LLM’s proposed OpenQASM code.Source: §C.1 · p. 18

C.2 QUANTUM OPTIMIZATION PROBLEMS

The section describes QUBO or HUBO formulations for graph and network optimization problems, encoding each problem’s constraints and objective in binary variables.

C.2.1 CONNECTED COMPONENT FOR A NODE

  • Connected-component QUBO terms enforce adjacency while discouraging unnecessary selected vertices; the fixed node is activated before solving.The authors describe this problem as a common graph optimization primitive that is not NP-hard and can be encoded as QUBO.Source: §C.2.1 · p. 18

C.2.2 COMMUNITY DETECTION

  • Community detection uses a modularity-maximizing QUBO with binary node-to-community assignments and a one-hot constraint ensuring each node belongs to one community.The section notes that the problem is NP-hard.Source: §C.2.2 · pp. 18–19

C.2.3 K-SIZED CLIQUE

  • The k-sized clique formulation selects k vertices and requires k(k −1)/2 edges, with positive weights chosen so A > kB.Source: §C.2.3 · p. 19

C.2.4 GRAPH ISOMORPHISM

  • Graph-isomorphism constraints require a bijection between vertices that respects graph edges.The text cites QUBO, adiabatic-computing, and boson-sampler formulations.Source: §C.2.4 · p. 19

C.2.5 GRAPH COLORING

  • Graph coloring uses one-hot variables to assign each vertex a color and penalizes adjacent vertices that share a color.Source: §C.2.5 · p. 19

C.2.6 TRAVELING SALESMAN

  • The traveling-salesman formulation uses two constraints, with penalty terms chosen so 0 < B max wuv < A.The passage identifies the decision problem as NP-complete.Source: §C.2.6 · p. 20

C.2.7 WEIGHTED MINIMUM MAXIMAL MATCHING

  • Weighted minimum maximal matching adapts an exact-set-cover formulation: one constraint activates exactly one edge for each node, and another minimizes matching weight.Source: §C.2.7 · p. 20

C.2.8 VERTEX AND EDGE COVERS

  • Vertex cover enforces edge coverage and minimizes selected vertices; edge cover enforces vertex coverage and minimizes selected edges.The edge-cover constraint can be reduced from higher-order to quadratic using slack variables.Source: §C.2.8 · pp. 20–21

D COMPARISON WITH RANDOM BASELINE.

QUASAR-generated circuit parameters were closer to optimal than uniformly random initializations on average across the reported distributional and expectation-value comparisons.

D COMPARISON WITH RANDOM BASELINE.

  • Both JS-divergence and expectation values were closer to optimal on average for QUASAR-generated circuits than for 100 uniformly randomized initializations in (−π, π].The comparison is reported in Table 4.Source: D COMPARISON WITH RANDOM BASELINE · p. 21

E LIMITATIONS

QUASAR remains behind WarmStartQAOAOptimizer as a warm-start method on the reported QUBO and HUBO comparisons, and the evaluation covers only syntactically correct cases. The authors suggest extending training data with QASMs prepared by WarmStartQAOAOptimizer.

E LIMITATIONS

  • The reported warm-start comparison covers syntactically correct cases: 564 of 580 total test cases.The evaluation used Gurobi for the WarmStartQAOAOptimizer presolver.Source: E LIMITATIONS · p. 22
  • 0.1600 for QUASAR versus 0.007 for WarmStartQAOAOptimizer on the reported average f min max value indicates QUASAR remains limited as a warm-start method.The comparison used 564 syntactically correct QASMs from 580 test cases.Source: E LIMITATIONS · p. 22
  • 0.2356 for QUASAR versus 0.0656 for WarmStartQAOAOptimizer on the 60 compiled HUBO problems, although the baseline performed worse on HUBO than QUBO.The test set contained 62 HUBO cases; the authors suggest extending training with QASMs prepared by WarmStartQAOAOptimizer.Source: E LIMITATIONS · p. 22

Related papers

Loading 2510.00967v1…