PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Replay Mitigates Sample Starvation in Quantum Architecture Search

Akash Kundu, Amit Kumar Jaiswal, Sebastian Feld, Prayag Tiwari

arXiv:2609.11248v1quant-phcs.AIcs.ETcs.LG

TL;DR

Quantum architecture search suffers from sample starvation as useful circuit trajectories become rare in expanding search spaces. GenQAS combines an MPS warm-start with learned, real-seed-anchored synthetic replay, improving fixed-budget success and resource efficiency across the evaluated benchmarks, including a 7.0× gain at 12 qubits.

  • Problem

    Quantum architecture search faces sample starvation because useful circuit trajectories occupy an increasingly small fraction of the expanding search space, while passive replay cannot create uncollected experience.

  • Method

    GenQAS combines a fixed matrix product state warm-start with a learned local transition model that generates synthetic one-step transitions from real state-action seeds for Double-DQN updates.

  • Results

    Across the evaluated benchmarks, GenQAS improves fixed-budget success probability and resource trade-offs, including a 7.0× improvement at 12 qubits and a 92.7% reduction in noisy-transfer search steps.

  • Takeaways & Limitations

    Generative replay can mitigate sample starvation and support more resource-efficient circuit discovery within the tested quantum architecture search settings.

  • Takeaways & Limitations

    The study is limited to state-vector simulation, a simple depolarizing-noise model, and a limited range of system sizes, while its warm-start assumptions may not apply to arbitrary Hamiltonians.

Abstract

from arXiv · show

Reinforcement learning (RL) can automate quantum architecture search, but its scalability is limited when useful circuit trajectories become rare in the rapidly expanding search space. Existing replay mechanisms reuse observed transitions; the proposed learned model produces additional predicted one step transitions from real state-action seeds. Here we introduce GenQAS, a tensor network-guided RL framework that combines a fixed matrix product state warm-start with prioritized generative replay. A learned local transition model generates synthetic circuit transitions on demand and mixes them with real experience during Double Deep Q-Network updates. Under a random exploration analysis, near ground state circuits occupy a rapidly shrinking region of the accessible state space. We investigate whether real data anchored synthetic replay can improve the effective training signal in this regime. Across chemical Hamiltonian benchmarks from 6 to 12 qubits, GenQAS improves fixed-budget success probability and identifies compact circuits at competitive energy error. At 12 qubits, it improves final success probability by up to $7.0\times$ over passive replay. On a 15-qubit transverse field Ising model, GenQAS increases success probability from $12\%$ to $21\%$. In a noisy 6-qubit BeH$_2$ transfer experiment, generative replay reduces the steps to chemical accuracy by $92.7\%$. These results show that generative replay can mitigate sample starvation in quantum architecture search and support more resource efficient circuit discovery.

1 Introduction

Quantum architecture search must navigate a rapidly expanding circuit space where useful trajectories become scarce and costly interactions limit learning. GenQAS addresses this sample-starvation regime by combining MPS initialization with learned, real-data-anchored synthetic replay, improving fixed-budget circuit discovery across several benchmarks.

  • Motivation: RL-QAS faces a combinatorially expanding architecture space, costly candidate evaluation, and an increasingly sparse supply of useful trajectories.At maximum construction depth T, the process can encounter up to |A_N|^T gate sequences, while candidate evaluation generally requires parameter optimization and repeated objective estimation.
  • Motivation: Uniform and prioritized replay only resample observed transitions, so they cannot expand the support of informative experience as the search space grows.The theoretical analysis shows that near-ground-state experience decreases exponentially with the dimension of the accessible Hilbert space under its stated weakly informed exploration assumptions.
  • Method: GenQAS combines a fixed MPS-derived warm start with a learned local transition model that predicts next states, rewards, and termination probabilities from real state-action seeds.Predicted one-step transitions are generated on demand and mixed with real transitions during Double-DQN value updates, without replacing the real replay distribution.
  • Empirical results: GenQAS increases final success probability from 12% to 77% at 12 qubits, a 7.0× improvement over the strongest passive replay baseline.Across matched-budget benchmarks, it also reaches 21% success on a 15-qubit transverse-field Ising model and identifies circuits with competitive energy errors, reduced CNOT counts, and reduced depth.
  • Empirical results: At 15 qubits, GenQAS improves the best success probability from 15% with prioritized replay to 21%, a 40% relative increase.
  • Empirical results: In noisy 6-qubit BeH2 transfer, generative replay reduces the steps to the first chemical-accuracy solution from 9053 to 660.This corresponds to a 92.7% reduction in search steps.

2 Results

GenQAS addresses sample starvation in quantum architecture search by augmenting real replay with learned synthetic transitions. Across scaling, transfer, diagnostics, and circuit-quality evaluations, it improves success, training stability, transfer efficiency, and resource use under the tested settings.

  • 2.1 Generative replay improves fixed budget scaling: At 12 qubits, GenQAS reaches 0.77 success probability, a 7.0× improvement over the strongest passive baseline.The passive baselines reach approximately 0.10 and 0.11, while GenQAS with Gr = 0.4 reaches 0.38 and Gr = 0.8 reaches 0.77.
  • 2.1 Generative replay improves fixed budget scaling: Across 6–12 qubits, GenQAS improves fixed-budget success, with the strongest generation ratio shifting from Gr = 0.4 at smaller systems to Gr = 0.8 at 12 qubits.The evaluated 6 to 12 qubit results show increasing relative advantage, while the optimal synthetic fraction is not strictly monotonic in Gr.
  • 2.2 Generative replay improves noisy replay buffer transfer: Transferred GenQAS replay reduces steps to chemical accuracy from 9053 to 660 in noisy 6-qubit BeH2, a 92.7% reduction.The transferred configuration also reduces CNOT count from 9 to 6, rotation count from 9 to 3, and circuit depth from 8 to 6.
  • 2.3 Generative replay diagnostics reveal a generation-ratio trade-off: At 10 qubits, Gr = 0.6 and Gr = 0.8 maintain bounded generator and DDQN losses, whereas Gr = 0.4 develops late loss increases and high-variance spikes.At 6 qubits, all investigated ratios remain comparatively bounded after initial training.
  • 2 Results: Across molecular benchmarks, GenQAS identifies competitive-accuracy circuits with substantially fewer gates or reduced depth than reference approaches.At 8 qubits, GenQAS reaches error 1.19 × 10^-3 using 3 CNOTs, 2 rotations, and depth 4, compared with 14 CNOTs, 6 rotations, and depth 8 for uniform-replay TensorRL-QAS.

3 Discussion

GenQAS combines MPS warm-starting and generative replay to address sample starvation in quantum architecture search, with gains across benchmarks and transfer settings. The study also identifies scope limits involving noise models, system sizes, assumptions, task coverage, and action-space design.

  • GenQAS combines a fixed MPS-derived warm start with a learned local transition model that generates replay transitions from real state-action seeds.Under stated theoretical conditions, the mixed replay distribution can retain a nonvanishing useful-transition contribution.
  • 77% final success probability at 12 qubits versus 12% for the strongest passive replay baseline, a 7.0× improvement.At 15 qubits, GenQAS reaches 21% success probability versus 12% for the baseline, although these finite problem sizes do not establish asymptotic scaling.
  • GenQAS often matches passive replay energy errors while using fewer CNOT gates, rotations, or circuit layers.The results represent a Pareto trade-off rather than universal dominance; at 12 qubits, GenQAS produces more compact circuits at comparable error, while uniform TensorRL-QAS attains the lowest reported energy error.
  • A noisy 6-qubit BeH2 transfer reduced steps to the first chemical-accuracy solution from 9053 to 660, a 92.7% reduction.The benefit was observed after transfer from noiseless to depolarizing evaluation and exceeded transfer using the uniform replay buffer.
  • Results on 15-qubit TFIM and 4-qubit Clifford synthesis suggest usefulness beyond molecular electronic-structure tasks.The Clifford benchmark replaces energy minimization with exact discrete target matching, where GenQAS matches or modestly exceeds the passive RL baseline.
  • The study is limited by state-vector simulation, simple depolarizing noise, limited system sizes, conditional MPS assumptions, and local rather than long-horizon model evaluation.GenQAS has also not been evaluated for quantum error correction or fault-tolerant circuit discovery, and its fixed elementary action space constrains search over higher-level motifs.

4 Methods

GenQAS combines a fixed tensor-network warm start with prioritized generative replay to augment Double-DQN quantum architecture search. Its theoretical results characterize passive replay’s sample-starvation barrier and motivate synthetic experience generation.

  • GenQAS framework: GenQAS combines a fixed MPS-derived circuit prefix with a learned local transition model that generates synthetic transitions for reinforcement-learning-based quantum architecture search.The model predicts resulting circuit states, rewards, and termination probabilities from state-action pairs.
  • GenQAS framework: The agent searches only over a learnable circuit suffix, reducing the observation dimension while keeping the tensor-network warm start fixed.The evaluated circuit is Ut = VtUTN, with the warm-start circuit excluded from the RL state.
  • RL search procedure: The action space contains valid RX, RY, RZ, and CNOT placements, with masks preventing depth, connectivity, redundant-gate, and repeated-entangling violations.The search uses DDQN, with variable-step n = 5 updates for larger problems and epsilon-greedy valid-action selection.
  • Theoretical guarantees: The good-circuit region is super-exponentially small in Hilbert-space dimension, making random exploration increasingly unlikely to find near-ground-state circuits.This sparsity is presented as the geometric basis for sample starvation in passive replay.
  • Theoretical guarantees: Passive replay cannot avoid exponential sample starvation as qubit count grows, whereas generative replay targets high-quality synthetic transitions to overcome the volume bottleneck.The theoretical analysis links the barrier to exponentially sparse high-relevance regions and states that generative replay yields an exponentially growing episode-complexity advantage.
  • Theoretical guarantees: MPS initialization is formulated as a one-step conditional generation process whose pre-training cost is polynomial in N and 1/δ.The resulting circuit is placed within a δ-neighborhood of the optimal manifold before the RL agent executes its first environment circuit.

Declarations

The supplied declarations section combines the GenQAS training algorithm requirements and steps with authorship, data, code, funding, and conflict-of-interest statements.

  • Funding and conflicts: The authors acknowledge QDNL financial support through the National Growth Fund KAT-1 program and report no conflict of interest.
  • Algorithm and implementation: Algorithm 1 requires a Hamiltonian, MPS bond dimension, RL depth, replay memory, Q networks, a generative model, and generation controls.The listed controls include the generation ratio and synthetic replay frequency.
  • Algorithm and implementation: Training initializes an MPS state and learnable suffix, evaluates real transitions, stores them, and generates synthetic batches from replayed state-action pairs.The algorithm forms synthetic transitions with the dynamics model after sampling real batches and seed pairs.
  • Data and code: The manuscript states that data can be generated from the code, with corresponding data available from the authors upon reasonable request.
  • Data and code: The code is stated to become available with the final version upon publication.
  • Author contributions: The contribution statement assigns conceptualization to A.K., code and theory preparation to A.K. and A.K.J., and supervision, investigation, and writing across the listed authors.

A.1 Proof of Theorem 4.1

The proof maps low-energy quantum states to small regions in projective state space and uses Haar/Fubini–Study geometry to bound their probability. The resulting bound explains why random circuit exploration requires rapidly increasing trials.

  • State-space geometry: The pure-state space of N qubits has real dimension 2^(N+1) − 2, and Haar measure on unitaries induces the Fubini–Study measure on this space.The unitary group acts transitively on pure states, allowing the random-circuit analysis to use projective-state geometry.
  • Good-circuit region: An energy threshold below E0 + ϵ defines a good-circuit set whose states must have fidelity above 1 − δ with the ground state, where δ = ϵ/∆.The good set is characterized as a geodesic ball around the ground state.
  • Random exploration bound: For polynomial-depth brickwork circuits, the induced distribution is exponentially close to Haar measure, so random action selection finds good circuits only with exponentially small probability.The proof converts this probability bound into a rapidly growing lower bound on the expected number of trials.

A.2 Proof of Theorem 4.2

The proof bounds the measure of low-energy states by combining local-Hamiltonian structure with concentration of measure. Selecting many disjoint local terms yields a super-exponentially small probability for the good region under Haar sampling.

  • Local-term concentration: The proof represents the low-energy condition through local Hamiltonian terms and applies Lipschitz concentration bounds to their expectation values.The centered local-term functions have bounded Lipschitz constants, enabling Lévy-type concentration on the state sphere.
  • Local-term concentration: For sufficiently large N, the median contribution of each local term is exponentially small, with |Mj| bounded by 8J · 2^(-N/2).The bound supports treating the local terms as concentrated near their Haar expectations.
  • Independent local constraints: A low total energy requires many local terms to remain simultaneously small, and a disjoint subset of terms supplies the independent events needed for a stronger probability bound.The argument uses locality to select an independent set whose terms act on disjoint qubit subsets.
  • Sparsity bound: Because the independent set grows polynomially while the Hilbert-space dimension grows exponentially, the probability of the good region becomes super-exponentially small.The proof identifies the dominant inner exponential and states that the resulting outer probability has double-exponential sparsity behavior.
  • Sparsity bound: The final asymptotic argument requires ϵ to exceed an exponentially small threshold, under which the inner exponential diverges and yields the stated sparsity bound.The proof notes that the theorem presents a practical single-exponential form after adjusting constants, while the stronger analysis is super-exponential.

A.3 Proof of Lemma 4.4

The proof identifies the MPS-to-circuit procedure as a conditional generator whose output is a high-quality synthetic transition for Hamiltonian H. Riemannian optimization maximizes circuit–MPS overlap while preserving unitary structure.

  • The MPS-to-circuit mapping acts as a conditional generator that maps H and latent entanglement structure to quantum transitions.The generated transition is τ = (s, a, s′, r), conditioned on H.
  • The circuit objective maximizes the overlap between the target MPS and the state prepared from |0⟩⊗N.This overlap is equivalent to the conditional model’s target-state likelihood objective.
  • Riemannian optimization on the unitary manifold preserves circuit unitarity while increasing the overlap objective toward a local maximum.The construction uses a brickwork circuit composed of two-qubit gates.
  • The resulting warm-start transition is injected into the synthetic replay buffer before reinforcement learning begins.It is conditioned on H and is analogous to relevance-conditioned synthetic transitions in PGR.
  • Different DMRG initializations or hyperparameters can produce an ensemble of MPS-derived circuits, increasing starting-point diversity.This parallels a fidelity–diversity trade-off in guided generative replay.

A.4 Proof of Lemma 4.5

For gapped one-dimensional Hamiltonians, the area law supplies an MPS approximation with manageable bond dimension. Compiling that MPS into a brickwork circuit yields a controlled approximation error under the stated optimization conditions.

  • For a gapped local one-dimensional Hamiltonian, the area law guarantees an MPS approximation whose bond dimension grows only polylogarithmically with system size and inverse accuracy.The resulting bond dimension is therefore bounded by a polynomial in N and 1/δ.
  • The MPS is mapped to a brickwork circuit using Riemannian optimization over products of two-qubit unitaries.The circuit uses m = O(Nχ^2) two-qubit gates.
  • Choosing the optimization error sufficiently small bounds the prepared circuit’s 2-norm distance from the MPS by δ/2.The bound follows from the overlap error and the stated tuning condition.
  • The construction therefore establishes the required approximation bound for the compiled warm-start circuit.

A.5 Proof of Theorem 4.6

The theorem argues that generative replay reduces quantum architecture-search sample complexity by shifting learning toward useful transitions instead of relying on random exploration. GenQAS implements this idea with a fixed tensor-network prefix and learned suffix search.

  • Proof of Theorem 4.6: Generative seeding shifts replay toward states near the tensor-network warm start, improving the initial value estimate relative to starting from the empty circuit.The analysis compares the tensor-network energy scale with the baseline energy gap.
  • Proof of Theorem 4.6: The base search requires an expected episode length proportional to |A|^D, whereas generative replay directly produces high-relevance transitions.This contrast drives the theorem’s sample-complexity comparison.
  • Proof of Theorem 4.6: The resulting sample-complexity ratio decays super-exponentially and is eventually bounded above by κ^-N for some κ > 1.
  • GenQAS implementation: GenQAS retains a fixed tensor-network circuit prefix while the agent searches only an additional suffix, reducing the RL state and trainable parameter dimensions.The environment still evaluates the complete circuit.
  • GenQAS implementation: The tensor-network warm start is compiled from a DMRG MPS into a shallow nearest-neighbor brickwork circuit optimized for MPS overlap.The bond dimension controls the trade-off between preprocessing cost and warm-start accuracy.

B.5 Relation to the GenQAS contribution

GenQAS combines a fixed tensor-network initialization with model-based generative replay. The two components address complementary parts of the search: initialization quality and training-data density.

  • GenQAS learns a transition-reward-termination model from real experience and uses it to generate synthetic transitions on demand.These transitions augment, rather than replace, the real replay experience.
  • The tensor-network module controls the initial state distribution and circuit-evaluation prefix, while generative replay controls the transitions used for DDQN updates.
  • The fixed warm start reduces the burden of discovering a useful initial circuit, while generative replay increases effective training-data density in the remaining search.

B.6 Warm-start resource cost and GenQAS refinement

The fixed MPS warm start has a constant compiled depth but increasing entangling-gate cost, while GenQAS refines its energy quality across evaluated systems. The section also specifies a separate exact Clifford-synthesis benchmark and its replay-enabled environment.

  • B.6 Warm-start resource cost and GenQAS refinement: The compiled MPS warm-start circuits maintain depth 27, while CNOT counts rise from 15 for 6-qubit BeH2 to 42 for 15-qubit TFIM.ROT denotes the total number of single-qubit rotations after decomposition into RX, RY, RZ, and CNOT gates.
  • B.6 Warm-start resource cost and GenQAS refinement: GenQAS improves the fixed tensor-network initialization across all evaluated systems, including a 98.5% relative error reduction for the 15-qubit TFIM.For 12-qubit H2O, the mapping error decreases from 5.07 × 10^-1 to 2.2 × 10^-2; for 15-qubit TFIM, it decreases from 1.025 to 1.5 × 10^-2.
  • Clifford synthesis benchmark: The Clifford experiment evaluates exact synthesis of target operators without variational parameter optimization or energy-expectation evaluation.Success is determined by exact equality between the synthesized and target Clifford operators.
  • Clifford synthesis benchmark: The 4-qubit Clifford environment uses 24 actions, comprising 12 single-qubit Clifford actions and 12 directed CNOT actions.The agent constructs circuits forward from the identity and uses target and current Clifford representations in its observation.
  • Clifford synthesis benchmark: Local action masking suppresses immediately repeated or cancelling gate sequences while preserving the Clifford synthesis objective.Masking applies to selected repeated H, S, S†, and ordered CNOT actions, as well as all actions at maximum episode length.

Appendix D GenQAS implementation details

The implementation combines prioritized generative replay with a DDQN agent, tensor-network initialization, molecular Hamiltonian construction, and exact noiseless simulation. Molecular benchmarks use Jordan–Wigner-mapped Hamiltonians and RL-generated circuit architectures evaluated through variational energies.

  • Implementation and training: GenQAS uses a five-layer, 1000-neuron-per-layer DDQN with prioritized replay, 20,000 stored transitions, batch size 1,000, and target updates every 100 optimization steps.The optimizer is ADAM with learning rate 3 × 10^-4 and LeakyReLU activations.
  • Implementation and training: Exploration starts at ϵ = 1.0, decays by 0.99995 to ϵmin = 0.05, and uses a tensor-network warm start with bond dimension χ = 2.No zero-parameter initialization is used, and noiseless results use exact expectation values with nshots = 0.
  • Molecular benchmarks: The molecular Hamiltonians are generated with PennyLane, mapped to qubits through Jordan–Wigner transformation, and represented as Pauli-word expansions for the variational objective.The objective is E(θ, A) = ⟨0|U†(θ, A)HU(θ, A)|0⟩, where A is the RL-generated architecture.
  • Molecular benchmarks: The molecular experiments include BeH2 and H2O instances with specified active spaces, while Qulacs performs state simulation and energy-expectation evaluation throughout RL training.The implementation also checks consistency between PennyLane and Qiskit Hamiltonian representations.
  • Tensor-network initialization: MPS ground-state approximations are optimized with Quimb at bond dimension χ = 2, converted through Qiskit into physics-informed initial circuits, and refined by sequential gate-level search.The resulting circuit provides the RL agent’s initial architecture before architecture search begins.

D.5 Mechanistic signal-density analysis

The mechanistic analysis models useful-transition density in passive and generative replay under explicit assumptions. It predicts that accurate warm starts and locally faithful transition models can prevent the synthetic training signal from vanishing as system size grows, consistent with finite-size results.

  • Passive replay: Passive replay’s useful-transition density decreases rapidly with qubit number, because replay can reweight observed transitions but cannot recover useful transitions that were never collected.The analysis defines useful transitions as those originating from circuit states within ϵ of the ground-state energy.
  • Generative replay mechanism: GenQAS mixes prioritized real transitions with on-demand synthetic transitions, with Gr denoting the synthetic fraction of each DDQN update batch.Synthetic samples are generated from real state-action seeds and are not stored persistently.
  • Mechanistic condition: The factor q separates warm-start fidelity from local transition-model accuracy, so the density condition is theoretical rather than an automatic consequence of neural-network training.Generator and critic diagnostics are used to assess stability for the investigated generation ratios.
  • Mechanistic condition: If the MPS warm start is sufficiently accurate and the learned model preserves local useful-transition structure with nonzero probability q, synthetic replay can maintain a nonvanishing contribution as system size grows.The analysis contrasts this mechanism with the rapidly shrinking useful density of passive replay.
  • Finite-size interpretation: Finite-size results are consistent with generative replay becoming relatively more valuable as passive replay signals become sparse, but they do not establish an asymptotic scaling law.The analysis connects useful-transition densification with the number of replay samples needed to obtain a fixed number of useful transitions.
Loading 2609.11248v1…