PaperlayerView exact revision on arXiv

Source-linked AI summary

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy, Hartmut Neven

arXiv:1803.01857v2quant-phmath.OC

TL;DR

Quantum control needs fast, high-fidelity, universal operations that remain robust to leakage and stochastic control errors. The paper proposes UFO, which combines an analytic leakage bound and reinforcement learning trained in noisy environments to optimize control trajectories. For a broad family of two-qubit gates, the approach reduces average gate infidelity by up to one order of magnitude over noise-free alternatives and gate time by up to one order of magnitude over optimal gate synthesis.

  • Problem

    A universal quantum-control framework is needed to optimize speed and fidelity while accounting for leakage, stochastic control errors, and experimental constraints.

  • Method

    UFO combines an analytic leakage bound with open-loop reinforcement learning trained under stochastic control noise and a cost function containing soft penalties for leakage, runtime, fidelity, and constraints.

  • Results

    Up to one order of magnitude lower average gate infidelity and up to one order of magnitude shorter gate time were achieved than the stated noise-free and optimal-synthesis alternatives.

  • Takeaways & Limitations

    The framework provides robust, fast, high-fidelity analog control for a broad family of two-qubit gates relevant to quantum simulation.

  • Takeaways & Limitations

    The current study does not focus on replacing adaptive step size with parallel RL agents dedicated to fixed target gates.

Abstract

from arXiv · show

Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts.

INTRODUCTION

Quantum control must produce fast, high-fidelity, universal operations despite leakage, stochastic control errors, and platform-specific Hamiltonian constraints. UFO combines analytic leakage modeling with deep reinforcement learning to optimize robust control trajectories, achieving substantial improvements in gate error and runtime.

  • Quantum controls translate algorithms into analog signals, and their design determines each operation’s fidelity and speed.
  • Universal analog controls can replace long gate-synthesis circuits, reducing circuit depth for near-term quantum computation.Unrestricted unitary gates have reduced required circuit depth by one order of magnitude in demonstrations.
  • Existing control optimization lacks a universal framework that jointly handles stochastic control errors, leakage, and systematic experimental constraints.Closed-loop methods require frequent measurements, while open-loop robustness analyses become computationally expensive for multi-qubit systems.
  • UFO derives an analytic leakage bound using time-dependent Schrieffer-Wolff transformations and a generalized adiabatic theorem.The framework combines physical knowledge of quantum dynamics with state-of-the-art reinforcement learning.
  • UFO’s universal cost function jointly optimizes leakage, control-boundary violations, runtime, and gate fidelity through soft penalties.
  • Up to one order of magnitude shorter gate times and around two orders of magnitude lower average infidelity were achieved than the reported synthesis and SGD baselines.The results also include an order-of-magnitude reduction in fidelity variance relative to noise-free RL and SGD solutions.

LEAKAGE ERROR BOUND

The leakage analysis decomposes the Hamiltonian into energy, intra-subspace, and inter-subspace couplings, then uses TSWT and an adiabatic theorem to bound dominant leakage mechanisms. The resulting framework accounts for both coherent and non-adiabatic leakage across resonant regimes.

  • The rotating-wave Hamiltonian is decomposed into energy, block-diagonal, and block-off-diagonal components to identify leakage sources.
  • The RL environment evaluates noisy Hamiltonian dynamics, produces an updated unitary gate and control cost, and feeds these trajectories to policy and value networks.
  • Single-qubit X and Y controls cannot be removed independently because they are also crucial for full qubit controllability.
  • TSWT suppresses the effective block-off-diagonal Hamiltonian to higher orders when coupling strengths are much smaller than the inter-subspace energy gap.The stated assumption is ϵ/∆ ≪ 1.
  • Equation-based bounds address coherent leakage from residual block-off-diagonal coupling and non-adiabatic leakage from fast time-dependent modulation.The non-adiabatic contribution is not dominant in the off-resonant regime, while the coherent bound captures dominant errors in both regimes.

UNIVERSAL COST FUNCTION

UFO uses a soft-penalty cost function to optimize gate fidelity, leakage, runtime, and boundary constraints together. Its terms are weighted so the framework can balance competing control objectives and adapt to other platforms.

  • The UFO cost function includes gate infidelity, accumulated leakage, total runtime, and control-boundary violations as optimizable soft penalties.
  • Gate infidelity measures the discrepancy between the implemented unitary and the target unitary, vanishing when they agree up to a global phase.
  • Boundary penalties enforce vanishing microwave and g-pulses at both endpoints, supporting gate concatenation and reducing non-RWA errors.
  • The coefficients χ, β, µ, and κ weight infidelity, leakage, boundary violations, and runtime, respectively.These hyperparameters are optimized to balance the joint control objective.

TWO-QUBIT GATE CONTROL OPTIMIZATION

The UFO framework optimizes fast, high-fidelity two-qubit controls while accounting for leakage and stochastic control errors, using RL to exploit system–gate regularities and transfer learning. Across the N(α, α, γ) family, the controls improve runtime and robustness relative to optimal gate synthesis and noise-free or SGD baselines.

  • Control framework: The framework defines robustness through bounded deviations in average gate fidelity under a specified control-noise model.The criterion uses average fidelity and fidelity variance over sampled noise instances.
  • Control framework: The target family N(α, α, γ) includes SWAP, ISWAP, CNOT, CZ, fermionic swap, and Givens rotations up to single-qubit rotations.These gates are relevant to quantum chemistry and quantum simulation, including Jordan–Wigner transformations.
  • Benchmark: The optimal synthesis counterpart is a depth-seven circuit with three two-qubit gates and five single-qubit gates, estimated at 215ns.The runtime estimate uses 20ns per single-qubit gate and 45ns per CNOT.
  • Results: Gate times for γ = π/2, π/6, and π/3 are compared against an approximately 200ns synthesis baseline, with leakage errors bounded by O(10^-4) and gate infidelity by O(10^-3).The same figure reports these error bounds for all cases.
  • Robustness: RL agents learn nearby target gates faster after successfully learning a related target, providing heuristic evidence of transfer learning.The experiment uses an adaptive step size in α to maintain a constant upper bound on total optimization time.
  • Results: A factor-of-10 runtime improvement is reported for N(α, α, π/2) with α ∈[0, π] over optimal gate synthesis.The improvement is attributed to direct realizability of the entangling component through the gmon Hamiltonian without detuning or microwave controls.
  • Limitations: The framework’s current adaptive-step-size approach could be replaced by parallel agents, while isolated gate-time peaks may reflect control singularities requiring further study.The paper identifies these issues as scope boundaries or open directions rather than established failures.
  • Robustness: Noise-optimized RL controls maintain average fidelity within [99.5%, 98%] across Gaussian noise variances from 0.1MHz to 3.5MHz.At σnoise = 1MHz, the reported robustness criterion uses ϵ0 = 0.007; comparisons include noise-free RL and Adam-based SGD.

CONCLUSION

The UFO framework optimizes fast, high-fidelity quantum gates with reinforcement learning while accounting for stochastic control errors and incomplete environmental models.

  • CONCLUSION: UFO performs open-loop quantum control optimization with reinforcement learning, encoding control trajectories and the control cost function in neural networks.The framework trains both networks in a stochastic environment that mimics noisy control actuation.
  • CONCLUSION: Up to one order of magnitude lower average gate infidelity is achieved than noise-free alternatives.
  • CONCLUSION: Up to a one-order-of-magnitude reduction in gate time is achieved relative to the optimal gate synthesis solution.
  • CONCLUSION: The work opens a direction for quantum analog control optimization that incorporates unpredictable control errors and incomplete physical models of environmental interactions.

Appendix A: Gmon Hamiltonian and Control Noise

The study focuses on the two-qubit gmon Hamiltonian in the perturbative regime as an example system for quantum control analysis.

  • Appendix A: Gmon Hamiltonian and Control Noise: The two-qubit gmon Hamiltonian is obtained by quantizing a phenomenological classical Hamiltonian for nonlinear L-C circuits containing Josephson junctions.

1. Quantum Oscillator Picture

The appendix develops a quantum-oscillator model of coupled Josephson L-C circuits and specifies the resulting Hamiltonian, controls, approximations, and stochastic control-noise model.

  • Quantum Oscillator Picture: The Josephson junction circuit is modeled as a nonlinear quantum oscillator with inductive and capacitive energy contributions.
  • Quantum Oscillator Picture: Second quantization represents the circuit phase as a quadrature of a bosonic field, yielding a harmonic oscillator Hamiltonian with frequency ω0 = √κ.
  • Quantum Oscillator Picture: Two-qubit coupling is realized by tunable capacitive coupling between the Josephson circuits, while microwave controls provide tunable frequencies, phases, and amplitudes.
  • Quantum Oscillator Picture: The interaction-picture Hamiltonian uses detunings and applies the rotating-wave approximation to omit highly oscillating components.
  • Quantum Oscillator Picture: The perturbative regime requires two-qubit coupling, detuning, and microwave amplitudes to remain at least one order of magnitude below η.
  • Quantum Oscillator Picture: The stochastic training environment adds zero-mean Gaussian amplitude fluctuations with 1 MHz variance to the control amplitudes at every discretized time step.

Appendix B: Complete Leakage Bound

The appendix derives the total leakage cost by suppressing direct coupling leakage through a rotated computational basis and bounding non-adiabatic leakage.

  • Appendix B: Complete Leakage Bound: The leakage-bound derivation begins with a time-dependent Schrieffer-Wolff transformation that defines a rotated computational basis and suppresses direct coupling leakage to higher order.It then proves that non-adiabatic leakage is sub-dominant under the generalized adiabatic-theorem analysis.

1. Time-dependent Schrieffer-Wolff transformation

The time-dependent Schrieffer-Wolff transformation perturbatively rotates the Hamiltonian to suppress couplings between energy subspaces. Its expansion separates block-diagonal and block-off-diagonal dynamics and cancels the latter order by order.

  • Hamiltonian decomposition: The Hamiltonian is decomposed into a static term, within-subspace coupling, and between-subspace coupling.The latter two are called the block-diagonal and block-off-diagonal terms, respectively.
  • Perturbative regime: The transformation assumes the inter-subspace and intra-subspace couplings are much smaller than the separating energy gap.This scale separation supports the perturbative expansion.
  • Perturbative rotation: The rotated basis is defined by an anti-Hermitian, block-off-diagonal generator whose perturbative series is S = ϵS_1 + ϵ^2S_2 + ... + ϵ^nS_n.The generator contains terms only between different subspaces.
  • Effective Hamiltonian: The effective Hamiltonian is expanded into block-diagonal and block-off-diagonal components using nested commutators.The commutator structure determines which orders contribute to each component.
  • Order-by-order cancellation: The rotation is solved order by order so that block-off-diagonal terms vanish through a chosen perturbative order.The paper derives first- and second-order rotations and uses the resulting effective coupling to evaluate leakage.

2. Leakage Bound

The leakage analysis combines a direct-coupling bound obtained in the rotated basis with a generalized adiabatic theorem for non-adiabatic transitions. Under the stated regimes, direct coupling provides the dominant leakage contribution.

  • Direct coupling leakage: The rotated-basis interaction-picture analysis evaluates population leaving the qubit subspace through the block-off-diagonal Hamiltonian.The block-diagonal evolution preserves the computational subspace, so leakage is attributed to the remaining inter-subspace coupling.
  • Bound simplification: The middle two terms of the non-adiabatic bound are at least one order of magnitude smaller than the leading terms in both frequency regimes.They are therefore omitted from the final leakage bound.
  • Frequency regimes: The leakage bound is analyzed separately in off-resonant and on-resonant frequency regimes.The regimes differ according to whether the Hamiltonian modulation frequency is much smaller than, or comparable to, the energy gap.
  • Non-adiabatic leakage: A generalized adiabatic theorem bounds leakage amplitude caused by time-dependent non-adiabatic transitions.The theorem compares the evolved state with an instantaneous eigenstate under a Hamiltonian with separated energy subspaces.
  • Dominant contribution: O(ϵ^4/∆^4) non-adiabatic leakage is one order lower than O(ϵ^3/∆^3) direct-coupling leakage in the off-resonant regime.Consequently, off-resonant leakage errors are dominated by direct coupling errors.

Appendix C: Control Filter Design

The control-filter design limits the frequency content of reinforcement-learning actions so that modulation remains sufficiently slow relative to the energy gap. A two-pole normalized double-exponential filter preserves the Markovian control formulation.

  • Trajectory discretization: Each control trajectory is represented as N piecewise-constant steps separated by ∆t.The modulation rate is set by F_sample = 1/∆t.
  • Frequency constraint: The control frequency components must remain sufficiently small compared with the energy gap ∆ to suppress the leakage-bound term involving the second time derivative.This requirement motivates filtering the agent’s proposed actions.
  • Filter choice: A two-pole normalized double-exponential smoothing filter is applied to each proposed control action.The filter is chosen without affecting the Markovianity of the control problem.
  • Filter recurrence: The filtered control at step n depends on the agent’s proposed action and the controls from the previous two steps.The recurrence uses c_RL,n, c_n−1, and c_n−2.
  • Coefficient selection: The filter coefficient is selected from the actuated-control bandwidth B_w and the modulation rate.This links the smoothing strength to the desired frequency bandwidth.

Appendix D: Evaluation of the Average Fidelity

The paper evaluates average gate fidelity under noisy quantum-channel realizations by averaging over stochastic control trajectories. The metric compares the resulting operation with the target unitary through Pauli-operator overlaps.

  • Fidelity definition: Average fidelity measures quantum-gate performance over uniformly distributed input states under the noisy channel E.The noisy channel represents actual control realizations.
  • Pauli representation: Theoretical average fidelity is computed by summing overlaps with Pauli operators.The two-qubit computational subspace has dimension d = 4.
  • Ideal operation: The ideal operation is represented by U_target, while E is the trace-preserving operation describing noisy realizations.Average fidelity equals one when E(ρ) = U_targetρU_target†.
  • Noise model: The noisy channel is evaluated by sampling full control trajectories with Gaussian amplitude fluctuations N(0, σ_noise) and averaging the evolved states.The fluctuations have zero mean and fixed variance σ_noise.
  • Stochastic averaging: Averaging noisy realizations can convert a pure state into a mixed state, thereby accounting for decoherence induced by stochastic control errors.Figure 4 uses 60 samples per noise-model parameter.
Loading 1803.01857v2…