PaperlayerView exact revision on arXiv

Source-linked AI summary

Reinforcement Learning for Quantum Technology

Marin Bukov, Florian Marquardt

arXiv:2601.18953v1quant-phcond-mat.quant-gascond-mat.stat-mechcs.AIcs.LG

TL;DR

Quantum technologies pose control, compilation, and error-correction challenges under quantum measurement constraints. This review explains RL for quantum systems and surveys theoretical and experimental applications, including state preparation, gates, circuits, feedback, and error correction. It identifies sparse observations, adaptation, and multi-objective optimization as important open challenges for practical deployment.

  • Problem

    Quantum technologies require solutions that respect inaccessible quantum states, probabilistic measurements, and irreversible wavefunction collapse across their development pipeline.

  • Method

    The review introduces RL concepts for quantum systems and synthesizes applications spanning state preparation, control, circuit design, feedback, error correction, metrology, and quantum RL.

  • Results

    RL has been demonstrated in experiments for state preparation, entangling-gate optimization, quantum-circuit construction, real-time feedback, and quantum error correction.

  • Takeaways & Limitations

    Practical quantum RL must learn from sparse measurements, adapt across changing systems and conditions, and balance competing performance and experimental constraints.

Abstract

from arXiv · show

Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.

I. INTRODUCTION

Reinforcement learning offers an adaptive, model-free framework for addressing quantum-technology challenges through interaction with quantum devices. This review motivates RL for quantum systems, introduces its core agent–environment loop, and surveys experimental applications while emphasizing quantum-specific constraints.

  • Quantum-technology challenges span noise-resilient state preparation, high-fidelity gate control, circuit compilation, feedback control, and error correction.
  • RL agents interact with an environment by selecting actions, receiving rewards, and learning strategies that maximize expected cumulative reward.
  • Model-free RL learns directly from the physical system without requiring an explicit model, while deep RL can adapt and identify effective features.
  • Quantum measurements are stochastic, making quantum systems a natural match for RL’s exploration and control of stochastic environments.
  • Recent experiments applied RL to spin-squeezed-state preparation, entangling-gate optimization, circuit construction, feedback control, error correction, and photonic quantum speedups.
  • Directly transferring classical machine-learning ideas is insufficient because quantum states cannot be directly observed and projective measurements irreversibly collapse wavefunctions.

III. REINFORCEMENT LEARNING FOR QUANTUM SYSTEMS

The RL framework for quantum systems formalizes agents, environments, states, observations, actions, rewards, and returns while accounting for restricted and probabilistic quantum measurements. These restrictions make learning from observations and designing experimentally useful reward structures central concerns.

  • Quantum systems require special RL design because wavefunctions are inaccessible, measurements are uncertain and discrete, and projective measurements collapse the state.
  • An RL environment may be a simulator or experiment with states, dynamical laws, and transition probabilities that can be deterministic or stochastic.
  • Quantum RL agents may receive observations such as observable expectation values or error-correction syndromes rather than complete states.
  • Quantum actions include fields, gates, unitary operations, and measurements, with measurements treated as actions because they change the environment state.
  • Rewards can depend on current and successive states and may be assigned at episode end from fidelity, entanglement, or observables, or immediately from stabilizer measurements.
  • The return sums rewards across an episode, and discounted returns use γ to control the relative weight of future rewards.

2. Policy, Value, and Action-Value (Q-) Functions

RL represents action selection with policies, evaluates states through value functions, and uses Q-functions to assess state-action choices. These quantities support sequential decision-making in Markov environments, including partially observable quantum systems and settings where measurements constrain feedback.

  • Policy: A policy π(a|s) specifies the probability of selecting action a in state s, balancing exploitation of known returns with exploration of potentially better actions.Stochastic policies also accommodate uncertainty and can explore new actions.
  • Value functions: The value function vπ(s) gives the expected return from state s under policy π, while the Bellman equation defines it implicitly through transition dynamics and future values.The expectation includes both environment transitions and the policy.
  • Action-value functions: The Q-function Qπ(s,a) resolves value over actions, enabling a policy π′(s)=argmax_aQπ(s,a) that is at least as good as the original policy under stated conditions.Unlike the state value function, Qπ(s,a) can directly identify an action choice.
  • Markov decision processes: RL models environments as Markov decision processes in which the next state depends on the current state and action, producing trajectories of state-action-reward tuples.Episodes begin at an initial state and end at a terminal state before reset; infinite-horizon tasks instead take T→∞.
  • Objective: The RL objective maximizes the expected total return over trajectories generated jointly by the policy and environment transition probabilities.The expectation supports stochastic optimization and can capture time-nonlocal correlations within an episode.
  • Quantum environments: Quantum RL commonly operates under partial observability because quantum states cannot be measured directly, while strong measurements cause collapse and typically restrict rewards to the episode’s final step.Ancilla qubits can help mitigate wave-function-collapse problems by extending the environment state.

B. Deep Reinforcement Learning

Deep RL uses variational function approximators and neural architectures to handle large or continuous state spaces, while algorithm design alternates data collection with policy improvement. Architecture choices can encode structure and symmetry, but approximation may weaken convergence guarantees and experimental deployment remains challenging.

  • Function approximation: Exponential or continuous state spaces make exhaustive visitation infeasible, motivating variational approximations to policies, value functions, and Q-functions.These approximations can interpolate and extrapolate to unseen states when their inputs support continuous observations.
  • Function approximation: The ansatz must preserve the defining properties of its target, and its choice affects learning by incorporating prior structure such as periodicity or a suitable basis.Selecting an ansatz is analogous to choosing a basis before truncating an approximation.
  • Neural architectures: Policy networks output action probabilities for discrete actions or learn state-dependent distribution parameters for continuous actions, while Q-networks and value networks estimate action-resolved or scalar values.For continuous actions, a common choice is a normal distribution parameterized by μ(s) and σ²(s).
  • Neural architectures: Neural architectures can exploit local structure with convolutional layers and encode symmetries through invariant value functions or equivariant policy and Q-function outputs.A permutation-equivariant policy was constructed with a transformer for multiqubit-state disentangling.
  • Limitations: Learning efficient quantum-state representations from binary observations remains an open question relevant to applying deep RL in experimental setups.Embedding layers would need to reconstruct the state as far as information-theoretic constraints permit.
  • Training: RL training alternates policy evaluation, which generates observations and rewards, with policy improvement, which updates the policy from collected data.The frequency of alternation shapes algorithm behavior.
  • Limitations: Function approximation is necessary for challenging state or observation spaces but can compromise convergence properties, whose strongest guarantees typically concern tabular or linear methods.This creates a trade-off between expressive power and theoretical convergence guarantees.
  • Policy-gradient methods: Policy-gradient methods optimize a parameterized policy using estimated objective gradients, but gradient variance can be reduced by reinforcing actions with future rather than past returns.Entropy regularization can prevent convergence toward a delta-function policy when multiple actions should remain likely.

2. Value Function Methods

Value-function methods choose actions by estimating expected returns, with Q-learning providing an iterative route to optimal action values and deep variants extending this approach with neural networks. These methods require exploration, replay-based updates, and safeguards against target instability and overestimation.

  • Exploration: Exploration can soften greedy action selection with ε-randomness, Boltzmann sampling, temporally correlated noise, or parametric network noise.These alternatives encourage actions beyond the current argmax choice.
  • Q-learning: Q-learning iteratively updates action values from sampled transitions and converges to the optimal Q-function when states are visited arbitrarily often.The update uses a step size and Bellman error; the convergence statement is asymptotic and assumes sufficient visitation.
  • Deep Q-learning: Deep Q-learning approximates the Q-function with a neural network trained by regression on transitions sampled from a replay buffer.Stochastic gradient descent uses independently sampled batches, forming the core idea behind DQN.
  • Practical limitations and extensions: Q-learning faces running-target instability and max-induced Q-value bias, motivating target networks and Double Q-learning; DDPG and TD3 extend value-based methods to continuous actions.Strictly negative rewards can also encourage shorter protocols by penalizing episode steps.

3. Other Related Algorithms

Related algorithmic choices span evolutionary search, planning, projective simulation, and model-based or off-policy methods. The central practical trade-off is between sample efficiency and wall-clock runtime, which depends on whether data come from experiments or simulators.

  • Evolutionary methods: Evolutionary methods search with populations undergoing mutation, crossover, and selection, rather than selecting actions from sequential agent–environment observations.Genetic algorithms are presented as an important subclass of evolutionary methods.
  • Search algorithms: Search algorithms such as MCTS, Beam Search, and A∗ can explore large state spaces and generate data for policy or Q-function pretraining.AlphaZero is identified as a prominent example combining MCTS with policy training.
  • Model-free and model-based methods: Model-free methods suit black-box experimental systems, whereas differentiable simulations can enable gradient-based training with drastically fewer samples.The latter approach takes gradients of overall return through simulated dynamics.
  • On-policy and off-policy learning: Off-policy algorithms reuse old and new data, allowing data collection and policy improvement to proceed asynchronously when measurements are scarce or slow.Q-learning is the example given for transition-based off-policy learning.
  • Choosing algorithms: Quantum experiments prioritize sample efficiency, favoring off-policy methods such as Deep Q-Learning when environmental data are costly, while simulator training can favor Policy Gradient and Actor-Critic methods.Figure 6 presents the efficiency–runtime relation qualitatively, with more sample-efficient methods typically requiring more wall-clock time.

IV. REINFORCEMENT LEARNING & QUANTUM OPTIMAL CONTROL

RL complements quantum optimal control through model-free, adaptive, interactive decision-making that can handle uncertainty and measurement noise. Its applications include quantum state preparation, where an experimentally demonstrated agent prepared a Dicke state rapidly by exploiting excited-level dynamics, while partial observability and computational cost remain important constraints.

  • RL and optimal control: Unlike conventional optimal control, model-free RL does not require derivatives of the control cost or a detailed quantum-dynamics model.This makes RL applicable when Hamiltonians, noise sources, or dissipative dynamics are unknown or imperfectly characterized.
  • RL and optimal control: RL agents can handle stochastic dynamics, discontinuous rewards, and probabilistic quantum measurements, yielding policies that may be robust to perturbations.The review attributes this to learning in expectation over policy and environmental dynamics.
  • RL and optimal control: After training, RL policies can generalize to unseen initial states and self-correct noise-induced errors without re-optimizing the environment.This learned-object advantage distinguishes RL from control procedures that require re-optimization after environmental changes.
  • Quantum state preparation: Quantum state preparation maps control parameters to actions and fidelity-related quantities to rewards, but state information may require tomography or partially observable formulations.Quantum states are not directly measurable, making partial observability a central challenge for RL applications.
  • Quantum state preparation: Fidelity F ≈0.99 was achieved for a Dicke state in |c2|τ = 15.5, versus approximately 600 for a linear adiabatic sweep and 350 for a locally adiabatic nonlinear sweep.The experiment used a spinor 87Rb Bose–Einstein condensate across a quantum phase transition.
  • Quantum state preparation: The learned protocol exploits excited-level dynamics through a nonadiabatic sequence that crosses the transition twice and uses controlled destructive interference to attenuate excitations.The review identifies three interpretable stages linked to the Hamiltonian’s excitation spectrum.

A. Few-Body Systems

In few-body quantum systems, RL has been applied to state preparation, control under dissipation and noise, measurement-driven tasks, and thermodynamic optimization. Across these settings, studies report adaptive protocols, robustness, and generalization to varied targets or operating conditions.

  • State preparation: RL prepares arbitrary single- and two-qubit states using representations ranging from Bloch-sphere coordinates to full state vectors and density matrices.Applications include Λ-systems, quantum dots, NV centers, and noisy multilevel platforms.
  • Robust control: RL control remains effective under dissipation, with reported protocols competitive with Raman and adiabatic schemes or capable of identifying restricted-control strategies.These studies address Lindblad dynamics, fixed couplings, and time-varying detunings.
  • Robust control: RL improves robustness to stochastic perturbations and unmodeled disturbances in semiconductor quantum dots and three-level feedback-control systems.Reported agents outperform gradient-based approaches or simple feedback strategies in the cited settings.
  • State preparation: DQN agents find protocols for preparing Bell states and arbitrary entangled states that generalize previously found human solutions.The setting includes projective measurements and drift terms in coupled quantum-dot systems.
  • Thermodynamic optimization: RL frameworks identify Pareto-optimal thermodynamic cycles because power, efficiency, and power fluctuations cannot be optimized simultaneously.Soft Actor-Critic is used to seek trade-offs in driven quantum heat engines, including quantum-dot platforms.
  • Methodological comparison: Energy-optimized deep RL can perform worse than gradient-based methods, underscoring the need to compare RL pulse engineering with traditional optimization.The comparison concerns jointly minimizing energetic cost and quantum-unitary-gate infidelity.

B. Many-Body Systems

Many-body quantum control is difficult because state spaces and preparation complexity grow rapidly with system size. Despite this, RL has produced faster protocols, adaptive control strategies, and substantial efficiency gains across spin chains, cooling, squeezing, and circuit-based preparation.

  • Challenges and scope: Many-body state preparation faces exponentially large state and configuration spaces, with fidelity generally decreasing as the number of degrees of freedom increases.This complexity motivates RL studies of cooling, spin-squeezing, and ground-state preparation.
  • Cooling: RL has been applied to cooling atomic gases, coupled oscillators, and mechanical resonators using direct or partial observations and feedback actions.These approaches target atom-number control, thermal-ground-state cooling, or reduction of vibrational energy.
  • Spin-squeezed states: PPO outperforms adiabatic dynamics for twin-Fock-state preparation in a many-body spin model.The agent uses selected local observables and controls the amplitude of a nonlinear zz interaction.
  • Spin chains: RL agents can become trapped in local optima when preparing ferromagnetic Ising-model ground states with bang-bang controls.The early Q-learning study used the control field and time as the RL state.
  • Spin chains: DQN finds faster high-fidelity state-transfer protocols in one-dimensional XY spin chains than previously known protocols.The agent controls the transverse z-field using wave-function components as observations.
  • Spin chains: Comparative studies report that deep RL adaptively reduces the number of control steps needed to find efficient solutions relative to conventional optimization methods.The comparisons include Q-learning, deep Q-learning, policy gradients, stochastic gradient descent, and Krotov optimization.
  • Advanced many-body control: RL frameworks address more complex tasks including nonergodicity control, circuit-based preparation, limited-measurement state preparation, and meta-learning of quantum-circuit training schedules.Reported applications include enhanced robustness, two-orders-of-magnitude CNOT reductions for N ≥12, and generalization from 3 to 5 qubits.

VI. PULSE-SHAPE ENGINEERING FOR QUANTUM GATE IMPLEMENTATION

RL pulse-shape engineering designs quantum gates directly through interaction with hardware, avoiding dependence on accurate Hamiltonian and noise models. Experimental studies report faster or more accurate gates and robustness over time, while also exposing the measurement and training challenges of hardware-in-the-loop optimization.

  • Motivation: Quantum gates are implemented through hardware-level Hamiltonian evolution driven by optimized microwave or laser pulse shapes.Gate synthesis therefore requires control of physical waveforms rather than only selecting abstract unitary operations.
  • Evaluation: Average gate fidelity is commonly estimated by sampling initial states from the Haar measure, a unitary k-design, or an orthonormal basis.The exact Haar integral is difficult to evaluate in practice.
  • Model-free optimization: Model-free RL can learn platform-specific imperfections, including leakage, parameter drift, and errors beyond theoretical approximations, without accurate noise models.This enables pulse optimization using measurement data from the controlled device.
  • Experimental applications: A superconducting-qubit RL agent synthesizes RX(π/2) rotations up to 3 times faster and ZX(−π/2) gates up to 2 times faster than stated reference implementations.The ZX gates reach FZX > 99.5% and both gate types show calibration-free performance over a few weeks.
  • Experimental integration: Hardware-in-the-loop training addresses limited observability through repeated measurements, batching, and runtime interaction between the agent and the quantum processor.One implementation uses simplified state tomography with 256 intermediate repetitions and 1024 final-state repetitions.
  • Two-qubit gates: Infidelity decreases from (1.05 ± 0.07) × 10^-2 for the theory protocol to (4.40 ± 0.58) × 10^-3 for the RL-optimized ZX(−π/2) pulse.The experiment compares three-channel cross-resonance waveforms on superconducting qubit hardware.
  • Two-qubit gates: RL-designed gates retain robust performance over several weeks and outperform the state-of-the-art theory pulse in randomized benchmarking comparisons.The cited experiment evaluates repeated ZX gates and interleaved CNOT sequences after training.
  • Single-qubit gates: Single-qubit pulse optimization can use experimental in-phase and quadrature signals directly, producing pulses up to twice as fast while maintaining equivalent fidelity and leakage rates.Intermediate measured signals provide rewards when gate fidelity is experimentally inaccessible.

B. Two-Qubit Gates

RL is applied to two-qubit gate synthesis because these systems require optimization across high-dimensional control landscapes while balancing fidelity, speed, and robustness. Reviewed approaches span platform-specific and general frameworks, with applications extending from pulse design to disentangling and state preparation.

  • Two-qubit gate synthesis must jointly address fidelity, speed, robustness, and high-dimensional control complexity.
  • Superconducting Transmon Systems: TRPO reduced average gate error by two orders of magnitude over stochastic-gradient solutions in noisy two- and three-state transmon models.
  • Superconducting Transmon Systems: DDPG discovered cross-resonance ZX and CNOT controls without prior physical-system knowledge, surpassing direct and echoed schemes in fidelity and execution time.
  • General Gate Synthesis Frameworks: Platform-agnostic methods use gate representations, discrete or continuous control actions, and fidelity-based rewards to synthesize CNOT, Hadamard, T, and entangling gates.
  • The review positions RL as a framework for sequential circuit decisions that can be trained in simulation and tested through interaction with quantum hardware.

A. Entanglement Control

RL is used to control and engineer entanglement across dissipative many-body systems, quantum circuits, and photonic experiments. The reviewed results include partial-information feedback, circuit architecture discovery, and improved optimization for variational and combinatorial tasks.

  • Entanglement control addresses decoherence, measurement backaction, and the complexity of many-body dynamics.
  • Continuous Control Approaches: DQN control of a dissipative Rabi model increases entanglement across parameter regimes while remaining robust against dissipation.
  • PPO disentanglers identify measurement patterns in random Clifford circuits, suggesting fewer measurements than theoretical predictions.
  • RL feedback control successfully masters individual control of up to 128 qubits using partial information and negative bond-entanglement rewards.
  • Projective simulation autonomously designs photonic experiments for high-dimensional entangled multiphoton states and rediscovered advanced techniques not explicitly programmed.
  • RL methods support VQE and architecture search through policy optimization, feedback control, and learned gate structures across noisy quantum tasks.
  • Combinatorial Optimization Applications: Max-Cut: RL solves or improves combinatorial quantum optimization, including Max-Cut, Maximum Clique, and Minimum Vertex Cover, with generalization and higher approximation ratios reported.
  • Circuit Architecture Discovery: Circuit-discovery methods find compact circuits under hardware or gate-count constraints, including fewer than three entangling gates for systems up to 16 qubits.

C. Quantum Circuit Compilation and Architecture Search

RL-based compilation and architecture search explore circuit structures, unitary synthesis, routing, and graph representations under hardware and resource constraints. The surveyed methods report reductions in gate counts, circuit depth, and synthesis costs while often generalizing across circuits or platforms.

  • Architecture-search agents discover ansätze and minimal circuits for state preparation, diagonalization, and Bell-state synthesis using structured circuit representations.
  • Gumbel AlphaZero performs exact Clifford+T unitary synthesis for circuits up to five qubits using tree search with gate-placement penalties.
  • AlphaTensor-Quantum minimizes T-count and finds the best human-designed solutions on arithmetic benchmarks relevant to Shor’s algorithm and quantum chemistry.
  • On 12-qubit random circuits, a convolutional agent achieved 27% average depth reduction and 15% average gate-count reduction, with demonstrations up to 50 qubits.
  • Model-based RL reduced SWAP usage by up to 62.5% for nearest-neighbor architecture constraints.
  • Graph-Theoretic and Tree-Search Approaches: ZX-diagram RL uses graph neural networks and rewrite rules to reduce node counts and gate counts while maintaining computational competitiveness.
  • Deep Q-learning with A* search improves compiled circuit depth and inference time using hardware-specific gate sets.
  • RL transpilation methods produce near-optimal or hardware-adapted circuits with reduced two-qubit resources, fewer gates, and fewer ancillary qubits.

VIII. QUANTUM FEEDBACK CONTROL

Quantum feedback control uses measurements from the physical system to choose actions in real time, but feedback strategies scale exponentially with measurement histories and must operate within short coherence-time windows. Experiments demonstrate RL agents controlling qubits, cavities, oscillators, and open quantum systems.

  • Physical quantum feedback differs from optimization feedback because the agent observes real measurements from the quantum system.
  • A superconducting-qubit experiment achieved a response time below one microsecond using a deep neural-network feedback agent.
  • A non-Markovian feedback strategy must specify actions for M^N measurement histories after N steps with M outcomes, unlike open-loop control.
  • Real-time feedback is experimentally demanding because actions may need to be selected much faster than a qubit’s coherence time.
  • FPGA deployment and simultaneous network evaluation and measurement-data extraction addressed the latency constraints of superconducting-qubit feedback.
  • The experimental agent initialized thermally excited qubits and qutrits in their ground state through measurement-and-pulse sequences.
  • PPO and soft actor-critic agents learn feedback strategies for oscillator state preparation, cavity Fock-state stabilization, and measurement-powered cooling.

IX. QUANTUM ERROR CORRECTION

Quantum error correction is difficult because device-specific error channels can be unknown or time-varying, while reliable protection requires accurate decoding and control. The review highlights RL methods that adapt QEC protocols and code designs directly to experimental behavior, including demonstrations improving logical-qubit protection.

  • Motivation: Unknown and drifting error channels make theory-only QEC solutions insufficient for achieving the accuracy needed on real devices.RL can adapt to platform-specific noise and control imperfections.
  • Experimental QEC: RL-enhanced GKP correction increased the logical qubit lifetime by 2.27±0.07 while learning 45 continuous QEC-cycle parameters.The agent first corrected large initial angle errors, then fine-tuned parameters; its solution used slower gates and reduced intermediate cavity photon number.
  • Quantum Decoders: Surface-code RL improved the logical error rate by a factor of 2.4, or 3.5 with decoder steering, and scaled to d = 15 with 40,000 gate-control parameters.Training uses error-detection events as a surrogate objective, enabling simultaneous QEC and real-time gate recalibration.
  • Quantum Decoders: RL decoders approached MWPM for toric-code bit-flip errors through d = 7 and outperformed it for depolarizing noise through d = 9.The depolarizing-noise decoder was close to optimal for uncorrelated noise but suboptimal for biased noise models.
  • Discovery of Error Correcting Codes: RL also optimized error-correcting code structures, finding near-optimal surface-code memories for up to 70 data qubits and reducing physical-qubit overhead by an order of magnitude for weight-6 codes.These approaches modify code families or Tanner graphs while preserving target logical-error or code-distance properties.

X. QUANTUM REINFORCEMENT LEARNING

Quantum reinforcement learning makes the agent or its communication with the environment quantum mechanical, extending classical RL applied to quantum systems. Experimental results show faster learning through quantum communication, while the review notes that practical advantages of fully quantum agents remain uncertain.

  • Definition and Scope: Quantum reinforcement learning includes quantum agents interacting with classical environments and quantum agents interacting with quantum environments.Earlier sections primarily considered classical RL agents acting on environments describing quantum transition dynamics.
  • Experimental Demonstration: The experimentally implemented hybrid agent learned a close-to-optimal strategy faster and more efficiently than a classical agent using a tunable nanophotonic processor.The platform supports both quantum and classical information transfer.
  • Open Challenges: Whether QRL offers a practical advantage over classical RL remains unknown because universal quantum computing is still at an early stage.The review also notes familiar quantum-circuit optimization difficulties such as barren plateaus and vanishing gradients.
  • Quantum-Enhanced Learning: Quantum communication can accelerate learning by alternating quantum episodes, which perform Grover-like amplitude amplification, with classical episodes that produce rewards and update the policy.Quantum episodes do not produce rewards; classical episodes test performance and provide feedback.
  • Experimental Demonstration: The purely quantum strategy peaks at an optimal training episode and then degrades, motivating a hybrid strategy that switches to classical training after η=0.39.The combined strategy outperforms both purely quantum and purely classical strategies and exhibits a quadratic speedup in learning time.

XI. QUANTUM METROLOGY: PARAMETER ESTIMATION & SENSING

Quantum metrology uses RL to optimize probe-state preparation, adaptive measurements, and feedback for parameter estimation and sensing. The review reports promising theoretical and experimental results, but emphasizes that compelling beyond-state-of-the-art experimental demonstrations remain absent.

  • Parameter Estimation: Quantum metrology seeks probe states and adaptive controls that improve parameter-estimation precision beyond the standard quantum limit toward Heisenberg scaling.Relevant probes include squeezed, GHZ, and critical many-body states, while QFI quantifies achievable precision.
  • Limitations and Outlook: A compelling experiment demonstrating RL for quantum parameter estimation, metrology, or sensing beyond the current state of the art has yet to be realized.The review nevertheless describes a growing theoretical literature on adaptive strategies, probe states, and measurement feedback.
  • Parameter Estimation: RL provides a framework for updating measurement settings from prior outcomes and optimizing feedback strategies for quantum parameter estimation.Applications include model-aware Bayesian estimation, device-parameter recalibration, frequency estimation, and noise-aware control.
  • Parameter Estimation: RL-designed protocols accelerate one-axis-twisting entanglement generation, satisfy Heisenberg scaling, and are implementable with current experimental capabilities.The protocols use QFI as their figure of merit.
  • Quantum Sensing: RL-optimized nonlinear pulses mitigate decoherence and improve quantum-chaotic sensor precision over periodic protocols, while an optical-lattice design achieves a 20-fold improvement over traditional Bragg interferometry.The latter protocol optimizes sensitivity to rotational signals through end-to-end lattice shaking.
  • Quantum Sensing: QRL protocols have been reported to saturate finite quantum speed limits and achieve Heisenberg and super-Heisenberg scaling in critical many-body sensing.These results are presented as theoretical evidence for QRL-based state preparation in quantum-critical sensing.

XII. OPEN CHALLENGES & FUTURE DIRECTIONS

The review identifies open challenges spanning physics-informed objectives, sparse-observation learning, rapid adaptation, specialized architectures, scalability, real-time deployment, interpretability, and integration with emerging AI capabilities.

  • Reward functions: Physics-informed reward design remains largely unexplored, despite its importance for incorporating physical constraints and symmetries into quantum RL objectives.The review frames reward formulation as central to RL success in quantum technologies.
  • Experimental learning: Learning from sparse quantum measurements while maintaining sample efficiency is necessary for transferring RL policies from simulations to experimental feedback control.Fully observable simulations may produce policies that require modification before real-world deployment.
  • Adaptability and optimization: Practical deployment requires rapid adaptation to new Hamiltonians, target states, and experimental conditions without extensive retraining, while balancing fidelity, decoherence resilience, and runtime.The review identifies multi-objective optimization and meta-learning as promising directions.
  • RL architectures: Quantum environments motivate memory-capable, model-based, and multi-agent RL architectures because of imperfect Markovianity, predictive-control opportunities, and distributed hardware.Transformer-based memory, tree search with quantum heuristics, and coordinated multi-agent systems are highlighted as directions.
  • Scalability: The exponential growth of Hilbert-space dimension complicates classical simulators for RL, motivating architectures that exploit quantum many-body structure and physical symmetries.Equivariant neural networks are identified as one route toward addressing state-space growth.
  • Deployment and scientific discovery: Real-time quantum RL is constrained by communication latency and finite feedback bandwidth, while interpretable policies and AI integration are proposed as routes toward discovery and more accessible experimentation.The review also discusses multimodal learning, LLM-assisted reasoning and coding, and the substantial cost and hardware requirements of current leading LLMs.
Loading 2601.18953v1…