PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Reinforcement Learning for Quantum Gate Control

Zheng An, D. L. Zhou

arXiv:1902.08418v2quant-ph

TL;DR

High-fidelity quantum gates are difficult to implement efficiently with limited control resources. The paper applies deep reinforcement learning to discrete control of Hadamard and CNOT gates. It reports that the agent learns effective control schemes, including better CNOT results than GRAPE after T = 0.9.

  • Problem

    Producing high-fidelity gates with limited control resources requires precise choices of control signals, while exhaustive search over d^N control sequences is infeasible for large N.

    Source: §I · p. 1, §II · p. 2

  • Method

    The paper maps quantum gate control to deep reinforcement learning, using an agent to select discrete, sequential controls for target gates.

    Source: §I · p. 1, §V · p. 5

  • Results

    The agent learns effective schemes for approximating target gates, and for CNOT control its results are better than GRAPE's after T = 0.9.

    Source: §V · p. 5, end · p. 5

  • Takeaways & Limitations

    The authors conclude that deep RL is suited to discrete-action quantum control and can balance exploration and exploitation while learning control schemes.

    Source: §V · p. 5

    Extending reinforcement learning to multi-qubit control remains challenging because the control space grows exponentially with qubit number.

    Source: §V · p. 5

Abstract

from arXiv · show

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

I. INTRODUCTION

The paper addresses the difficulty of producing high-fidelity gates with limited control resources and proposes deep reinforcement learning for discrete, sequential gate control. Its bang-bang model fixes the total time, selects controls from a finite set, and seeks a sequence that maximizes gate fidelity.

I. INTRODUCTION

  • High-quality gates are difficult to obtain under limited control resources, motivating a deep-RL approach to quantum gate control.The paper targets a Hadamard gate and a CNOT gate using optimal discrete, sequential controls.Source: §I · p. 1

II. BANG-BANG CONTROL MODEL TO IMPLEMENT QUANTUM GATES

  • The bang-bang protocol divides fixed total time T into N equal intervals and holds a selected control vector constant in each interval.Each vector is chosen from a finite set of d possible choices.Source: §II · p. 2
  • The control objective is to choose the interval-wise parameter vectors to maximize fidelity between the resulting unitary U(T) and target gate Uf.Fidelity is 1 if and only if U(T) equals Uf up to a phase factor.Source: §II · p. 2
  • Because the number of possible parameter sequences is d^N, exhaustive search is impossible for large N.Source: §II · p. 2

A. Hadamard gate

For the Hadamard-gate task, the paper uses a two-level system with a real control parameter and a bang-bang control set of two choices.

A. Hadamard gate

  • The Hadamard-gate task is modeled with a two-level system whose Hamiltonian is specified for the control problem.Source: A. Hadamard gate · p. 2
  • The model identifies σz and σx as Pauli matrices and ϵ(t) as a real control parameter.Source: A. Hadamard gate · p. 2
  • The bang-bang protocol uses d = 2 possible control values, A(ϵ) ∈ {±4}.Source: A. Hadamard gate · p. 2

B. CNOT gate

For the CNOT task, the paper specifies a four-component control vector and a finite set of 16 possible parameter-vector choices.

B. CNOT gate

  • The CNOT gate is one of the two target gates considered in the paper.Source: B. CNOT gate · p. 2
  • The CNOT control model uses a four-component parameter vector, ϵ(t) = (ϵ1(t), …, ϵ4(t)).Source: B. CNOT gate · p. 2
  • The bang-bang protocol takes d = 16 possible choices for the CNOT parameter vector.Source: B. CNOT gate · p. 2

III. DEEP REINFORCEMENT LEARNING METHODS

The paper applies deep reinforcement learning to approximately solve the fidelity-maximization problem in its bang-bang gate-control protocol. It reviews DDDQN and then combines the reinforcement-learning methods with the control protocol.

III. DEEP REINFORCEMENT LEARNING METHODS

  • The method applies deep reinforcement learning to approximately solve the fidelity-maximization problem in the bang-bang protocol.Source: §III · p. 2
  • The section reviews deep-RL concepts, with particular attention to the dueling double deep Q-learning neural network (DDDQN).Source: §III · p. 2
  • The approach combines the bang-bang control protocol with deep reinforcement-learning methods.Source: §III · p. 2

A. Reinforcement learning

Reinforcement learning frames control as an agent choosing actions in an environment and learning from delayed rewards. The agent seeks a policy that maximizes expected discounted return, with Q-learning iteratively improving that policy.

A. Reinforcement learning

  • An RL agent chooses actions in an environment to optimize performance using delayed scalar rewards.Source: A. Reinforcement learning · p. 2
  • A finite Markov decision process represents interaction as a sequence of states, actions, and rewards, with finite state, action, and reward sets.Source: A. Reinforcement learning · p. 3
  • A policy specifies action probabilities for each state, and the objective is to maximize expected discounted return under that policy.Source: A. Reinforcement learning · p. 3
  • Q-learning updates state-action values iteratively, selecting value-maximizing actions to improve a policy toward an optimal one.Source: A. Reinforcement learning · p. 3

B. Dueling Double Deep Q-learning

The paper applies Dueling Double Deep Q-learning (DDDQN), using evaluation and target networks to learn state-action values. Its dueling architecture combines state value and action advantage estimates, while training uses replayed experiences and periodic target-network updates.

B. Dueling Double Deep Q-learning

C. Quantum gate control with DDDQN

The DDDQN agent selects discrete control actions for quantum gate evolution and learns from the resulting unitary and reward. Its state uses unitary-matrix information, and the reward is delivered at the final step according to logarithmic infidelity.

C. Quantum gate control with DDDQN

IV. NUMERICAL RESULTS

The numerical section evaluates logarithmic infidelity for Hadamard and CNOT gate control using RL, GRAPE, DE, and GA. For the Hadamard gate, the methods agree below T = 0.8, while their relative performance differs at T = 0.9.

IV. NUMERICAL RESULTS

  • The numerical comparison evaluates logarithmic infidelity for Hadamard and CNOT gates using RL, GRAPE, differential evolution, and genetic algorithm.Source: §IV · p. 4
  • For the Hadamard gate, all four methods agree for T < 0.8; at T = 0.9, RL and DE agree, outperform GA, and trail GRAPE.Source: §IV · p. 4
  • The agent architecture uses an encoder and value and advantage networks, while training initializes two networks, selects actions, stores experiences, and samples prioritized replay minibatches.Source: Figure 2 · p. 4, §IV · p. 4

end

The results compare RL with GRAPE, DE, and GA for Hadamard and CNOT gate preparation across evolution times. RL reaches high fidelity for Hadamard preparation at T = 1.0 and outperforms GRAPE for CNOT after T = 0.9, while GRAPE requires fidelity gradients not readily accessible through experimental measurements.

end

  • At T = 1.0, RL and GRAPE achieve L < −3 (F > 99.9%) for Hadamard preparation, with GRAPE performing best among the four methods.RL and GRAPE infidelities abruptly decrease in the T = 0.9–1.0 region, possibly indicating the speed limit.Source: end · p. 4, Figure 3 · p. 5
  • For CNOT, RL outperforms GRAPE after T = 0.9, and at T = 1.1 all algorithms except RL get trapped.The comparison covers RL, GRAPE, DE, and GA; RL is described as maintaining good, robust performance in the dumped-landscape region.Source: Figure 4 · p. 5, end · p. 5
  • GRAPE allows the control field ϵ(t) to take any value in [−4, 4]; Hadamard and CNOT results use time steps N = 28 and N = 38, respectively.The Hadamard caption reports 400 optimization iterations and 100000 RL training episodes; the CNOT caption reports 5000 iterations and 150000 RL training episodes.Source: end · p. 5, Figure 3 · p. 5, Figure 4 · p. 5

V. CONCLUSION

The conclusion reports that deep RL robustly and efficiently implements arbitrary single- and two-qubit gates, while identifying exponential growth of the control space with qubit number as a challenge to extending the approach.

V. CONCLUSION

  • Numerical results show deep RL is robust and efficient for implementing arbitrary single- and two-qubit gates.The authors compare four algorithms on Hadamard and CNOT control tasks and report that the agent learns control schemes approximating the target gates.Source: §V · p. 5
  • Extending RL to multi-qubit control remains challenging because the control space grows exponentially as the number of qubits increases.The authors hope the approach will inspire further deep RL applications in quantum control.Source: §V · p. 5

Appendix A: Hyper-Parameters and Learning Curves

The appendix reports the DDDQN network settings and training-time comparison, and presents learning curves for Hadamard and CNOT gate control.

Appendix A: Hyper-Parameters and Learning Curves

  • The Q-value network uses four layers per subnetwork, and learning depends strongly on the neural-network layer count.The listed neuron settings are {600, 600, 600} for the decoder and {600, 600, 600, 600} for the advantage and value networks.Source: Appendix A · pp. 5–6
  • Two-qubit gate training takes 6 to 30 times longer than one-qubit training, while the RL agent’s required resources increase slowest among the algorithms discussed.Training time is summarized in Table II.Source: Appendix A · p. 6, Table II · p. 6
  • Figure 5 compares Hadamard-gate learning curves at T = 1 across architectures with different layer counts and neuron numbers.Source: Appendix A · p. 5, Figure 5 · p. 6
  • Figures 6 and 7 show learning curves for Hadamard at T = 1 and CNOT at T = 1.1, respectively.Figure 6 distinguishes instantaneous fidelity with red dots from the average fidelity of five agents with a blue line.Source: Figure 6 · p. 6, Figure 7 · p. 6

Related papers

Loading 1902.08418v2…