Source-linked AI summary
Quantum Compiling by Deep Reinforcement Learning
Lorenzo Moro, Matteo G. A. Paris, Marcello Restelli, Enrico Prati
TL;DR
A key challenge in quantum compiling is efficiently returning gate sequences, since poor scaling of execution time and sequence length with accuracy can erase quantum advantage. The authors train deep reinforcement learning agents to learn general strategies for approximating single-qubit unitary matrices and encode those strategies in neural networks. The compiler empirically achieves sequence lengths scaling as O(log^1.25(1/δ)) and returns the approximating sequence in a fraction of a second after training, without guaranteeing a suitable solution.
Problem
A key challenge in quantum compiling is efficiently returning gate sequences, since poor scaling of execution time and sequence length with accuracy can erase quantum advantage.
Source: Discussion · p. 7Method
The authors train deep reinforcement learning agents to learn general strategies for approximating single-qubit unitary matrices and encode those strategies in neural networks.
Source: Discussion · p. 7Results
The compiler empirically achieves sequence lengths scaling as O(log^1.25(1/δ)) and returns the approximating sequence in a fraction of a second after training, without guaranteeing a suitable solution.
Source: Discussion · p. 7Takeaways & Limitations
After one-time precompilation, the trained policy can return gate sequences in minimal time, enabling online quantum compilation within the reported scope.
Source: Discussion · p. 7Tests of hardware-specialized environments and extension to n-qubit systems remain future work.
Source: Discussion · p. 7
Abstract
from arXiv · showhide
The architecture of circuital quantum computers requires computing layers devoted to compiling high-level quantum algorithms into lower-level circuits of quantum gates. The general problem of quantum compiling is to approximate any unitary transformation that describes the quantum computation, as a sequence of elements selected from a finite base of universal quantum gates. The existence of an approximating sequence of one qubit quantum gates is guaranteed by the Solovay-Kitaev theorem, which implies sub-optimal algorithms to establish it explicitly. Since a unitary transformation may require significantly different gate sequences, depending on the base considered, such a problem is of great complexity and does not admit an efficient approximating algorithm. Therefore, traditional approaches are time-consuming tasks, unsuitable to be employed during quantum computation. We exploit the deep reinforcement learning method as an alternative strategy, which has a significantly different trade-off between search time and exploitation time. Deep reinforcement learning allows creating single-qubit operations in real time, after an arbitrary long training period during which a strategy for creating sequences to approximate unitary operators is built. The deep reinforcement learning based compiling method allows for fast computation times, which could in principle be exploited for real-time quantum compiling.
Results
The paper frames quantum compiling as a sequential decision problem: an agent builds a gate sequence from a chosen base to approximate a target unitary. It trains agents for fixed-target and Haar-target tasks using rotation and discrete gate bases, with reward designs and algorithms suited to each setting.
Deep reinforcement learning
- Deep reinforcement learning learns a policy through agent–environment interactions, with training performed once before the policy is exploited for fast decisions.The agent selects gates as actions and receives rewards from the circuit environment; the policy is encoded in a deep neural network.Source: Deep reinforcement learning · p. 2, Figure 1 · p. 3
Deep reinforcement learning as quantum compiler
- For small-rotation bases, the agent incrementally builds a circuit from gates in the chosen base, using the real and imaginary parts of its observation as input.The observation encodes the current gate composition and target; the tolerance is fixed and not provided to the agent.Source: Deep reinforcement learning as quantum… · p. 2
Quantum compiling by the HRC efficiently universal base of gates
- The small-rotation task uses a dense reward, whereas the HRC task uses sparse binary rewards with Hindsight Experience Replay and Deep Q-Learning.The paper says the dense reward is adequate only for small rotations; sparse rewards provide little information about how to find a solution.Source: Deep reinforcement learning as quantum… · p. 3, Training neural networks for… · p. 4, Quantum compiling by the HRC… · p. 4
Training neural networks for approximating a single-qubit gate
- After about 10^4 episodes, the fixed-target agent found a 76-gate circuit within tolerance, improving its solution over training.A random agent found no solution in 10^4 episodes under the same tolerance and episode-length thresholds.Source: Training neural networks for… · pp. 3–4
Quantum compiling by rotations operators
- On a validation set of 106 Haar unitary targets, the agent approximated more than 96% within the requested tolerance.Source: Quantum compiling by rotations operators · p. 4
Quantum compiling by the HRC efficiently universal base of gates
- More than 95% of 10^6 Haar-random targets were solved with the HRC base, using on average less than 36 gates.The task used a 0.99 average gate fidelity tolerance and a 130-gate episode limit.Source: Quantum compiling by the HRC… · p. 4
Discussion
The discussion presents deep reinforcement learning as a flexible compiler strategy that learns gate sequences for different bases and can return them quickly after training. Its reported scope includes single-qubit compilation, an early two-qubit prototype, and possible hardware-specific extensions.
Discussion
- The authors characterize the approach as basis-independent and flexible, contrasting it with decomposition methods limited to particular bases or qubit counts.Y-Z-Y decomposition is described as limited to y and z rotations, while KAK is limited to two qubits and specified gates.Source: Discussion · pp. 4, 7
- Figure 2 tracks two representative state trajectories and, during training, solution frequency and average sequence length; the agent learns after about 10^4 episodes.The agent continues improving the solution over time.Source: Figure 2 · p. 5
- O(log1.25(1/δ)) sequence scaling was measured empirically on a specific task, while trained networks return sequences in a fraction of a second.The reported execution rate is 5.4·10^-4 s per time-step; suitable solutions are not guaranteed.Source: Discussion · p. 7, Figure 4 · p. 7
- The method uses two gate bases and reward designs tailored to their differing characteristics, with Haar-distributed targets for broad coverage.The authors state that additional target-distribution information may allow higher tolerance without tuning network architecture or RL hyperparameters.Source: Discussion · p. 7
- An early prototype applies the compiler to two-qubit gates, with performance described as consistent with one-qubit gates at 0.99 AGF.The paper identifies extension to n-qubits as future work.Source: Discussion · p. 7
- Hardware-specific constraints can be incorporated into the environment, while testing such specialization and extending the approach to n-qubits remain future work.Source: Discussion · p. 7
Methods
The methods use Haar-random unitary targets, hindsight relabeling to address sparse rewards, and a compact neural-network architecture. They also specify the fidelity measure and implementation environment.
Generation of Haar random unitary matrices
- Haar-random unitaries provide an unbiased training-target dataset, with selection strategy chosen according to the gate set of interest.The probability of selecting a unitary from a region is proportional to that region’s volume in unitary space.Source: Generation of Haar random unitary… · pp. 7–8
Learning by Hindsight Experience Replay (HER)
- Hindsight Experience Replay (HER) relabels completed-episode experiences with new goals, using randomly selected states from that episode, to help learn the primary task from failed attempts.The strategy selects k-percent of states from the same episode; HER addresses sparse rewards by training toward alternative targets.Source: Learning by Hindsight Experience… · p. 8
Average gate fidelity
- Average gate fidelity is defined through an integral over the state space using a Haar measure.Source: Average gate fidelity · p. 8
Neural network architectures
- The authors selected two hidden layers of 128 neurons as the smallest architecture they found sufficient for the tasks, rather than optimizing architecture.Their architecture search gradually increased network depth and width from one hidden layer with few neurons.Source: Neural network architectures · p. 8
Software and Hardware
- The implementation used Python, Stable Baselines for the PPO agent, and GNU parallel for most simulations on an Intel Xeon W-2195 and Nvidia GV100.Source: Software and Hardware · p. 8
Extended Figures and Tables
The extended tables list hyperparameters and values for fixed-target, rotation-operator, and HRC compilation, followed by validation performance for PPO and DQL agents.
Extended Figures and Tables
- Table 1 lists the hyperparameters and their values for the fixed-target problem.Source: Table 1 · p. 11
- Table 2 lists the hyperparameters and their values for the PPO agent compiling rotation operators.Source: Table 2 · p. 11
- Table 3 lists the hyperparameters and their values for a DQL agent in the HRC problem.Source: Table 3 · p. 11
- Table 4 reports PPO and DQL performance after training on a validation set of 106 Haar unitary targets, with columns for the 95th percentiles of solved-sequence lengths.The caption repeats “95th” for both percentile columns.Source: Table 4 · p. 11