PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantum Long Short-Term Memory

Samuel Yen-Chi Chen, Shinjae Yoo, Yao-Lung L. Fang

arXiv:2009.01783v1quant-phcs.LG

TL;DR

Quantum machine learning had not previously investigated learning sequential or temporal data, despite LSTM’s established role in sequence modeling. The paper proposes QLSTM, a hybrid quantum-classical LSTM using variational quantum circuits. Under similar parameter counts, simulations show faster and more stable learning than classical LSTM, while the simulations assume a perfect quantum computer.

  • Problem

    Quantum machine learning had not previously investigated learning sequential or temporal data, despite LSTM’s established role in sequence modeling.

  • Method

    QLSTM is a hybrid quantum-classical LSTM that uses variational quantum circuits and iterative optimization to model temporal dependencies.

  • Results

    Under similar parameter counts, QLSTM learns more information and decreases its loss faster and more stably than classical LSTM.

  • Takeaways & Limitations

    The work paves the way toward using quantum circuits to model sequential data.

  • Takeaways & Limitations

    The numerical simulations assume a perfect quantum computer.

Abstract

from arXiv · show

Long short-term memory (LSTM) is a kind of recurrent neural networks (RNN) for sequence and temporal dependency data modeling and its effectiveness has been extensively established. In this work, we propose a hybrid quantum-classical model of LSTM, which we dub QLSTM. We demonstrate that the proposed model successfully learns several kinds of temporal data. In particular, we show that for certain testing cases, this quantum version of LSTM converges faster, or equivalently, reaches a better accuracy, than its classical counterpart. Due to the variational nature of our approach, the requirements on qubit counts and circuit depth are eased, and our work thus paves the way toward implementing machine learning algorithms for sequence modeling on noisy intermediate-scale quantum (NISQ) devices.

I. INTRODUCTION

The paper addresses sequential-data learning with quantum machine learning by introducing QLSTM, a hybrid quantum-classical LSTM built from variational quantum circuits. Simulations indicate faster and more stable learning than classical LSTM under similar parameter counts, while targeting NISQ-compatible implementation.

  • NISQ implementation is constrained by the difficulty of faithfully executing circuits with many qubits or long depth without quantum error correction.
  • Sequential-data learning had not been investigated in the quantum domain, motivating a quantum approach to temporal modeling.
  • QLSTM implements an LSTM with variational quantum circuits in a hybrid quantum-classical framework optimized iteratively.The framework uses quantum entanglement while retaining classical parameter optimization.
  • Under similar network-parameter counts, QLSTM learns faster and converges more stably than classical LSTM in numerical simulations.The QLSTM loss shows no peculiar spikes typical of LSTM loss functions.
  • The authors identify non-Markovian quantum dynamics as a potential testbed because QLSTM’s internal memory may capture memory effects.
  • The study evaluates QLSTM on several temporal datasets, including two open-quantum-system problems.

II. CLASSICAL MACHINE LEARNING

RNNs model sequential data by feeding a hidden state from each time step into the next, allowing information from previous inputs to be retained. Unfolding the network over time exposes its recurrent cell structure.

  • RNNs process input sequences and produce outputs while retaining a hidden state that carries information across time steps.
  • RNNs support temporal modeling by memorizing previous inputs and using finite time-domain memory for prediction.
  • At time step t, the input x_t produces an output y_t and hidden value h_t, which is fed into the network at the next step.
  • This feedback mechanism distinguishes RNNs from feed-forward networks that do not retain information from previous steps.
  • Unfolding an RNN along the time axis represents each time step as a unit cell of the architecture.

B. Long short-term memory

LSTM extends RNNs for longer-range sequential dependencies by adding a cell-state memory alongside the hidden state. The paper introduces variational quantum circuits as the building blocks for quantum LSTM models.

  • LSTM is a specialized RNN designed to learn longer-range sequential dependencies.
  • The additional cell state allows gradients to flow unchanged, partially addressing vanishing gradients and improving numerical stability during training.
  • An LSTM cell maintains both hidden state h_t and cell state c_t, whereas a basic RNN maintains only h_t.
  • The proposed quantum architecture replaces classical neural networks in LSTM cells with variational quantum circuits.
  • A variational quantum circuit encodes classical input data, applies a tunable circuit V(θ), and then performs quantum measurement.
  • Prior work reports applications of variational quantum circuits in approximation, classification, generative modeling, reinforcement learning, and transfer learning.

IV. QUANTUM LSTM

QLSTM extends classical LSTM by replacing cell neural networks with variational quantum circuits that perform feature extraction and data compression. Its circuits encode classical inputs into quantum states, apply trainable operations, and measure fixed-length outputs for subsequent processing.

  • QLSTM replaces the classical neural networks in LSTM cells with variational quantum circuits serving feature-extraction and data-compression roles.
  • Each VQC uses data encoding, variational, and quantum measurement layers, with adjustable qubit and measurement counts.
  • 1. Data Encoding Layer: The encoding procedure transforms a classical input vector into rotation angles that guide single-qubit rotations and define the corresponding quantum state.
  • 1. Data Encoding Layer: The encoding uses arctan-based angles because input values can lie in R rather than being restricted to [−1, 1].

2. Variational Layer

A QLSTM cell stacks six VQCs into forget, input/update, and output blocks. Their measured outputs are passed through classical nonlinearities and element-wise operations to update cell, hidden, and output states.

  • B. Stack All the Blocks: A QLSTM cell stacks six VQCs organized into forget, input/update, and output blocks.
  • For VQC1–VQC4, the input concatenates the previous hidden state with the current input, while measured outputs form four vectors.
  • Forget Block: The forget block uses VQC1 and a sigmoid output to determine how much of the previous cell state is retained element-wise.
  • Input and Update Block: The input/update block uses VQC2 for an addition gate and VQC3 to generate a new cell-state candidate, whose element-wise product updates the state.
  • Output Block: The output block uses VQC4 to select relevant cell-state values, then VQC5 or VQC6 produces the hidden state or output.
  • VQC5 and VQC6 transform the cell state to match the hidden-state and output dimensions when those vector dimensions differ.

C. Optimization Procedure

The optimization procedure uses the parameter-shift method to obtain analytical gradients of quantum-circuit expectation values, enabling classical gradient descent for VQC-based models.

  • The parameter-shift method analytically evaluates gradients of quantum-circuit expectation values with respect to circuit parameters.
  • These analytical gradients allow gradient-descent optimization from classical machine learning to be applied to VQC-based models.

V. EXPERIMENTS AND RESULTS

Experiments compare QLSTM with a parameter-matched classical LSTM on next-step prediction for temporal sequences. The evaluation uses a fixed four-value history, train/test splits, and RMSprop optimization across several simulated scenarios.

  • The experiments study QLSTM’s capability to learn representations of various time-dependent functions through numerical simulations.
  • The comparison uses a classical LSTM with 166 parameters and a QLSTM with 146 parameters.
  • Each QLSTM VQC uses 4 qubits at depth 2, with six VQCs per cell.
  • The task predicts the (N + 1)-th sequence value from the first N values, using N = 4 throughout the experiments.
  • Data are split into 67% for training and 33% for testing, with mathematical-function sequences rescaled to [−1, 1].
  • Both models use RMSprop, with learning rate η = 0.01, smoothing constant α = 0.99, and ϵ = 10^-8.

A. Periodic Functions

The experiments examine QLSTM and LSTM on periodic and damped-oscillation sequences. QLSTM learns both functions successfully, with earlier learning, more stable loss decreases, and milder local extrema errors in the reported cases.

  • Periodic functions: Both QLSTM and LSTM successfully learn the sine function and converge well.The sine experiment uses a simple periodic function with constant amplitude and period.
  • Periodic functions: QLSTM learns significantly more information after the first sine-function training epoch than LSTM.The reported comparison highlights QLSTM’s earlier learning during training.
  • Periodic functions: QLSTM’s loss decreases more stably than LSTM’s, without the quantum case’s loss spikes or peculiar bumps.This behavior is reported for the sine-function experiment.
  • Damped oscillations: QLSTM learns damped oscillations earlier, refines faster than classical LSTM, and has a more stable loss decrease.The comparison includes performance as early as Epoch 1 and at Epoch 15.
  • Damped oscillations: QLSTM has milder undershooting near damped-oscillation extrema and avoids the overshots observed in LSTM.Both models eventually converge to low training and testing loss; the testing set has smaller amplitude than the training set.
  • Cross-case observation: The authors present these case studies as evidence that QLSTM advantages may recur across different input functions.The stated scope is a proposed pattern across the experiments discussed.

2. Bessel functions

The experiments extend QLSTM evaluation to Bessel-function learning and delayed quantum feedback. Across these cases, QLSTM is reported to learn faster, converge more stably, and predict certain features more accurately than LSTM.

  • Bessel functions: For the Bessel-function J2 experiment, QLSTM achieves better prediction and convergence than LSTM with a slightly more complicated non-exponential-decay input.The experiment trains the models on J2 and reports improved convergence relative to the earlier cases.
  • Bessel functions: In damped-oscillation comparisons, QLSTM learns faster, converges more stably, and exhibits milder undershooting than LSTM.The LSTM performs poorly at Epochs 1 and 15 in contrast to QLSTM.
  • Delayed quantum feedback: The delayed-feedback system models a two-level atom or qubit coupled to a semi-infinite waveguide with non-Markovian dynamics.Delayed photon round trips between the qubit and mirror form a bound state in the continuum under the stated wavelength condition.
  • Delayed quantum feedback: QLSTM predicts local minima better than LSTM in the delayed quantum-feedback experiment.The comparison is reported at Epoch 100, while the models learn the temporal dependence of the modulated qubit-frequency system.
  • Delayed quantum feedback: By Epoch 15, QLSTM’s training loss is almost one order of magnitude smaller than LSTM’s in the delayed-feedback case.This result is reported alongside the delayed-control loss comparison.

4. Population Inversion

The population-inversion experiment evaluates QLSTM and LSTM on cavity quantum electrodynamics dynamics. QLSTM outperforms LSTM in learning speed, accuracy, and convergence stability, including the balanced-population zero crossing.

  • Population inversion: The experiment studies population inversion D(t) in a cavity quantum electrodynamics system where a qubit coherently interacts with a cavity.The reported setup prepares the cavity in a coherent state and observes population death and revival.
  • Population inversion: The population inversion is characterized by the difference between ground-state and excited-state populations, with D = 0 when they are balanced.The paper defines D(t) through the population difference and identifies the balanced-population condition.
  • Results: With g = 1, n̄ = |α|^2 = 40, and n_max = 100, QLSTM outperforms LSTM in learning speed, accuracy, and convergence stability.These settings are used for the Figure 10 population-inversion study.
  • Results: QLSTM learns the zero offset by Epochs 15 and 30, whereas LSTM retains a large nonzero offset at those epochs.The zero offset corresponds to equal ground- and excited-state populations.
  • Results: By Epoch 15, QLSTM’s training loss is one order of magnitude smaller than LSTM’s.The loss comparison is reported for the population-inversion experiment.

VI. CONCLUSION AND OUTLOOK

The QLSTM extends LSTM to temporal data and, with a similar parameter count, learns faster and more stably than classical LSTM in the reported tests. The framework is general, but practical training on NISQ hardware and robustness to quantum noise remain open challenges.

  • The paper presents the first hybrid quantum-classical model of long short-term memory that learns data with temporal dependency.
  • With a similar number of network parameters, QLSTM learns faster and converges more stably than classical LSTM.Its loss decreases faster and avoids the peculiar spikes typical in LSTM loss.
  • QLSTM learns local features better than LSTM in general, especially when the input has a complicated temporal structure.
  • Training large-scale time-dependent models is impractical in the simulator because of performance limitations in the quantum simulator software and classical ML training framework.
  • The framework is general: its variational circuits can use different gate sequences, more qubits, and more variational parameters.The data-encoding method can also change, including possible amplitude encoding that may provide more quantum advantage in theory.
  • Training on an actual NISQ device remains challenging because parameter-shift gradients require at least O(nms) quantum circuit evaluations per epoch.The simulations assume a perfect, noiseless, fully error-corrected quantum computer, so robustness against quantum noise remains open.
Loading 2009.01783v1…