Source-linked AI summary
Quantum Equilibrium Propagation: Gradient-Descent Training of Quantum Systems
Benjamin Scellier
TL;DR
Physical learning frameworks seek local ways to optimize cost functions in physical systems, but extracting weight gradients remains a challenge. This paper extends equilibrium propagation to quantum systems, using the Hamiltonian’s mean energy and ground states. It concludes that QEP can train quantum systems and may also inform the use of EP where quantum effects arise.
Problem
Method
Results
QEP extends EP to quantum systems by extremizing mean energy at Hamiltonian eigenstates, and the paper studies its potential as a framework for training quantum systems.
Source: §5 · p. 9Takeaways & Limitations
QEP could provide a normative framework for training quantum systems and help justify EP in physical systems where quantum effects may arise.
Source: §5 · p. 9For the gradient relation to hold, the nudge eigenstate must in principle be obtained as a smooth deformation of the free eigenstate; whether this condition can be relaxed remains open.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Equilibrium propagation (EP) is a training framework for energy-based systems, i.e. systems whose physics minimizes an energy function. EP has been explored in various classical physical systems such as resistor networks, elastic networks, the classical Ising model and coupled phase oscillators. A key advantage of EP is that it achieves gradient descent on a cost function using the physics of the system to extract the weight gradients, making it a candidate for the development of energy-efficient processors for machine learning. We extend EP to quantum systems, where the energy function that is minimized is the mean energy functional (expectation value of the Hamiltonian), whose minimum is the ground state of the Hamiltonian. As examples, we study the settings of the transverse-field Ising model and the quantum harmonic oscillator network -- quantum analogues of the Ising model and elastic network.
1 Introduction
Equilibrium propagation (EP) offers a way to train energy-based physical systems using their own dynamics to obtain weight gradients, addressing demand for local learning in physical computing. The paper extends EP to quantum systems and studies two quantum network settings.
1 Introduction
- Physical learning uses the inherent physics of physical systems for computation without requiring them to mimic neurons and synapses.It shares goals with neuromorphic computing.Source: §1 · p. 1
- Quantum and neuromorphic computing are explored as alternatives to classical digital computing, while physical learning seeks training frameworks compatible with local computation and learning rules.The introduction also notes other proposed gradient-descent frameworks for physical systems, including methods for time-reversal-invariant Hamiltonian systems and optical systems.Source: §1 · p. 1
- EP uses two equilibria under different boundary conditions to extract cost gradients and locally adjust trainable weights.It has been applied to resistor, elastic, spiking, classical Ising, and coupled-phase-oscillator networks, with experimental demonstrations on several physical platforms.Source: §1 · pp. 1–2
- Quantum Equilibrium Propagation (QEP) extends EP by using a Hamiltonian’s ground state for prediction and the expectation of a cost observable as the objective.The paper studies the transverse-field Ising network and quantum harmonic oscillator network as examples.Source: §1 · p. 2
- Massar and Mognetti [2024] independently extended EP to quantum systems using a variational formulation and also studied thermal systems through free-energy minimization.Their thermal method extracts weight gradients from thermal fluctuations.Source: §1 · p. 2
2 Equilibrium propagation
Equilibrium propagation trains energy-based systems by comparing free and nudged equilibria, using the system’s physics to estimate cost gradients. Its gradient result, locality conditions, and classical Ising and elastic-network examples establish the framework’s scope and qualifications.
2 Equilibrium propagation
- EP trains systems whose state settles to an energy minimum, using free and nudged equilibria to estimate weight gradients and update trainable weights.The nudging parameter controls the strength of the cost interaction incorporated into the total energy.Source: §2 · p. 3
- The contrastive learning rule approximates one step of gradient descent on the cost function.The theorem also holds for critical points, including saddle points, if the nudge state is a smooth deformation of the free state as the nudging parameter varies.Source: §2 · pp. 3–4
- When the energy separates into weight-specific terms involving nearby state variables, EP’s learning rule can be local in space.This locality condition is satisfied by various physical systems in which EP has been applied.Source: §2 · p. 3
- The symmetric finite-difference gradient estimator reduces the error from O(β) to O(β^2), and the contrastive rule performs exact gradient descent on contrastive functions even beyond infinitesimal nudging.For β > 0, the contrastive function is a lower bound on the cost; for β < 0, it is an upper bound, and the cited passage reports better results with β < 0.Source: §2 · p. 4
2.1 Ising model
- The classical Ising model represents N coupled spins with states σ_k ∈ {+1, −1}, with couplings and bias fields serving as trainable weights.Laydevant et al. [2024] trained an Ising network on MNIST using EP and D-Wave quantum annealing to reach its ground state.Source: §2.1 · p. 4
2.2 Elastic network
- An elastic network consists of masses connected by springs, with mass positions as system states and spring constants and rest lengths usable as trainable weights.Altman et al. [2023] demonstrated an EP variant using spring rest lengths as weights while keeping spring constants fixed.Source: §2.2 · pp. 4–5
3 Concepts of quantum mechanics
The paper introduces the quantum concepts used to formulate QEP, focusing on Hamiltonians, states, observables, and measurement. The key variational result is that the ground state minimizes the Hamiltonian’s expectation value, while measurement outcomes are probabilistic and can alter the state.
3 Concepts of quantum mechanics
- The Hamiltonian is a self-adjoint operator whose eigenstates have real energy levels, with the lowest-energy eigenstate called the ground state.The paper uses finite-dimensional Hilbert spaces for simplicity.Source: §3 · p. 5
- Observables are self-adjoint operators with probabilistic measurement outcomes, and their expectation value is the average result over many measurements.The Hamiltonian is itself an observable, with energy levels as possible outcomes.Source: §3 · p. 5
- Measuring an observable collapses the state, so estimating another observable’s expectation in the original state generally requires resetting the system.Successive measurements avoid this issue when the observables commute.Source: §3 · p. 6
- The ground state minimizes the Hamiltonian’s expectation value, and more generally Hamiltonian eigenstates are critical points of the Rayleigh quotient.Source: §3 · p. 6
4 Quantum equilibrium propagation
Quantum equilibrium propagation (QEP) adapts equilibrium propagation to quantum systems by using Hamiltonian ground states and cost-observable expectation values. Its learning rule approximates a gradient step, and the section illustrates the method with transverse-field Ising and quantum harmonic oscillator networks.
4 Quantum equilibrium propagation
- The ground state of a trainable Hamiltonian encodes the prediction, while the expectation of an output-parameterized observable supplies the cost.The ground-state energy is the Hamiltonian’s lowest eigenvalue.Source: §4 · p. 6
- QEP adds the cost interaction to the Hamiltonian and compares derivative measurements in free and nudged ground states to update the weights.The procedure uses β = 0 and then β > 0, repeating measurements for each weight.Source: §4 · p. 7
- QEP’s learning rule approximates one step of gradient descent on the cost observable’s expectation value.Theorem 3 gives the corresponding approximation for the cost-function gradient.Source: §4 · p. 7
- Quantum derivative measurements are probabilistic, can change the state, and generally cannot be simultaneous unless the observables commute.The system must be reset to its ground state after each measurement.Source: §4 · p. 7
- QEP can use an eigenstate other than the ground state if the nudged eigenstate varies smoothly from the free eigenstate as β changes.This smooth deformation is the stated requirement for the gradient result to hold.Source: §4 · p. 8
4.1 Transverse-field Ising model
- The transverse-field Ising example uses trainable spin couplings and biases, with an N-spin state represented across 2^N basis configurations.The Pauli Z operators commute with one another, as do the Pauli X operators, but Z and X operators do not commute.Source: §4.1 · pp. 8–9
4.2 Quantum harmonic oscillator network
- The quantum harmonic oscillator network models N one-dimensional particles with harmonic interactions, and its spring constants are trainable weights.Position operators commute, allowing simultaneous measurement of those observables.Source: §4.2 · p. 9
5 Discussion
QEP extends equilibrium propagation to quantum systems by extremizing mean energy, with extrema at Hamiltonian eigenstates. The discussion identifies measurement, eigenstate-preparation, and simulation constraints, alongside possible extensions through other EP frameworks.
QEP scope
- QEP extends EP to quantum systems, where mean energy is extremized at Hamiltonian eigenstates, and could serve as a training framework or justify EP where quantum effects matter.The discussion also notes possible relevance to quantum annealing used in classical EP by Laydevant et al. [2024].Source: §5 · p. 9
Measurement constraints
- Quantum measurements are probabilistic, so accurate gradient estimates may require multiple measurements and additional memory to store outcomes.The authors suggest studying single-measurement gradient-estimator variance.Source: §5 · p. 9
- Hamiltonian derivatives generally cannot be measured simultaneously unless they commute, although measurements in the transverse-field Ising and quantum harmonic oscillator models can largely be parallelized.Source: §5 · p. 10
Eigenstate condition
- QEP can use any Hamiltonian eigenstate, but the nudge eigenstate must in principle be a smooth, adiabatic deformation of the free eigenstate for Eq. (30) to hold.Whether this condition is necessary or can be relaxed in practice remains unresolved.Source: §5 · p. 10
Numerical simulation
- Exact diagonalization becomes impractical for quantum Ising networks above a few dozen spins because the state-space dimension grows as d = 2^N.DMRG and VMC are suggested as approximate methods for larger systems.Source: §5 · p. 10
Potential extensions
- Holomorphic EP could provide exact gradients but a complex nudge makes the total Hamiltonian non-self-adjoint, while Agnostic EP avoids measurement and feedback through physical weight dynamics.Agnostic EP couples trainable weights to control variables that perform homeostatic control.Source: §5 · p. 10
A Proof of Theorem 3
The proof establishes that the Hamiltonian ground state minimizes its expectation value, then uses this variational result in the proof of the QEP gradient approximation. It defines the mean energy and cost functionals used in the derivation.
Variational formulation
- The ground state minimizes the Hamiltonian expectation value, with equality for normalized states that are phase multiples of that ground state.Source: A Proof of Theorem 3 · p. 12
- The proof establishes an orthonormal eigenbasis of the self-adjoint Hamiltonian and orders its eigenvalues from lowest to highest.Source: A Proof of Theorem 3 · p. 12
QEP gradient proof
- Theorem 3 gives an approximation to the cost gradient, and its proof defines mean energy as the Hamiltonian expectation and mean cost as the cost-observable expectation.Source: A Proof of Theorem 3 · p. 12
- The derivation invokes Lemma 2, which permits application of Theorem 1 in the QEP algorithm; explicit forms of the mean cost and mean total energy functionals then yield Eq. (29).Source: A Proof of Theorem 3 · p. 13