PaperlayerView exact revision on arXiv

Source-linked AI summary

A deep learning theory for neural networks grounded in physics

Benjamin Scellier

arXiv:2103.09985v2cs.LG

TL;DR

Deep learning training relies on gradients, while conventional hardware faces energy and latency costs associated with separating memory from processing. The thesis develops equilibrium propagation for systems described by variational equations, with local learning rules under sum-separability, and discusses implications for neuromorphic computing and neuroscience.

  • Problem

    Deep learning hardware faces speed and energy inefficiencies associated with separating memory and processing, motivating interest in alternatives that preserve learning while exploiting physical substrates.

    Source: Introduction · p. 21, §1.2.8 · p. 32

  • Method

    The thesis presents equilibrium propagation, a gradient-computation procedure for systems described by variational equations that can use locally available information for each parameter when the system's energy or Lagrangian is sum-separable.

    Source: §1.3.2 · p. 35, §2.3 · p. 42

  • Results

    The thesis presents a framework for variational systems that maintains benefits of backpropagation, and shows that EqProp's learning rule is local for sum-separable energy-based models and takes a contrastive Hebbian form for gradient systems.

    Source: §1.3.2 · p. 35, §1.4 · p. 37, Conclusion · p. 103

  • Takeaways & Limitations

    The framework may have implications for neuromorphic computing and neuroscience, and may apply to physical systems in engineering beyond neuromorphic computing.

    Source: Introduction · p. 21, Conclusion · p. 103

    EqProp training has practical constraints: experiments require clipped neuron states, while the Hopfield formulation's bidirectional symmetric weights are described as biologically unrealistic.

    Source: §3.2.1 · p. 55, §5.3.1 · p. 88

Abstract

from arXiv · show

In the last decade, deep learning has become a major component of artificial intelligence. The workhorse of deep learning is the optimization of loss functions by stochastic gradient descent (SGD). Traditionally in deep learning, neural networks are differentiable mathematical functions, and the loss gradients required for SGD are computed with the backpropagation algorithm. However, the computer architectures on which these neural networks are implemented and trained suffer from speed and energy inefficiency issues, due to the separation of memory and processing in these architectures. To solve these problems, the field of neuromorphic computing aims at implementing neural networks on hardware architectures that merge memory and processing, just like brains do. In this thesis, we argue that building large, fast and efficient neural networks on neuromorphic architectures also requires rethinking the algorithms to implement and train them. We present an alternative mathematical framework, also compatible with SGD, which offers the possibility to design neural networks in substrates that directly exploit the laws of physics. Our framework applies to a very broad class of models, namely those whose state or dynamics are described by variational equations. This includes physical systems whose equilibrium state minimizes an energy function, and physical systems whose trajectory minimizes an action functional. We present a simple procedure to compute the loss gradients in such systems, called equilibrium propagation (EqProp), which requires solely locally available information for each trainable parameter. Since many models in physics and engineering can be described by variational principles, our framework has the potential to be applied to a broad variety of physical systems whose applications extend to various fields of engineering, beyond neuromorphic computing.

Université de Montréal

The supplied contents and abbreviation entries outline a thesis on neural networks grounded in physics, from background and equilibrium propagation to applications and implications.

for neural networks grounded in physics

  • The contents list chapters on equilibrium propagation and training continuous Hopfield networks, followed by numerical experiments on MNIST.The listed sections include energy-based models, gradient formulas, and examples of sum-separable models.Source: for neural networks grounded in physics · p. 8
  • Later chapters list training nonlinear resistive networks, recurrent neural network models, and contrastive meta-learning.The contents also include MNIST simulations and sections on fully connected and convolutional layers.Source: for neural networks grounded in physics · pp. 8–10
  • The conclusion contents include implications for neuromorphic computing, neuroscience, and synergy between neuroscience and AI.Listed neuroscience topics include variational formulations of neural computation, the SGD hypothesis of learning, and evolution.Source: for neural networks grounded in physics · p. 10

Abbreviations List

  • The abbreviation list expands EqProp as Equilibrium Propagation and GDD as Gradient Descending Dynamics, a property relating EqProp dynamics to loss gradients in Chapter 5's discrete-time setting.It also expands CHL as Contrastive Hebbian Learning and LBM as Lagrangian-Based Model.Source: Abbreviations List · pp. 15–16

Introduction

The introduction motivates brain-inspired AI by tracing neural-network learning and the efficiency limits of current hardware, then presents a mathematical framework for computation in physical substrates. It proposes retaining SGD while using variational principles and equilibrium propagation to estimate gradients from locally available information.

Introduction

  • The proposed framework preserves core AI principles while offering a path to physical substrates that may support larger, faster, and more efficient AI systems.It is presented as a novel mathematical framework for exploiting physics to perform computations.Source: Introduction · p. 21

1.1.2. Machine Learning Basics

  • Machine learning replaces fixed rules with adjustable parameters tuned from examples, while human intelligence serves as a practical benchmark for AI.In supervised learning, examples include labels; unsupervised learning uses unlabeled data, and reinforcement learning uses rewards.Source: §1.1.2 · pp. 23–24, §1.1.1 · p. 23

1.2.1. Neuroscience Basics

  • The brain’s synapses combine computing and memory, change with experience, and motivate efforts to understand the brain’s learning algorithms.Understanding how experience changes synapses and behavior is presented as important both for explaining intelligence and developing intelligent machines.Source: §1.2.1 · p. 26

1.2.3. Energy-Based Models vs Differentiable Neural Networks

  • Differentiable neural networks use backpropagation to compute gradients for SGD, while network architectures and training techniques shape loss landscapes that affect optimization.The introduction contrasts these networks with energy-based models, whose dynamics converge to equilibrium but may be slow because they require many iterations.Source: §1.2.4 · pp. 28–29, §1.2.3 · p. 28, §1.2.5 · p. 30

1.2.6. Deep Learning Revolution

  • Neural networks achieve strong performance across many tasks, but remain task-specific and lack humans’ flexible handling of abstract concepts.The introduction suggests that more multifunctional, continually learning systems and advances in causal reasoning may be needed.Source: §1.2.6 · p. 30

1.2.8. The Von Neumann Bottleneck

  • Separating memory and processing in von Neumann hardware requires data movement that makes neural-network training extraordinarily energy-inefficient.Training can take months even across thousands of GPUs, and electricity costs can reach millions of dollars.Source: §1.2.8 · p. 31, §1.2.7 · p. 31
  • 1500 kW.h to train BERT equals the energy a 20W human brain consumes in nine years, illustrating the energy cost of the von Neumann bottleneck.The thesis motivates in-memory computing, which performs computation where synaptic weights are stored and adjusted, as a way to build energy-efficient, massively parallel networks.Source: §1.2.8 · p. 32, §1.2.9 · p. 32

1.2.10. Challenges of Analog Computing

  • Analog noise and manufacturing mismatches limit precision and make large circuits difficult to predict, while reproducing ideal software operations may require low precision or energy-intensive conversion.The variability shared by biological neurons motivates rethinking neural-network learning in the presence of imperfect devices.Source: §1.2.10 · p. 33

1.3. A Deep Learning Theory for Neural Networks Grounded in Physics

  • The thesis frames physical systems as neural networks whose adjustable parameters can be tuned by SGD, and proposes EqProp to estimate gradients using local information.The framework applies when system states or dynamics derive from variational principles and is intended to exploit physics directly for neural-network implementation and training.Source: §1.3 · pp. 33–34, §1.3.1 · p. 34

1.3.2. Variational Principles of Physics as First Principles

  • Energy-based models describe equilibrium states through an energy function, while Lagrangian-based models describe trajectories through stationary action and can serve as recurrent networks.When the energy or Lagrangian is sum-separable, EqProp computes gradients for arbitrary loss functions using information locally available for each parameter.Source: §1.3.2 · pp. 34–35

1.3.3. Universality of Variational Principles in Physics

  • Because variational principles describe systems across physics and engineering, the framework could be implemented on many physical substrates, including nonlinear resistive networks.The thesis presents nonlinear resistive networks as one such implementation option.Source: §1.3.3 · p. 35

1.3.4. Rethinking the Notion of Computation

  • The framework proposes computation grounded in physical variational laws, with EqProp’s computations differing from conventional step-by-step algorithms.It is presented as an alternative to the Turing-style computing paradigm and the von Neumann architecture.Source: §1.3.4 · pp. 35–36

1.3.5. A Novel Differentiation Method Compatible with Variational Principles

  • The thesis presents a novel differentiation identity for computing the gradient of a loss evaluated at a state satisfying an equilibrium condition, using a β-perturbed objective.It defines F(θ, β, s) = E(θ, s) + β C(s) and relates the perturbed state to the unperturbed equilibrium state.Source: §1.3.5 · p. 36

1.4. Overview of the Manuscript and Link to Prior Works

  • The manuscript develops EqProp for energy-based models, gradient systems, and nonlinear resistive networks, then covers discrete-time models, Lagrangian-based models, stochastic systems, and contrastive meta-learning.The overview links chapters to earlier work and notes applications including local learning rules, Hopfield networks, and recurrent-network models.Source: §1.4 · p. 37

Equilibrium Propagation: A Learning

The section heading presents equilibrium propagation as a learning algorithm for a class of systems.

Equilibrium Propagation: A Learning

Variational Equations

Equilibrium propagation (EqProp) replaces backpropagation for systems described by variational equations, including energy-based models and physical systems. It estimates loss gradients through changes in equilibrium, with local parameter updates when the energy is sum-separable.

2.2. Energy-Based Models

  • EqProp trains energy-based models whose equilibrium states are stationary points of an energy function, including minima, maxima, or saddle points.The equilibrium state can be implicit and need not be an energy minimum.Source: §2.2 · pp. 41–42, §2.7 · p. 47

2.3. Gradient Formula

  • EqProp computes loss gradients by varying the nudging factor in the total energy F = E + βC and comparing the resulting equilibrium states.The gradient formula follows from the change in energy with respect to parameters as β varies around zero.Source: §2.3 · pp. 42–43
  • For a sum-separable energy, each parameter’s loss gradient depends only on information locally available to that parameter.This locality condition motivates a two-phase procedure using free and nudged equilibria; finite differences estimate the gradient.Source: §2.3 · p. 43, §2.4 · pp. 43–44

2.4. Equilibrium Propagation

  • Using finite differences gives a biased gradient estimate, while a symmetric estimator using positive and negative nudging reduces the mismatch to O(β^2).Higher-order estimators using additional nudging values are also possible.Source: §2.4 · p. 44

2.5. Examples of Sum-Separable Energy-Based Models

  • Hopfield, resistive, flow, and elastic networks are examples of sum-separable energy-based models that can be trained with EqProp.In Hopfield networks the parameter gradient yields a contrastive Hebbian rule; resistive circuits use dissipated power as energy, while nonlinear circuits can use co-content.Source: §2.5 · pp. 44–45

2.7. Remarks

  • The framework also covers physical systems with infinite-dimensional states and time-varying systems described by stationary action.For time-varying systems, the state is a trajectory and the energy is an action functional.Source: §2.7 · p. 47
  • The gradient derivation assumes an invertible energy Hessian; behavior at singular Hessians is left unresolved.The authors state that training may pass through parameter values with a singular Hessian and set aside the resulting questions.Source: §2.7 · p. 48
  • EqProp’s applicability extends beyond supervised learning to input-gradient computation, reinforcement learning, and meta-learning.The input-gradient formula is identified as useful for image synthesis and discriminator gradients in generative adversarial networks.Source: §2.7 · p. 48

Equilibrium Propagation

This chapter applies equilibrium propagation (EqProp) to continuous Hopfield networks and related gradient systems. It develops the theory and training rule, then examines numerical experiments and the relationship to contrastive Hebbian learning.

Chapter 3

  • The chapter is presented as a compilation of earlier work on EqProp and continuous Hopfield networks.The cited works are Scellier and Bengio [2017, 2019].Source: Equilibrium Propagation · p. 49

Equilibrium Propagation

  • The chapter applies EqProp to gradient systems, whose states descend an energy gradient and stabilize at an energy minimum.It gives an analytical expression for transient states and connects them to recurrent backpropagation.Source: Equilibrium Propagation · p. 49
  • For continuous Hopfield networks, the energy gradient yields leaky-integrator neuron dynamics with bidirectional symmetric synapses and a local, Hebbian EqProp update.The chapter also reports numerical experiments on deep Hopfield networks for MNIST and studies the relationship to contrastive Hebbian learning.Source: Equilibrium Propagation · p. 49

3.1. Gradient Systems

This section applies equilibrium propagation to gradient systems that descend an energy function. It relates the nudged dynamics to gradients of a projected cost and establishes a connection with recurrent backpropagation.

3.1.1. Gradient Systems as Energy-Based Models

  • In gradient systems, the state follows the energy gradient until reaching a local or global energy minimum.These equilibria are stable under small perturbations and can act as point attractors that recover stored memories from incomplete data.Source: §3.1.1 · p. 50

3.1.2. Training Gradient Systems with Equilibrium Propagation

  • EqProp first relaxes the system to a free equilibrium, then adds a cost-gradient force that nudges the dynamics toward lower cost.In the limit β → 0, the loss gradient is estimated from the free and nudged equilibria; with sum-separable energy, each parameter’s update is local.Source: §3.1.2 · pp. 50–51

3.1.3. Transient Dynamics

  • During nudging, transient states encode gradients of the projected cost, which is the cost reached after following free dynamics from an initial state.The projected cost converges to the equilibrium loss as the free dynamics settle; the transient-state result also yields an analytical formula for truncated EqProp.Source: §3.1.3 · pp. 51–52

3.1.4. Recurrent Backpropagation

  • Theorem 3.1 identifies the transient processes used to describe EqProp and recurrent backpropagation, and their state and parameter processes coincide at every time.The equivalence follows because both process pairs solve the same linear differential equation, whose solution is unique.Source: §3.1.3 · p. 52, §3.1.4 · p. 53
  • Recurrent backpropagation computes parameter-gradient dynamics after the free phase and recovers the loss gradient as the process approaches its limit.Unlike recurrent backpropagation, EqProp uses one kind of dynamics for both training phases; recurrent backpropagation requires a special gradient-computation circuit in its second phase.Source: §3.1.4 · p. 54

3.2. Continuous Hopfield Networks

The continuous Hopfield model uses symmetric, bidirectional synapses and state dynamics derived from an energy function. EqProp trains it through free and nudged phases, with the nudged dynamics carrying error signals from outputs through the network.

3.2.1. Hopfield Energy

  • The Hopfield energy produces leaky-integrator-like neuron dynamics, but requires bidirectional symmetric weights, the biologically unrealistic weight-transport constraint.The dynamics also include a σ′(si) factor that modulates each neuron's total input.Source: §3.2.1 · p. 55
  • For supervised learning, inputs remain clamped, output equilibrium states are predictions, and squared error measures output-target discrepancy.The free state comprises hidden and output neurons, while the input neurons are fixed at x.Source: §3.2.1 · p. 55
  • EqProp combines internal energy E and target cost C into total energy F = E + β C, making the target an external potential acting on output neurons.The force attracts outputs toward targets for β ≥ 0 and repels them for β ≤ 0; at β = 0, outputs are target-insensitive.Source: §3.2.1 · pp. 55–56

3.2.2. Training Continuous Hopfield Networks with Equilibrium Propagation

  • EqProp uses a free phase followed by a nudged phase: outputs first settle freely, then target-driven perturbations propagate backward to a new equilibrium.The free-state output is the prediction; the nudged phase begins with hidden layers at equilibrium and lets the perturbation travel through them.Source: §3.2.2 · pp. 56–57
  • EqProp estimates loss gradients from free and nudged states, yielding a local synaptic update that is a form of contrastive Hebbian learning.The gradient result applies the general theorem to the sum-separable Hopfield energy and squared-error cost.Source: §3.2.2 · p. 57

3.2.3. ‘Backpropagation’ of Error Signals

  • EqProp's free phase resembles backpropagation's forward pass, while its nudged phase propagates error signals backward through network layers.Unlike backpropagation, EqProp's neural computation serves both inference and error back-propagation; temporal derivatives can carry gradient signals.Source: §3.2.3 · pp. 57–58
  • The symmetric-weight requirement can be relaxed, but the cited alternatives compute a biased gradient.The cited alternatives are Scellier et al. [2018], Ernoult et al. [2020], Laborieux et al. [2021], and Tristany et al. [2020].Source: §3.2.3 · p. 58

3.3. Numerical Experiments on MNIST

The experiments train recurrent, symmetric deep Hopfield networks on MNIST using EqProp. Precise free-phase convergence is important in these simulations, and removing state caching lowers test error while requiring more free-phase iterations.

3.3. Numerical Experiments on MNIST

  • The MNIST experiments use recurrent deep Hopfield networks with symmetric bidirectional connections to classify 28 × 28 images into ten digits.The dataset contains 60,000 training and 10,000 test examples; networks have 1–3 hidden layers of 500 neurons each.Source: §3.3 · p. 58, §3.3.1 · p. 59

3.3.1. Implementation Details

  • Free-phase equilibrium must be reached with high precision for reliable EqProp gradients, and the required iteration count grows quickly with network depth.The authors say this high iteration count is a concern for conventional computer simulations, while analog hardware may enable fast inference by exploiting circuit physics.Source: §3.3.1 · p. 60
  • Full relaxation is unnecessary in the nudged phase; the authors heuristically set its iteration count proportional to layer count so error signals can propagate backward.They choose β = 1, while randomizing its sign slightly improves test accuracy and cancels the one-sided estimator's O(β) error on average.Source: §3.3.1 · p. 60

3.3.2. Experimental Results

  • Caching free-phase equilibrium states speeds relaxation and improves precision, but the reported cached results overfit the training set and generalize worse.The caching experiments aim for 0.00% training error; the table reports single-trial errors for cached runs and five-trial averages for uncached runs.Source: §3.3.2 · p. 61, Table 1 · p. 61
  • Without caching equilibrium states, test error is lower, but more free-phase iterations are needed for convergence.The uncached experiments initialize each example's network state to zero at the start of each free-phase relaxation.Source: §3.3.2 · p. 61, Table 1 · p. 61
  • Later experiments improved simulation speed, test accuracy, and task complexity, while the thesis identifies neuromorphic substrates as EqProp's likelier practical potential.These later results are presented in Chapter 5; the passage gives no quantitative comparison.Source: §3.3.2 · p. 61

3.4. Contrastive Hebbian Learning (CHL)

In continuous Hopfield networks, EqProp is related to GeneRec and contrastive Hebbian learning (CHL), but uses a nudged rather than clamped second phase. The chapter explains when CHL updates cancel and how EqProp's loss avoids a mode-switching problem.

3.4.1. Contrastive Hebbian Learning in the Continuous Hopfield Model

  • CHL uses a free phase with anti-Hebbian updates, then clamps outputs to targets for a second phase with Hebbian updates.Hidden neurons evolve freely in both phases; the second phase is called the clamped or positive phase.Source: §3.4.1 · p. 62

3.4.2. An Intuition Behind Contrastive Hebbian Learning

  • CHL and EqProp stop learning when the free and target-conditioned states coincide, because their opposing update terms cancel.For EqProp, this property also holds with the symmetric-difference gradient estimator.Source: §3.4.2 · p. 63

3.4.3. A Loss Function for Contrastive Hebbian Learning

  • CHL's loss can yield inconsistent updates when its two phases settle in different energy modes, and prior work reports learning deterioration and basin-hopping discontinuities.Movellan [1991] points out the inconsistency; Baldi and Pineda [1991] note abrupt discontinuities.Source: §3.4.3 · p. 64
  • EqProp instead optimizes the loss at the free state and estimates its gradient using nudged states that continuously deform from that state within the same energy mode.In continuous Hopfield networks, EqProp is similar to GeneRec, while its nudging-factor and total-energy formalism also covers energy- and Lagrangian-based models.Source: §3.4 · p. 62, §3.4.3 · p. 64

Equilibrium Propagation

This section examines equilibrium propagation in physical systems described by variational principles, focusing on local training and gradient estimates in analog resistive networks. It also presents a deep analog architecture, MNIST simulations, and circuit design choices for its implementation.

Equilibrium Propagation

  • EqProp applies to systems whose equilibrium states satisfy variational equations, offering a route to train energy-based models grounded in physical laws.The chapter emphasizes that this scope extends beyond Hopfield networks and gradient systems.Source: Equilibrium Propagation · p. 65
  • The chapter presents a deep-Hopfield-inspired analog architecture and numerical MNIST simulations, while emphasizing that EqProp can train other resistive-network architectures.The optimization procedure is presented separately from the specific architecture, allowing exploration of the design space of EqProp-trainable analog networks.Source: Equilibrium Propagation · pp. 66–67

4.1. Nonlinear Resistive Networks as Analog Neural Networks

  • Nonlinear resistive circuits can map input voltages to output predictions, with programmable resistor conductances serving as adjustable weights.Ideal tunability is a simplifying assumption; practical programmable devices still pose challenges that the chapter does not address.Source: §4.1 · p. 67

4.2. Nonlinear Resistive Networks are Energy-Based Models

  • Kirchhoff’s laws make the circuit’s steady state a stationary point of its sum-separable total pseudo-power, so nonlinear resistive networks are energy-based models.The total pseudo-power, also called co-content, is the sum of the pseudo-powers of individual circuit elements.Source: §4.2 · pp. 67–68

4.2.1. Linear Resistance Networks

  • In linear resistance networks, the steady-state voltages globally minimize dissipated power, but these networks cannot perform nonlinear operations.The global-minimum result follows from the convex power functional; nonlinear elements such as diodes are needed for nonlinear operations.Source: §4.2.1 · pp. 68–69

4.2.3. Nonlinear Resistive Networks

  • For a nonlinear resistive network, each element’s pseudo-power is the primitive of its current-voltage characteristic, and the total pseudo-power is stationary at the circuit steady state.EqProp requires this first-order stationary-point condition, not that the steady state be a global minimum.Source: §4.2.2 · p. 70, §4.2.3 · pp. 70–71

4.3.1. Supervised Learning Setting

  • The sum-separable energy enables EqProp to estimate a conductance’s loss gradient using only the voltage drop across its corresponding resistor.This provides a principled way to train analog networks with stochastic gradient descent.Source: Equilibrium Propagation · p. 66, §4.3.1 · p. 72

4.3.2. Training Procedure

  • EqProp compares a free phase with a nudged phase, using local voltage-drop changes to estimate loss gradients for adjustable circuit parameters.In the free phase, the circuit predicts and the loss is computed; in the nudged phase, currents proportional to the loss derivatives are applied at output nodes.Source: §4.3.2 · pp. 72–73
  • EqProp’s training procedure uses component-wise pseudo-power contributions, and two nudged phases with positive and negative β can reduce gradient-estimator bias and variance.For a linear resistor, the adjustable parameter is its conductance.Source: §4.3.2 · pp. 73–74

4.3.3. On the Loss Gradient Estimates

  • EqProp can estimate a resistor’s gradient sign by comparing the absolute terminal voltages in the free and nudged phases, without requiring precise voltage measurements.The authors suggest EqProp estimates may be more robust to device mismatches because the same circuit operates in both phases, but note this has no experimental evidence and does not resolve weight-update asymmetry.Source: §4.3.3 · pp. 74–75

4.4. Example of a Deep Analog Neural Network Architecture

  • The example deep analog network alternates programmable-resistor crossbars for linear transformations with nonlinear processing stages, following an architecture proposed by Kendall et al. [2020].Its synaptic conductances represent weights, and bidirectional amplifiers support signal propagation in both directions.Source: §4.4 · p. 75, §4.4.1 · p. 76

4.4.1. Antiparallel Diodes

  • Antiparallel diodes in series with voltage sources implement bounded, sigmoidal neuron responses by sinking excess input current to ground.Each diode pair is placed between a neuron node and ground.Source: §4.4.1 · pp. 75–76

4.4.2. Bidirectional Amplifiers

  • Resistive layers cause voltages to decay, so the architecture combines forward-directed voltage-controlled voltage sources with backward-directed current-controlled current sources to amplify signals.Source: §4.4.2 · pp. 76–77

4.4.3. Positive Weights

  • To accommodate positive-only programmable-resistor conductances, the proposed approach doubles input nodes with one set of inputs inverted and doubles output nodes for classification.Source: §4.4.3 · p. 77

4.4.4. Current Sources

  • During the nudged phase, current sources inject output-gradient signals scaled by β; in the free phase, they are set to zero current and act as open circuits.Here, β has the physical dimensions of a conductance.Source: §4.4.4 · p. 77

4.5. Numerical Simulations on MNIST

  • 3.43% test error was achieved by the SPICE network after 10 epochs, versus > 7% for the linear classifiers and logistic regression models reported by LeCun et al. [1998].The simulation used one hidden layer of 100 neurons; the authors attribute the advantage to diode nonlinearities.Source: §4.5 · p. 78

Models with Equilibrium Propagation

This section develops approaches for training EqProp models, including discrete-time models aimed at accelerating software simulations and scaling training, and gradient descending dynamics to address training difficulties. It reports MNIST and CIFAR-10 experiments, relates transient nudged-phase computation to loss-gradient computation and BPTT, and highlights gradient-estimation and inference-time challenges.

Models with Equilibrium Propagation

  • The chapter introduces software-oriented EqProp models to reduce inference cost and scale training beyond tasks such as MNIST, broadening EqProp’s scope for benchmarking against advanced deep-learning models.Some choices, including shared weights in convolutional models, move away from biorealism and neuromorphic considerations.Source: Models with Equilibrium Propagation · p. 79
  • The chapter’s gradient-descending-dynamics property links transient states in EqProp’s second phase to loss partial derivatives and provides a criterion for deciding when first-phase dynamics have converged.The chapter identifies this result as Theorem 5.1 and Fig. 8.Source: Models with Equilibrium Propagation · p. 80

5.1. Training Discrete-Time Dynamical Systems with Equilibrium Propagation

  • Discrete-time EqProp applies to recurrent systems with static input that converge to a fixed point, using a free phase followed by a cost-nudged phase to estimate loss gradients.A symmetrized estimator uses two nudged phases, with positive and negative nudging, to reduce gradient-estimate bias and variance.Source: §5.1.1 · p. 81, §5.1.2 · pp. 81–82

5.2. RNN Models with Static Input

  • The chapter presents fully connected recurrent models with symmetric weights and convolutional recurrent models, and extends the cost functions beyond squared error to cross-entropy.The convolutional model can combine convolutional and fully connected layers; its cross-entropy implementation uses a softmax readout.Source: §5.2.1 · p. 83, §5.2.2 · p. 84, §5.2.4 · p. 84

5.3. Experiments on MNIST and CIFAR-10

  • 11.68% test error is the best CIFAR-10 result, achieved with cross-entropy and approaching BPTT with less than 0.6% degradation in accuracy.The symmetric difference estimator yields 12.45% test error and stabilizes training across runs; random-sign nudging yields 12.61% mean test error in four of five runs, while constant-sign one-sided estimation has 84.90% train error.Source: Table 3 · p. 86, §5.3 · pp. 86–87
  • The discrete-time models show no accuracy degradation relative to continuous-time networks on MNIST, and the ConvNet achieves the lowest test error rate, approximately 1%.The discrete-time models require many fewer first-phase time steps (T) than the continuous-time networks.Source: §5.3 · pp. 85–86, Table 2 · p. 86
  • The symmetric estimator reduces both bias and variance and stabilizes CIFAR-10 training, whereas constant-sign one-sided nudging is unstable and fails to fit the training data.With random-sign nudging, training proceeds well in four of five runs but collapses in the fifth; the reported 12.61% mean test error includes only the four successful runs.Source: §5.3 · pp. 86–87, Table 3 · p. 86

5.3.1. Challenges with EqProp Training

  • On digital computers, EqProp requires highly accurate free-phase fixed points for useful gradient estimates, leading to hundreds of inference iterations even in networks with only a few layers.The chapter notes that long inference times are specific to numerical simulations and may not affect neuromorphic hardware, where physics performs energy minimization.Source: §5.3.1 · p. 87
  • EqProp faces neuron saturation that impedes small-β error propagation and sensitive hyperparameter tuning, including layer-wise learning rates and phase lengths.Larger β can improve error propagation but degrade gradient estimates; proposed mitigations include a leak activation, while EqProp-specific weight initialization remains largely unexplored. Training also tunes T, K, and β.Source: §5.3.1 · p. 88

5.4. Gradient Descending Dynamics (GDD)

  • Although EqProp’s gradient formula depends only on free and nudged fixed points, the network’s transient states can perform step-by-step gradient computation.This result assumes the neurons gradually move from their free fixed-point values toward their nudged fixed-point values.Source: §5.4 · p. 88

5.4.1. Transient Dynamics

  • When the free-phase states have reached the fixed point by T−K, nudged-phase activity and weight increments encode loss gradients step by step, yielding a necessary fixed-point test (Ernoult et al. [2019]).The theorem’s condition is s⋆ = sT = sT−1 = … = sT−K; its identities provide a criterion for deciding whether the first-phase fixed point has been reached accurately enough.Source: §5.4.1 · pp. 89–90

5.4.2. Backpropagation Through Time

  • In the static-input RNN setting, GDD matches EqProp increments step by step to BPTT gradients, and truncated EqProp corresponds to truncated BPTT.Forward-time computation in EqProp corresponds to backward-time computation in BPTT.Source: §5.4.2 · pp. 90–91

Extensions of Equilibrium Propagation

The chapter outlines three extensions of equilibrium propagation: dynamical systems with time-varying inputs, stochastic systems, and contrastive meta-learning.

Extensions of Equilibrium Propagation

6.1. Equilibrium Propagation in Dynamical Systems with Time-Varying Inputs

This section extends equilibrium propagation from energy-based models to dynamical systems whose trajectories are determined by stationary action. It gives a gradient formula and a locally computed training procedure, and shows that energy-based models arise as a special case under static conditions.

6.1.1. Lagrangian-Based Models

  • Lagrangian-based models describe systems with time-varying inputs whose effective trajectories make the action functional stationary.Source: §6.1 · p. 93, §6.1.1 · p. 94
  • The loss integrates state costs along the effective trajectory, while the Lagrangian may be sum-separable across parameters using locally available state, input, and velocity information.Source: §6.1.1 · p. 94

6.1.2. Gradient Formula

  • For sum-separable Lagrangians, Theorem 6.1 gives a loss-gradient formula whose parameter-wise gradients use only locally available information.Source: §6.1.2 · p. 95

6.1.3. Training Sum-Separable Lagrangian-Based Models

  • Training runs a free trajectory and a nudged trajectory from the same initial state, then updates each parameter locally using its gradient estimate.Source: §6.1.3 · p. 96
  • The procedure runs the input twice but does not require storing past system states, unlike backpropagation through time.Source: §6.1.3 · p. 96

6.1.4. From Energy-Based to Lagrangian-Based Models

  • The framework replaces configurations and energy with trajectories and action, and equilibrium states with trajectories stationary under the action.Source: §6.1.4 · p. 97

6.1.5. Lagrangian-Based Models Include Energy-Based Models

  • With a static input and a Lagrangian independent of state velocity, an energy-minimizing equilibrium is a stationary trajectory, making energy-based models a special case.Source: §6.1.5 · p. 97
  • When targets and costs are static too, the trajectory loss reduces to the static loss up to a factor of T, and Lagrangian EqProp reduces to energy-based EqProp up to the same factor.Source: §6.1.5 · p. 97

6.2. Equilibrium Propagation in Stochastic Systems

This section extends equilibrium propagation to stochastic systems, representing equilibrium as a probability distribution rather than a single state. It gives a stochastic gradient result and describes Langevin sampling and the corresponding free and nudged phases.

6.2. Equilibrium Propagation in Stochastic Systems

  • Because physical systems such as analog circuits and biological networks are subject to noise, the stochastic extension may be useful in neuromorphic computing and neuroscience.Source: §6.2 · p. 97

6.2.1. From Deterministic to Stochastic Systems

  • In the stochastic setting, equilibrium is a distribution over configurations, and the loss is the expected cost under the equilibrium Boltzmann distribution.Source: §6.2.1 · p. 98

6.2.2. Gradient Formula

  • Theorem 6.2 extends the gradient result to stochastic systems, with a further formula when the energy is sum-separable.Source: §6.2.2 · p. 98

6.2.3. Langevin Dynamics

  • Langevin dynamics adds a Brownian noise term to energy-gradient dynamics, and the resulting state settles to the Boltzmann distribution.Source: §6.2.3 · p. 99

6.2.4. Equilibrium Propagation in Langevin Dynamics

  • In Langevin EqProp, the free phase samples the equilibrium distribution and the nudged phase samples the distribution associated with energy plus β times cost; samples from both estimate the loss gradient.Source: §6.2.4 · p. 100

6.3. Contrastive Meta-Learning

This section presents contrastive meta-learning as an application of equilibrium propagation to meta-parameter training. In few-shot learning, the objective is to choose shared parameters that yield low expected test loss after task-specific adaptation.

6.3.1. Meta-Learning and Few-Shot Learning

  • Few-shot meta-learning seeks shared parameters that minimize expected test loss after task-specific adaptation using limited training data.Source: §6.3.1 · p. 101

6.3.2. Contrastive Meta-Learning

  • Contrastive meta-learning applies EqProp to compute gradients of the meta-loss when adaptation leaves the task-specific parameter at an equilibrium of the inner loss.Source: §6.3.2 · p. 101
  • The approach extends to bilevel optimization of an outer loss under the constraint that the task-specific parameter satisfies the inner-loss equilibrium condition.Source: §6.3.2 · p. 102

Conclusion

The thesis presents a framework for systems governed by variational equations that preserves key benefits of SGD and may inform neuromorphic computing and neuroscience. Its discussion connects this framework to questions about neural computation, learning, evolution, and the relationship between AI and neuroscience.

Conclusion

  • The framework applies to systems whose state or dynamics solve variational equations while maintaining the benefits of backpropagation.The thesis notes potential implications for both neuromorphic computing and neuroscience.Source: Conclusion · p. 103

7.1. Implications for Neuromorphic Computing

  • Neuromorphic processors could exploit physics and in-memory computing while retaining SGD, although matching current deep-learning performance remains a future prospect.The thesis identifies potential speedup, power reduction, and scaling beyond current GPU-based models as appealing possibilities.Source: §7.1 · p. 103

7.2.1. Variational Formulations of Neural Computation ?

  • Neural models range from firing-rate abstractions to spiking and multicompartment models, raising the question of which biological details are essential to explain brain function.The thesis notes proposals for more realistic models and asks which elements of neurobiology can be abstracted away.Source: §7.2.1 · p. 104
  • The thesis asks whether neural dynamics can be derived from variational principles and notes that neuroscience models using such principles have been proposed [Friston, 2010, Betti et al., 2019, Dold et al., 2019, Kendall, 2021].It presents this question in light of its framework for systems governed by variational equations.Source: §7.2.1 · p. 105

7.2.2. SGD Hypothesis of Learning

  • EqProp suggests local SGD-compatible learning rules when neural dynamics have variational principles with sum-separable energy or Lagrangian functions.In energy-based models, gradients are suggested to be encoded in activity differences; in the Lagrangian setting, in trajectory differences [Hinton and McClelland, 1988, Lillicrap et al., 2020].Source: §7.2.2 · pp. 105–106
  • The SGD hypothesis leaves open which loss functions and synaptic-weight metric the brain uses, and whether learning algorithms vary across brain areas.The thesis notes that the hippocampus may use algorithms different from those hypothesized for the cortex.Source: §7.2.2 · p. 106

7.2.3. The Role of Evolution

  • The thesis presents evolution as another optimization process and suggests that evolving inductive biases may complement manually crafting them.It identifies meta-learning and reverse-engineering the genome's developmental program as related paths [Zador, 2019].Source: §7.2.3 · p. 106

7.3. Synergy Between Neuroscience and AI

  • The thesis argues that building or simulating brain-like systems could make theories of the brain more plausible and improve understanding beyond mathematical models alone.It suggests neuromorphic hardware may enable more elaborated neural dynamics and ease information flow between AI and neuroscience.Source: §7.3 · p. 107

Appendix A Gradient Estimators

The appendix introduces one-sided and symmetric gradient estimators and gives a smoothness condition for a lemma about their approximation error.

Appendix A Gradient Estimators

  • The appendix defines one-sided and symmetric gradient estimators for the loss-gradient calculation.The supplied passages introduce both estimators but do not provide enough intact formula text to state their full expressions.Source: Appendix A · p. 119
  • Lemma A.1 assumes three-times differentiability with respect to β and identifies a first-order error term for the one-sided estimator.The passage states that the error term depends on β; it contrasts the symmetric estimator but does not provide an intact complete comparison.Source: Appendix A · p. 119
  • The appendix describes the symmetric estimator as the mean of two estimates and gives a Taylor expansion involving f(β) near β = 0.The supplied expansion is incomplete, so no additional order or quantitative error comparison is stated here.Source: Appendix A · p. 119
Loading 2103.09985v2…