PaperlayerView exact revision on arXiv

Source-linked AI summary

Agnostic Physics-Driven Deep Learning

Benjamin Scellier, Siddhartha Mishra, Yoshua Bengio, Yann Ollivier

arXiv:2205.15021v1cs.LG

TL;DR

The paper asks how physical systems might perform gradient-based training rather than leaving gradient training to digital models. It introduces Æqprop, which uses energy minimization, homeostatic control, and output nudging to produce gradient-descent updates, and proves the parameter change matches one gradient-descent step.

  • Problem

    Physical systems can perform inference efficiently, but gradient training is still done in silico on a digital model, motivating learning algorithms integrated with system physics.

    Source: §1 · p. 1

  • Method

    Æqprop treats parameters as energy-minimizing floating variables held near current values by control knobs, then nudges output so parameters move toward an updated value.

    Source: §1 · p. 2

  • Results

    The second-phase parameter change corresponds to one gradient-descent step on the loss, while Æqprop's Lyapunov function improves monotonically along its trajectory.

    Source: §1 · p. 2, §7 · p. 12

  • Takeaways & Limitations

    Æqprop in principle broadens hardware for statistical learning and suggests homeostatic control plus output nudging can support genuine gradient learning in natural physical systems.

    Source: §7 · p. 12

    Æqprop requires parameter homeostatic control and smooth local energy minimizers; with nonunique minimizers, training must remain around one mode.

    Source: §7 · p. 12

Abstract

from arXiv · show

This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging towards the correct response. In Aeqprop, the specifics of the system do not have to be known: the procedure is based only on external manipulations, and produces a stochastic gradient descent without explicit gradient computations. Thanks to nudging, the system performs a true, order-one gradient step for each training sample, in contrast with order-zero methods like reinforcement or evolutionary strategies, which rely on trial and error. This procedure considerably widens the range of potential hardware for statistical learning to any system with enough controllable parameters, even if the details of the system are poorly known. Aeqprop also establishes that in natural (bio)physical systems, genuine gradient-based statistical learning may result from generic, relatively simple mechanisms, without backpropagation and its requirement for analytic knowledge of partial derivatives.

1 Introduction

The paper addresses the difficulty of training physical learning hardware when Eqprop requires explicit energy-function knowledge, physical parameter updates, and storage of an earlier equilibrium. Æqprop uses energy minimization, homeostatic control, and output nudging to produce gradient-based parameter updates through physical manipulation.

1 Introduction

  • The motivation is to integrate learning algorithms with physical laws, since training and running scaled neural networks on GPUs is slow and energy intensive.The passage attributes these inefficiencies to the von Neumann bottleneck between processing and memory.Source: §1 · p. 1
  • Æqprop physically adjusts parameters in proportion to their gradients, while avoiding explicit energy-function knowledge, first-equilibrium storage, and a separate parameter-update mechanism.The approach treats parameters as floating variables that minimize system energy and couples them to control knobs that hold them near their current values during equilibration.Source: §1 · pp. 1–2
  • The procedure uses an unnudged inference phase followed by a phase that nudges the output toward its desired response while allowing parameters to evolve.Inputs and controls are fixed during the second phase, and the parameters move toward new values.Source: §1 · p. 2
  • Each training sample produces a first-order gradient step, unlike zero-order methods such as evolutionary strategies.The update follows the gradient of the error on the given sample without explicitly computing gradients.Source: §1 · p. 2

2 Æqprop: an Agnostic Physical Procedure for Gradient Descent

Æqprop uses physical equilibration and external manipulations to simulate stochastic gradient descent without explicit knowledge of the system’s energy function or gradients. Its two-phase procedure holds parameters fixed during inference, then nudges the output and lets parameters evolve to make a gradient step.

2 Æqprop: an Agnostic Physical Procedure for Gradient Descent

  • The learning objective is an expected cost over input-output examples, with the physical system’s state encoding a prediction from parameters and input.The method assumes the system’s energy need not be known explicitly, but must be sufficiently expressive to realize desired behaviors through parameter adjustment.Source: §2 · pp. 2–3
  • The operator physically equilibrates the system to compute its response, with parameters themselves included among variables that follow energy minimization.The procedure assumes the operator can clamp inputs, read outputs, nudge toward desired outputs, and control parameters through homeostatic control knobs.Source: §2 · p. 3
  • In the first phase, inputs are presented while controls are adjusted to keep parameters fixed; in the second, controls are clamped, the output is nudged, and the system reaches a new equilibrium.The resulting parameter value is read after the second equilibration; inference after training can proceed without nudging.Source: §2 · pp. 3–4
  • Under technical assumptions and small ε and β, Æqprop performs a stochastic gradient-descent step for each input-output pair with step size εβ, without using the energy function or gradients explicitly.Extensions allow one of ε or β to be small and permit non-quadratic control coupling, yielding a Riemannian SGD step.Source: §2 · p. 4

3 Monotonous Improvement: A Lyapunov Function for Æqprop

For nonzero nudging and control coupling, Æqprop has a Lyapunov function that remains close to the true loss and decreases on the current training example. With sufficiently representative large batches, its averaged counterpart can serve as a monotonically improving proxy for the objective.

3 Monotonous Improvement: A Lyapunov Function for Æqprop

  • For any ε and β, the Lyapunov function Lβ decreases along an Æqprop step on the current example and approaches the true loss as β tends to zero.The guarantee concerns the current example; standard SGD does not satisfy even this per-example property.Source: §3 · pp. 4–5
  • When batches are sufficiently representative of the training set, the averaged Lyapunov function Jβ decreases monotonically along the learning procedure.In this regime, the Lyapunov function depends less on the batch and serves as a proxy for the objective.Source: §3 · p. 5

Define the Lyapunov function

  • The theorem establishes the Lyapunov property along the Æqprop trajectory for any β > 0, including values far from the infinitesimal-nudging regime and regardless of ε.The passage introduces this result as the theorem for defining the Lyapunov function.Source: Define the Lyapunov function · p. 5

4 Optimistic Æqprop, Pessimistic Æqprop, and Centered Æqprop

Æqprop variants alter the nudging schedule to change the loss bound they optimize. The pessimistic variant decreases an upper bound on true loss, while the centered variant more closely approximates the loss for small β.

4 Optimistic Æqprop, Pessimistic Æqprop, and Centered Æqprop

  • Centered Æqprop uses −β/2 in the first phase and β/2 in the second, making its Lyapunov function O(β2)-close to the true loss instead of O(β).Its test-time interpretation is a mixture of positive and negative nudging.Source: §4 · p. 5
  • Pessimistic Æqprop optimizes an upper bound of the true loss for each sample, whereas optimistic and pessimistic Lyapunov functions bound sample loss.Pessimistic Æqprop uses negative nudging in the first phase and zero nudging in the second, corresponding to an assumption of adversarial nudging at test time.Source: §4 · pp. 5–6
  • Negative β requires care when the cost is unbounded, because the energy may decrease without bound as the system output grows.The passage says this can be corrected by ensuring the main energy grows sufficiently for large outputs.Source: §4 · p. 6

5 A Numerical Illustration

The numerical studies use energy minimization and homeostatic control to simulate Æqprop, then test its behavior in linear regression and Hopfield-like networks. They report learning on MNIST and FashionMNIST, and identify settings where Æqprop variants differ in stability and error.

5.1 Simulating Convergence to Equilibrium and Homeostatic Control

  • The linear-regression simulation uses energy relaxation by gradient descent and a proportional controller on u; the Hopfield-like network simulations instead use layerwise energy minimization and algebraically computed control values.The linear-regression setup uses 50 equilibration steps; the real-dataset simulations use these algebraic shortcuts because explicit physical simulation was slow.Source: §5 · p. 6, §5.1 · p. 7

5.2 A Simple Linear Regression Example

  • The experiments compare Æqprop variants with SGD on a linear model using Fourier features to approximate random degree-10 polynomials, and test dense and convolutional Hopfield-like networks on real datasets.The linear-regression comparison uses SGD with learning rate εβ; the Hopfield-like networks are trained on MNIST and FashionMNIST.Source: §5.2 · p. 7, §5.3 · p. 8
  • For β = 0.1, Centered Æqprop is virtually indistinguishable from SGD, Pessimistic Æqprop has slightly lower error, and Æqprop has slightly higher error.At β = 0.01, the curves are virtually indistinguishable.Source: §5.2 · p. 7
  • At β = 0.5, Centered Æqprop remains close to SGD; Pessimistic Æqprop performs best for small ε but becomes less numerically stable as ε grows.For β = 1.5 with an added ∥s∥2 energy term, Pessimistic and Centered Æqprop become stable again, and all Æqprop variants seem to work where SGD is unstable, though convergence is slower.Source: §5.2 · p. 8, Figure 3 · p. 9
  • At β = 0.5 and ε = 0.5, Æqprop optimizes well while SGD, Pessimistic Æqprop, and Centered Æqprop become unstable.The authors characterize this as the most aggressive setting and conclude that Æqprop seems more stable than SGD there.Source: §5.2 · p. 8

5.3 Hopfield-Like Networks and Real Datasets

  • Æqprop learns on MNIST and FashionMNIST with relatively large β; Centered Æqprop seems to offer the best precision.The comparison covers dense and convolutional Hopfield-like networks; the autodiff baseline is numerically unstable for reasons the authors did not identify.Source: §5.3 · p. 10, Table 1 · p. 10

6 Related Work

Related work spans explicit digital or hybrid approaches and implicit approaches that use physical laws for inference and training. Æqprop builds on Eqprop while addressing its requirements for known energy functions, stored equilibria, and parameter-update mechanisms.

6 Related Work

  • Existing work on fast, energy-efficient learning systems includes both improved digital hardware for existing algorithms and new algorithms for energy-efficient hardware.The paper organizes this literature by whether the physical computation has an explicit model.Source: §6 · p. 10
  • Explicit approaches include specialized digital processors and analog crossbar arrays, while physics-aware training uses physical inference but trains parameters from an explicit digital model.Analog-device mismatches require mixed digital/analog hardware for the crossbar approach.Source: §6 · p. 11
  • Implicit approaches use underlying physical laws for both training and inference, often characterized by energy minimization; Eqprop was developed for this setting.Eqprop has been applied to Hopfield networks, nonlinear resistive networks, and other physical systems.Source: §6 · p. 11
  • Æqprop addresses three limitations attributed to Eqprop and coupled learning: explicit energy-function knowledge, storage of the first equilibrium state, and additional parameter-update mechanisms.Continual Eqprop updates parameters during its second phase but uses ad hoc parameter dynamics; another approach uses a physical quantity such as chemical concentration as an error signal.Source: §6 · p. 11

7 Discussion, Limitations, and Conclusion

Æqprop combines energy minimization, homeostatic control, and output nudging to produce gradient-descent learning without explicit gradient computation. The discussion presents its theoretical and numerical support, while identifying assumptions that bound its application.

7 Discussion, Limitations, and Conclusion

  • Æqprop performs stochastic gradient descent without explicitly computing gradients, using energy minimization, homeostatic control, and nudging toward the desired output.The method requires neither explicit knowledge of the underlying energy function nor a separate mechanism for updating parameters.Source: §7 · pp. 11–12
  • A Lyapunov function improves monotonically along the Æqprop trajectory, suggesting greater robustness with non-infinitesimal step sizes than standard SGD.The pessimistic variant optimizes an upper bound of the true loss, while the centered variant provides a better second-order approximation.Source: §7 · p. 12
  • Numerical examples show gradient-descent learning with linear regression and Hopfield-like networks on MNIST and FashionMNIST.Source: §7 · p. 12
  • Æqprop assumes energy-minimizing systems and output nudging; when minimizers are non-unique, training can be perturbed if the system switches modes.The energy must have smooth local minimizers, but their uniqueness is not required.Source: §7 · p. 12
  • Homeostatic control is a limiting assumption, although control knobs need only maintain the parameters in stasis.The coupling energy need not be quadratic, and relatively large coupling values are allowed.Source: §7 · p. 12
  • The paper argues that homeostatic control and output nudging are sufficient, in principle, for natural physical systems to exhibit genuine gradient-descent learning.Source: §7 · p. 12

A A Generalization of Theorem 1: Æqprop with Large ε or β, Centered and Pessimistic Æqprop

The generalization characterizes Æqprop with finite nudging and coupling parameters as a Riemannian stochastic gradient step on a related Lyapunov function. It also states the technical assumptions behind the results and relates that function to the ordinary loss.

A A Generalization of Theorem 1: Æqprop with Large ε or β, Centered and Pessimistic Æqprop

  • For ε > 0 and β2 > β1, Æqprop has a Lyapunov property and, as ε or β2 − β1 tends to zero, its update approximates Riemannian SGD on Lβ1;β2.When both nudging values tend to zero, the Lyapunov function tends to the ordinary loss; the pessimistic variant optimizes an upper bound.Source: A A Generalization of Theorem 1:… · pp. 15–16
  • At fixed nonzero ε, the preconditioning metric depends on the sample and may bias expected gradients; this bias disappears as ε tends to zero.The metric can also depend on the desired output when β1 is nonzero.Source: A A Generalization of Theorem 1:… · p. 16

B.2 Technical Assumption: Smooth, Strict Energy Minimizers

  • The technical assumptions require smooth functions and strict local minima, plus controls that can hold parameters at equilibrium as ε varies.A sufficient condition includes equal parameter and control dimensions and an invertible mixed derivative of the coupling energy.Source: §B.2 · pp. 18–19

B.3 Relationships Between the Loss, Lyapunov Function, and Energy

  • The loss-related quantity C(sβ(θ)) is non-increasing in β, and averaging it over the nudging interval yields the Lyapunov function.The associated function F(β, θ) is concave.Source: §B.3 · p. 21

B.4 Proof of the Lyapunov Property (Theorem 2)

The proof compares energy-regularized objectives at two nudging values to establish monotonic Lyapunov improvement, then uses a technical lemma to control the small-parameter regime.

B.4 Proof of the Lyapunov Property (Theorem 2)

  • The proof compares minimizers at two nudging values, combines the resulting inequalities, and invokes Theorem 8 to establish the claimed Lyapunov improvement.The comparison is set up for β2 > β1, with ε and the control value fixed.Source: §B.4 · p. 21

B.5 A Technical Lemma

  • When ε and β both tend to zero, the difference between the two equilibrium states is O(εβ).The proof notes that the difference is separately O(β) and O(ε) before establishing the stronger joint bound.Source: §B.5 · p. 22
  • The lemma’s O(εβ) bound has a uniform constant near u0 when applied to uε, by continuity of the functions involved.This supports applying the lemma even though uε varies with ε.Source: §B.5 · p. 22

B.6 Proof of the Riemannian SGD Property (Theorem 3)

The proof develops the update expression used for the Riemannian SGD property and connects its limiting control value to the preceding iterate.

B.6 Proof of the Riemannian SGD Property (Theorem 3)

  • The proof establishes the update expression for θt − θt−1 as part of the Riemannian SGD property.The passage states this is the remaining part of Theorem 3 being proved.Source: §B.6 · p. 23
  • The argument treats the regime where either ε or β tends to zero, or both do, under the assumptions of Section B.2.It assumes β1 = 0 and β2 = β > 0.Source: §B.6 · p. 23
  • As ε tends to zero, uε tends to u0, defined so that minimizing U(u0, ·) yields the preceding iterate θt−1.The proof uses this limit to complete the last claim of Theorem 3.Source: §B.6 · p. 23

B.7 Proof of the SGD Property with Quadractic Coupling and ε, β →0 (Theorem 1)

This proof addresses the SGD property for quadratic coupling as ε and β approach zero, including related results for pessimistic and centered Æqprop.

B.7 Proof of the SGD Property with Quadractic Coupling and ε, β →0 (Theorem 1)

  • For optimistic Æqprop, the displayed expansion includes a gradient term and a remainder of O(εβ² + ε²β).The passage presents this expansion in the regime where ε and β both tend to zero.Source: §B.7 · p. 24
  • With quadratic control energy U(u, θ) = ∥u − θ∥²/2, the proof specifies the corresponding matrix M0.The matrix expression itself is not included in the passage.Source: §B.7 · p. 24
  • Similar results hold for pessimistic and centered Æqprop, whose losses satisfy L−β;0(θ) = L(θ) + O(β) and L−β/2;β/2(θ) = L(θ) + O(β²), respectively.These loss approximations are stated in Proposition 9.Source: §B.7 · p. 24

C Simulation Details

The simulations compare three Æqprop variants with autodiff on MNIST and FashionMNIST using dense and convolutional Hopfield-like networks. The appendix details the training phases, energy minimization, convergence criterion, and simulation settings.

C Simulation Details

  • The experiments use MNIST and FashionMNIST, each with 60,000 training and 10,000 test images of 28 × 28 grayscale pixels.MNIST contains handwritten digits; FashionMNIST contains ten fashion-product categories.Source: C Simulation Details · p. 25
  • Energy minimization alternates exact variable relaxations because the global energy is quadratic in each layer and parameter.Layers and parameters are relaxed until a minimum is reached; convergence uses the L1 change in layer states and an adaptive threshold initialized at 10^-3.Source: C Simulation Details · p. 25
  • The training procedure uses zero-nudging homeostasis, followed by variant-specific nudging phases and a final phase with clamped control knobs.Pessimistic and centered Æqprop add an initial homeostatic phase at −β and −β/2, respectively.Source: C Simulation Details · p. 25
  • The appendix provides simulation hyperparameters and initialization schemes, and reports that parameter choices were based on trial and error.Dense runs averaged 6 hours per run and convolutional runs 30 hours per run on a single GPU.Source: Table 2 · p. 26, C Simulation Details · p. 26
  • The benchmark compares optimistic, pessimistic, and centered Æqprop with autodiff, whose gradients are computed by unfolding the free-phase minimization.The autodiff baseline takes one gradient-descent step per parameter with step size βεk.Source: C Simulation Details · p. 26

D From Eqprop to Agnostic Eqprop

Equilibrium propagation estimates loss gradients by comparing equilibria before and after nudging the system toward a target. Agnostic Eqprop addresses physical implementation challenges by treating parameters as floating variables that minimize system energy, establishing SGD and a more general Riemannian SGD property.

D From Eqprop to Agnostic Eqprop

  • Eqprop estimates loss gradients by measuring the system at equilibrium with no nudging and after a target-dependent nudge, then using finite differences to update parameters.The method adds βC(s, y) to the energy and lets the system settle in each phase.Source: D From Eqprop to Agnostic Eqprop · p. 29
  • Physical Eqprop training requires energy-function derivative knowledge, storage of first-phase measurements, and a separate physical procedure for updating parameters.Agnostic Eqprop is presented as resolving all three challenges.Source: D From Eqprop to Agnostic Eqprop · p. 29
  • For sufficiently small positive β, the nudged equilibrium has lower cost than the unnudged equilibrium: C(sβ) < C(s0).The paper derives the equilibrium’s change with β by differentiating its equilibrium condition.Source: D From Eqprop to Agnostic Eqprop · p. 30
  • Agnostic Eqprop views parameters as floating variables minimizing system energy, and the paper establishes SGD and a more general Riemannian SGD property.The method is presented as addressing the physical parameter-update challenge as well as the need to measure and store derivatives.Source: D From Eqprop to Agnostic Eqprop · pp. 29–30
  • Unlike Agnostic Eqprop, Eqprop’s Lyapunov function need not decrease after each training step when the learning rate is too large.The passage compares this behavior with standard SGD.Source: D From Eqprop to Agnostic Eqprop · p. 30
  • Coupled learning clamps the output toward the target, but it generally does not compute the loss gradient, even as β approaches zero, except when the energy Hessian is the identity.The stated setup uses squared-error cost and assumes the energy Hessian is approximately the identity to derive the clamp rule.Source: D From Eqprop to Agnostic Eqprop · p. 30
  • Eqprop’s differentiation method is also used by contrastive meta-learning to compute gradients of meta-parameters.The passage points to a review of implicit-gradient methods for bilevel optimization.Source: D From Eqprop to Agnostic Eqprop · p. 30

Related papers

Loading 2205.15021v1…