PaperlayerView exact revision on arXiv

Source-linked AI summary

Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation

Benjamin Scellier, Yoshua Bengio

arXiv:1602.05179v5cs.LG

TL;DR

Backpropagation’s special second-phase computation is a biological plausibility concern. The paper introduces Equilibrium Propagation, which uses weak output nudging and the same leaky-integrator computation across phases; it derives error-derivative and weight updates and demonstrates MNIST training. The approach remains constrained by symmetric weights and slow free-phase relaxation.

  • Problem

    Backpropagation requires a special computational circuit and computation in its second training phase, motivating a more biologically plausible framework for energy-based models.

  • Method

    Equilibrium Propagation uses one leaky-integrator computation, weakly nudging output units toward targets so perturbations propagate through an energy-based network.

  • Results

    The method propagates error derivatives to synaptic weights and achieves 0.00% training error with 2%–3% generalization error on MNIST, depending on architecture.

  • Takeaways & Limitations

    Equilibrium Propagation makes a Backpropagation-like mechanism more plausible for brains by using the same neural computation for inference and error back-propagation.

  • Takeaways & Limitations

    The approach requires symmetric weights, and the free-phase iteration count grows rapidly with network depth, slowing training.

Abstract

from arXiv · show

We introduce Equilibrium Propagation, a learning framework for energy-based models. It involves only one kind of neural computation, performed in both the first phase (when the prediction is made) and the second phase of training (after the target or prediction error is revealed). Although this algorithm computes the gradient of an objective function just like Backpropagation, it does not need a special computation or circuit for the second phase, where errors are implicitly propagated. Equilibrium Propagation shares similarities with Contrastive Hebbian Learning and Contrastive Divergence while solving the theoretical issues of both algorithms: our algorithm computes the gradient of a well defined objective function. Because the objective function is defined in terms of local perturbations, the second phase of Equilibrium Propagation corresponds to only nudging the prediction (fixed point, or stationary distribution) towards a configuration that reduces prediction error. In the case of a recurrent multi-layer supervised network, the output units are slightly nudged towards their target in the second phase, and the perturbation introduced at the output layer propagates backward in the hidden layers. We show that the signal 'back-propagated' during this second phase corresponds to the propagation of error derivatives and encodes the gradient of the objective function, when the synaptic update corresponds to a standard form of spike-timing dependent plasticity. This work makes it more plausible that a mechanism similar to Backpropagation could be implemented by brains, since leaky integrator neural computation performs both inference and error back-propagation in our model. The only local difference between the two phases is whether synaptic changes are allowed or not.

1 Introduction

The paper introduces Equilibrium Propagation as a biologically motivated learning framework for energy-based models. It uses one computational circuit across training phases, propagates error derivatives through network dynamics, and is experimentally trainable on MNIST.

  • Motivation and framework: Equilibrium Propagation trains a broad class of energy-based models using one computational circuit and one type of computation in both training phases.The framework applies beyond regular multilayer networks to energy-based models including the continuous Hopfield model.
  • Error propagation: During the second phase, output perturbations propagate backward through hidden layers, implicitly carrying error derivatives.This extends earlier activity-level results to backward propagation through the network.
  • Theoretical contribution: The general formulation covers arbitrary dynamics whose fixed points or stationary distributions correspond to minima of an energy function.This makes the framework broader than the continuous Hopfield prototype.
  • Theoretical contribution: The learning rule computes the gradient of a well-defined objective based on local perturbations, addressing theoretical issues in related energy-based learning algorithms.The authors contrast it with recurrent back-propagation and contrastive Hebbian learning.
  • Experiments: 0.00% training error was achieved on MNIST, while generalization error ranged between 2% and 3% depending on architecture.The experiments used recurrent networks with 1, 2, and 3 hidden layers.

2 The Continuous Hopfield Model Revisited: Equilibrium Propagation as a More Biologically Plausible Backpropagation

Equilibrium Propagation embeds target influence as a weak external potential in a symmetric recurrent energy-based network. Relaxation first produces a prediction, then a small output nudge propagates backward and yields weight-error derivatives through local dynamics.

  • Architecture: Symmetric recurrent connections support arbitrary architectures, while layered networks make the correspondence with backpropagation more apparent.The experimental architecture has no skip-layer or within-layer lateral connections.
  • Total energy: The total energy F combines internal network energy E with external potential βC, allowing outputs to be weakly clamped toward targets.β controls whether and how strongly the target influences the outputs.
  • Neuronal dynamics: The model is a continuous-time recurrent system whose energy decreases during relaxation until the network reaches a fixed point.The state variable excludes always-clamped inputs and contains hidden and output units.
  • Free and weakly clamped phases: In the free phase, the network relaxes with β = 0 and reads its prediction at the output fixed point; in the second phase, β > 0 nudges outputs toward targets.The resulting perturbation propagates backward through hidden layers to a nearby weakly clamped fixed point.
  • Backpropagation of errors: The second-phase perturbation corresponds to propagated error derivatives, including derivatives with respect to synaptic weights in the limit β → 0.The weight update applies beyond the layered architecture considered in earlier work.
  • Biological connection: Leaky-integrator computation performs both inference and error back-propagation, while the update rule can be interpreted as continuous-time integration of an STDP rule.The interpretation assumes symmetric weights and follows the path from the free to weakly clamped fixed point.

3 A Machine Learning Framework for Energy Based Models

The framework defines predictions, costs, objectives, and gradients through an energy function and trains models by comparing free and weakly clamped fixed points. Equilibrium Propagation uses a small cost-weighted perturbation to estimate the objective gradient, while remaining applicable to broader energy-based dynamics.

  • General framework: Equilibrium Propagation generalizes the continuous Hopfield setting into a framework for energy-based models whose fixed points or stationary distributions minimize an energy function.The framework is presented as an analogue of Backpropagation for energy-based models, beyond the continuous Hopfield model.
  • Training objective: The state s moves toward low-energy configurations, while the cost function C measures state quality and training aims to align low energy with low cost.The state, data point v, and parameters θ are real-valued vectors; the cost may also include parameter regularization.
  • Training objective: The model prediction is an energy minimum or fixed point defined implicitly by θ and v, and the objective is the cost evaluated at that free fixed point.The cost function applies to arbitrary states, whereas the objective function evaluates the cost associated with the model’s free fixed point.
  • Equilibrium Propagation: The total energy F(θ, v, β, s) := E(θ, v, s) + β C(θ, v, s) introduces an influence parameter that defines free and nudged fixed points.β = 0 yields the free phase, while β ≠ 0 yields a nudged phase; the training procedure settles the system in each phase and collects energy derivatives.
  • Equilibrium Propagation: A small positive perturbation moves the system from the free fixed point to a nearby weakly clamped fixed point with lower cost, enabling a contrastive update that decreases the nudged state’s energy and increases the free state’s energy.This local perturbation underlies the gradient formula and makes the weakly clamped state a slightly better state in prediction-error terms.
  • Implementation considerations: The gradient is estimated numerically from the two fixed points, making digital implementations approximate and potentially inefficient because inference requires numerical energy minimization.The authors expect analog circuits could exploit the framework more efficiently, while discrete-time simulations can require a fairly long free phase.

4 Related Work

Equilibrium Propagation is compared with Contrastive Hebbian Learning, Boltzmann-machine learning, and recurrent back-propagation through their objectives, phases, and neural computations. The comparisons emphasize local perturbations, flexible cost functions, and a single computation type across phases.

  • Equilibrium Propagation: Equilibrium Propagation uses weakly clamped outputs to encode error derivatives and the objective-function gradient during backward propagation.The perturbation begins at the output layer and propagates through hidden layers.
  • Contrastive Hebbian Learning: Equilibrium Propagation avoids CHL’s different-mode problem because local perturbations keep the weakly clamped fixed point close to the free fixed point.The paper states that this avoids negative objective values, inconsistent updates, and deteriorating learning in that case.
  • Contrastive Hebbian Learning: Unlike Contrastive Hebbian Learning, Equilibrium Propagation permits independently chosen cost and energy functions.CHL’s contrastive function is determined by the energy function.
  • Boltzmann Machine Learning: Compared with Contrastive Divergence, Equilibrium Propagation computes the correct objective gradient in the β → 0 limit, whereas CD provides a biased estimator.The paper also states that CD1 is not the gradient of any objective function and may cycle in pathological cases.
  • Recurrent Back-Propagation: Recurrent back-propagation uses a linearized second-phase computation, whereas Equilibrium Propagation uses the same kind of neural computation in both phases.This difference is presented as relevant to biological plausibility.

5 Implementation of the Model and Experimental Results

The model is implemented as a recurrent network trained on MNIST with free and weakly clamped phases, achieving zero training error and 2–3% generalization error across architectures. Practical training uses iterative energy minimization, layer-specific updates, and heuristics to reduce computation, but free-phase relaxation becomes increasingly costly with depth.

  • Experimental results: The MNIST experiments train recurrent networks with 1, 2, and 3 hidden layers, achieving 0.00% training error and 2–3% generalization error depending on architecture.The networks use symmetric connections, no skip-layer connections, and no lateral connections within layers.
  • Training procedure: Training alternates free and weakly clamped phases, collecting pairwise unit activities before updating each synapse.The weakly clamped phase uses a small β, and the synaptic update is scaled by 1/β.
  • Training and prediction: Predictions are read from the free fixed point by selecting the output unit with maximal activation.The model does not impose a softmax constraint; its objective compares the prediction with the one-hot target.
  • Implementation: The state dynamics are implemented by discretizing the energy-driven differential equation, with each update corresponding to a gradient-descent step on total energy.A hard-sigmoid activation keeps hidden and output states within the domain 0 ≤ s_i ≤ 1.
  • Computational considerations: The step size has little influence within 0 < ϵ < 1, whereas free-phase relaxation requires many iterations and grows rapidly with network depth.Persistent hidden states are reused between epochs to help reduce the cost of long free-phase relaxations.

6 Discussion, Looking Forward

The discussion identifies biological and computational limitations of Equilibrium Propagation, including symmetric weights, lengthy relaxation, restricted plasticity modeling, and static inputs. It also points to autoencoder initialization and online gradient estimation as possible directions forward.

  • Biological constraints: Symmetric weights remain a biological concern, and the paper leaves open how learning might produce symmetry or eliminate the requirement.The model units may represent groups of neurons rather than individual biological neurons.
  • Computational constraints: Lengthy relaxation, especially in the free phase, remains a practical issue that may be reduced by layerwise autoencoder training.The proposed rationale is that feedback paths can agree with states established by an initial feedforward phase.
  • Synaptic plasticity: The proposed synaptic rule differs from conventional Hebbian rules because it uses the temporal derivative of postsynaptic activity rather than its activity level.The comparison is made with rules such as BCM.
  • Synaptic plasticity: The rate-based model does not consider synaptic cooperativity, depolarization, or spike-timing statistics involving triplets and quadruplets.The paper identifies these effects as subjects for future work.
  • Temporal inputs: The approach is more plausible for static inputs than for time-varying inputs, while online gradient estimators trade stored past states for noisier gradients.The discussion relates this boundary to the greater implausibility of back-propagation through time.

A Proof of the Gradient Formula (Theorem 1)

The proof derives Equilibrium Propagation’s gradient formula from properties of the energy at fixed points. It establishes that the objective gradient with respect to parameters equals the response of the weakly clamped system as β approaches zero.

  • Assumptions: The proof assumes a twice differentiable energy function and fixed points satisfying conditions from the implicit function theorem.The deterministic lemma treats the fixed point as a continuously differentiable function of the parameters and clamping factor.
  • Proof strategy: The proof uses the total energy evaluated at the fixed point and the equality of its cross-derivatives.Chain-rule calculations connect derivatives with respect to parameters and β.
  • Conclusion: The theorem is obtained by combining the lemma’s derivative relation with the objective-gradient interpretation at β = 0.The appendix also proves the result through a constrained-optimization formulation.
  • Gradient identity: At β = 0, the left-hand side of the central identity represents the gradient of the objective with respect to θ.The proof identifies this term using the objective-function definition and fixed-point relations.
  • Gradient identity: The right-hand side represents the derivative of the energy-related function with respect to the clamping factor.Combining the two sides yields the theorem’s desired gradient formula.

B Reformulation of the Training Objective as a Constrained Optimization Problem

The training objective can be reformulated as a constrained optimization problem in which the network state is an implicit parameter linked to θ by the fixed-point constraint. A Lagrangian then recovers the free fixed point and supports a parameter-gradient update.

  • Constrained formulation: The reformulation treats the network state s as an optimization variable because prediction depends implicitly on θ through the fixed-point constraint.This differs from conventional algorithms where prediction is an explicit function of parameters.
  • Lagrangian: A Lagrangian is introduced with parameter θ, state s, and multiplier vector λ, while the data point v is held fixed.The construction converts the fixed-point-constrained objective into a standard constrained-optimization form.
  • Parameter update: After solving for the stationary state and multipliers, one gradient-descent step on the Lagrangian with respect to θ yields the parameter update.The reformulation therefore recovers the training gradient while enforcing the fixed-point constraint.
  • Stationarity conditions: The first stationarity condition identifies s* with the free fixed point.Substituting this condition into the second stationarity condition determines the corresponding multiplier relation.
  • Gradient recovery: The multiplier solution is obtained by expressing the conditions through total energy F and comparing them with the fixed-point derivative identity.This connects the constrained formulation to the earlier gradient proof.

C Stochastic Framework

The stochastic framework extends Equilibrium Propagation from deterministic dynamics to noisy network dynamics. It uses weakly clamped and free stationary distributions to estimate gradients of an objective function.

  • Stochastic dynamics: Gaussian noise added to the deterministic gradient system produces Langevin dynamics for the stochastic framework.The stochastic setting is presented as a possible basis for machine learning with spiking networks.
  • Stochastic dynamics: For fixed parameters and clamping strength, Langevin dynamics converges to a Boltzmann distribution with temperature T = 1/(2σ^2).The Brownian term models network noise, and the convergence follows from the Fokker–Planck equation.
  • Phase sampling: The free phase samples from p0 and the weakly clamped phase samples from pβ after the dynamics run long enough.The free phase uses β = 0, while the weakly clamped phase uses β > 0.
  • Gradient estimation: The stochastic theorem shows that the phase-based estimator is an unbiased estimator of the gradient of a defined objective function.Theorem 4 generalizes Theorem 1 to the stochastic framework.
  • Gradient estimation: A stochastic proposition further relates the derivative of the objective to expected cost under the nudged distribution.The nudged distribution pβ is considered for small β > 0.
Loading 1602.05179v5…