PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias

Axel Laborieux, Maxence Ernoult, Benjamin Scellier, Yoshua Bengio, Julie Grollier, Damien Querlioz

arXiv:2006.03824v1cs.NE

TL;DR

EP has theoretical links to BPTT, but conventional gradient estimates have not scaled to hard visual tasks. The paper reduces finite-nudging bias with symmetric nudging and introduces a softmax readout for cross-entropy, reporting CIFAR-10 results near BPTT. Its three-steady-state estimate requires an additional phase.

  • Problem

    Prior EP gradient estimates had not scaled beyond MNIST to hard visual tasks such as deep ConvNet training.

    Source: §1 · p. 2

  • Method

    The paper reduces finite-nudging bias with a three-steady-state symmetric gradient estimate and uses an external softmax readout with cross-entropy loss.

    Source: §3.1 · p. 5, §3.2 · p. 6

  • Results

    11.68% test error on CIFAR-10 is 0.6% worse than BPTT; constant-sign nudging yields 86.64% test error.

    Source: §1 · p. 2

  • Takeaways & Limitations

    EP also trains asymmetric connections on CIFAR-10 with only 1.5% performance degradation relative to symmetric connections.

    Source: §1 · p. 2, §5 · p. 8

    The three-steady-state estimate costs an additional phase, and the free-phase steady state is experimentally needed to initialize the second and third phases.

    Source: §5 · p. 8

Abstract

from arXiv · show

Equilibrium Propagation (EP) is a biologically-inspired algorithm for convergent RNNs with a local learning rule that comes with strong theoretical guarantees. The parameter updates of the neural network during the credit assignment phase have been shown mathematically to approach the gradients provided by Backpropagation Through Time (BPTT) when the network is infinitesimally nudged toward its target. In practice, however, training a network with the gradient estimates provided by EP does not scale to visual tasks harder than MNIST. In this work, we show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon and that cancelling it allows training deep ConvNets by EP. We show that this bias can be greatly reduced by using symmetric nudging (a positive nudging and a negative one). We also generalize previous EP equations to the case of cross-entropy loss (by opposition to squared error). As a result of these advances, we are able to achieve a test error of 11.7% on CIFAR-10 by EP, which approaches the one achieved by BPTT and provides a major improvement with respect to the standard EP approach with same-sign nudging that gives 86% test error. We also apply these techniques to train an architecture with asymmetric forward and backward connections, yielding a 13.2% test error. These results highlight EP as a compelling biologically-plausible approach to compute error gradients in deep neural networks.

1 Introduction

Equilibrium Propagation (EP) offers a biologically plausible, local approach to credit assignment, but prior experiments had not matched deep-learning performance on hard visual tasks. The paper targets finite-nudging bias and introduces changes that improve deep ConvNet training with symmetric or asymmetric connections.

1 Introduction

  • EP's spatially local learning rule, which can also be local in time, motivates its use in energy-efficient neuromorphic hardware.The paper presents biological learning and neuromorphic implementation as motivations for scaling EP.Source: §1 · pp. 1–2
  • Prior EP experiments were limited to MNIST and shallow networks, and had not matched standard deep-learning performance on hard visual tasks.Asymmetric-connectivity proposals also degraded accuracy on MNIST relative to standard EP.Source: §1 · p. 2
  • Unlike Feedback Alignment and Target Propagation, EP routes error feedback through forward neural activity rather than a distinct backward pathway.The paper contrasts this circuit property with approaches whose feedback does not affect forward activity.Source: §1 · p. 2
  • 11.68% test error on CIFAR-10 is 0.6% worse than BPTT, while constant-sign nudging yields 86.64% test error.The paper identifies the three-steady-state gradient estimate as the method behind the stronger result.Source: §1 · p. 2
  • A softmax readout enables EP to optimize cross-entropy rather than the squared-error loss used in previous approaches.The readout keeps the predictor outside the free dynamics of the system.Source: §1 · p. 2
  • EP is adapted to asymmetric forward and backward connections, with a 1.5% performance degradation on CIFAR-10 relative to symmetric connections.The introduction frames this as extending EP to a more biologically plausible topology.Source: §1 · p. 2

2 Background

EP trains convergent recurrent networks by nudging their steady-state dynamics, while BPTT provides a reference gradient in the infinitesimal-nudging limit. The background also introduces convolutional architectures and a more general EP formulation for asymmetric connections.

2.1 Convergent RNNs With Static Input

  • With static input, a convergent RNN evolves to a steady state, and learning minimizes the loss evaluated at that state.EP considers networks whose transition function derives from a scalar primitive.Source: §2.1 · pp. 2–3

2.2.1 Equilibrium Propagation (EP)

  • EP has a free phase followed by a nudged phase, and its learning rule performs stochastic gradient descent as the nudging strength β approaches zero.The nudged dynamics add a loss-gradient term to the state evolution.Source: §2.2.1 · p. 3

2.2.2 Backpropagation Through Time (BPTT)

  • When the free phase has converged, EP gradients match BPTT gradients over the first K nudged time steps in the limit β → 0.The stated condition is that the state has reached its steady state by time T − K.Source: §2.2.2 · p. 3

2.3 Convolutional Architectures for Convergent RNNs

  • The convolutional architecture uses recurrent convolutions and transpose convolutions through time, with class prediction at the final time step.The schematic depicts forward and backward operations separately and uses temporal variations as EP error signals.Source: Figure 1 · p. 4, §2.3 · p. 4

2.4 Equilibrium Propagation with asymmetric synaptic connections

  • EP can use distinct forward and backward connections by replacing the primitive-function dynamics with a more general vector-field formulation.When the transition function derives from a primitive, the vector-field and standard EP estimates agree as β → 0.Source: §2.4 · p. 4

3 Improving EP Training

This section revisits EP gradient estimation and proposes three phases instead of two, alongside a cross-entropy approach and a tailored learning rule for asymmetric connections. Together, these techniques achieve the paper’s best CIFAR-10 performance with symmetric and asymmetric connections.

3 Improving EP Training

  • The section combines three-phase gradient estimation and cross-entropy optimization to achieve the paper’s best CIFAR-10 performance with both symmetric and asymmetric connections.The section introduces the three-phase method and the cross-entropy approach as the techniques behind this result.Source: §3 · p. 5

3.1 Reducing bias and variance in the gradient estimate of the loss function

  • The three-phase estimate uses positive and negative nudges to eliminate the first-order bias induced by finite nudging in the original two-phase estimate.Randomly choosing the sign of β can also cancel the first-order term on average, but the paper reports high variance during training.Source: §3.1 · pp. 5–6

3.2 Changing the loss function

  • The cross-entropy approach makes the prediction a softmax readout rather than part of the network’s state and free dynamics.The readout uses the network state and an output weight matrix; in practice, it reads the last convolutional layer.Source: §3.2 · p. 6, Figure 3 · p. 6

3.3 Changing the learning rule of EP with asymmetric synaptic connections

  • For asymmetric connections, the tailored rule applies the same updates and leakage to forward and backward weights, which naturally converge to identical values during learning.The traditional EP rule can make the forward and backward updates differ; the stated theoretical equivalence between EP and BPTT holds only for symmetric connections.Source: §3.3 · p. 7

4 Experimental results

With the symmetric gradient estimate, EP reaches CIFAR-10 performance close to BPTT; the paper also reports results for cross-entropy readout and asymmetric connections.

4 Experimental results

  • The CIFAR-10 ConvNet uses four 3 × 3 convolutional layers with max pooling, followed by one fully connected layer.Table 1 compares BPTT with EP gradient estimates; Table 2 reports asymmetric-connection results.Source: §4 · p. 7, Table 1 · p. 7, Table 2 · p. 8

4.1 ConvNets with symmetric connections

  • 12.45% test error with the symmetric estimate is only 1.35% worse than BPTT on CIFAR-10.The symmetric estimate consistently reaches this result, unlike randomized-sign nudging, which shows high variance.Source: Table 1 · p. 7, §4.1 · p. 8
  • 21.55% average test error with randomized-sign nudging masks high variance: one of five runs is unstable, while four reach 12.61% test error.The one-sided estimate also produces unstable training; the symmetric estimate is more reliable than randomized nudging.Source: §4.1 · pp. 7–8
  • The softmax readout for cross-entropy narrows EP’s performance gap with BPTT to 0.56% and beats the squared-error setting by 0.77%.Adding dropout at 0.1 after the convolutional layers does not improve EP test error.Source: §4.1 · p. 8

4.2 ConvNets with asymmetric connections

  • Asymmetric connections recover performance to 1.5% degradation versus symmetric connections and 3% versus BPTT.Forward and backward weights are reported as perfectly aligned by epoch 50.Source: Table 2 · p. 8, §4.2 · p. 8

5 Discussion

The discussion argues that reducing EP’s gradient-estimation bias supports scaling to deep ConvNets, including architectures with distinct forward and backward connections. It also notes added computational cost and a possible self-regularizing effect.

5 Discussion

  • Removing the first-order gradient-estimate bias brings EP performance on CIFAR-10 close to BPTT, while BPTT fits training data at least 2.8% better.The authors hypothesize that EP’s residual estimation bias may underlie a self-regularizing effect similar to dropout.Source: §5 · p. 8
  • EP also scales to distinct forward and backward connections, with only 1.5% performance degradation versus the symmetric architecture.The authors frame this as avoiding the biologically implausible requirement of symmetric connectivity.Source: §5 · p. 8
  • The three-steady-state gradient estimate costs one additional phase, and experiments find the free steady state is needed to start both nudged phases.That state is not used to compute the gradient estimate itself.Source: §5 · p. 8
  • Opposite-sign nudging phases could fit naturally on neuromorphic systems that use differential operation to cancel device biases.The paper presents EP as a potential route to local neuromorphic on-chip training.Source: §5 · pp. 8–9

Broader Impact

The broader-impact discussion points to possible long-term relevance for energy-efficient hardware and neuroscience, while noting that outcomes remain uncertain. The appendix defines a BPTT gradient truncated to the final time steps.

Broader Impact

  • Potential long-term impacts include energy-efficient hardware design and insights into brain credit assignment, but positive and negative outcomes cannot yet be stated.The discussion describes these impacts as long term.Source: Broader Impact · p. 9

A Gradients of BPTT

  • The appendix defines ∇BPTT(t) as the gradient computed by BPTT truncated to the final t time steps.It treats the network parameter as shared across time steps and defines sensitivity to a time-indexed parameter while other time-step parameters remain fixed.Source: A Gradients of BPTT · p. 11

B Error terms in the estimates of the loss gradient

The appendices compare one-sided and symmetric gradient estimates, specify their estimation procedures, and document operations and learning rules for convolutional RNNs.

B Error terms in the estimates of the loss gradient

  • The error-term analysis recalls both gradient estimates, expands around zero at −β, and subtracts the expansions to express the symmetric difference.The stated third-derivative condition is needed for the O(β^3) term; assuming only twice differentiability gives o(β^2).Source: B Error terms in the estimates of the… · pp. 11–12

C.1 Random one-sided estimation of the loss gradient

  • Random one-sided estimation chooses the sign of β randomly after the free phase, then performs a single nudged phase.The appendix presents this as an estimation procedure used in this work and earlier work.Source: §C.1 · p. 12

C.2 Symmetric difference estimation of the loss gradient

  • The symmetric-difference procedure stores the free steady state and runs nudged phases at β and −β from that state.The algorithm passage shows returning to the free steady state before the negative-nudging phase.Source: §C.2 · pp. 12–13

D Convolutional RNNs

  • The convolutional-RNN appendix defines layer counts and identifies the input and output layers for squared-error and softmax readouts.It defines total layers as N_tot = N_conv + N_fc.Source: D Convolutional RNNs · p. 13

D.1 Definition of the operations

  • The operations appendix specifies convolution, transpose convolution, tensor dot product, pooling, unpooling, and flattening for convolutional RNNs.Unpooling places values at the max-pooling indices and fills other positions with zeros.Source: §D.1 · pp. 13–14

D.2 Convolutional RNNs with symmetric connections

  • The symmetric-connection appendix writes dynamics and learning rules for squared-error and cross-entropy losses under one-sided and symmetric estimates..Source: §D.2 · p. 14

D.2.1 Squared Error loss

The appendix gives dynamics and EP learning-rule formulations for both squared-error and cross-entropy losses. It distinguishes whether output-layer dynamics derive from the primitive function.

D.2.1 Squared Error loss

  • The squared-error formulation derives output-layer dynamics from the primitive function Φ.The state includes the output as well as hidden layers.Source: §D.2.1 · p. 14
  • Both loss formulations provide learning rules for one-sided and symmetric EP estimators.Source: §D.2.1 · pp. 14–15, §D.2.2 · pp. 15–16

D.2.2 Cross-Entropy loss

  • The cross-entropy formulation specifies dynamics and learning rules separately, with output-layer dynamics not deriving from Φ.Source: §D.2.2 · pp. 15–16

D.2.3 Implementation details in PyTorch.

The implementation section describes differentiating a primitive function to implement standard EP dynamics and estimates. It also presents a Kolen-Pollack-inspired rule for asymmetric convolutional connections.

D.2.3 Implementation details in PyTorch.

  • In standard EP, PyTorch automatic differentiation of the primitive function Φ implements the dynamics and EP estimates, though more slowly than explicit equations.The approach is described as more flexible across architectures because Φ is relatively easy to compute.Source: §D.2.3 · p. 16

D.3 Convolutional RNNs with asymmetric connections

  • Asymmetric forward and backward connections use cross-entropy with a softmax readout, and their dynamics do not derive from one primitive function.The implementation instead differentiates a layer-specific quantity designed to produce the desired dynamics.Source: §D.3 · pp. 16, 18
  • With shared gradient estimates and weight decay, the Kolen-Pollack update makes forward and backward weights converge to the same values.Source: §D.3 · p. 17
  • For asymmetric connections, the paper introduces the Kolen-Pollack Vector Field learning rule (KP-VF) to train convolutional networks.Source: §D.3 · pp. 16–17
  • Inverse pooling makes convolutional-filter updates depend on maximal-element locations, motivating averaging the vector-field estimate before applying the Kolen-Pollack technique.Source: §D.3 · p. 18

E Experimental details

The experimental details specify the CIFAR-10 ConvNet setup and describe additional analyses of random-sign estimates, dropout, activation scaling, and weight alignment.

E Experimental details

  • Figure 4 and Figure 5 plot train and test errors on CIFAR-10 for squared-error and cross-entropy losses, respectively.Both figures average five runs and show ±1 standard deviation.Source: Figure 4 · p. 19, Figure 5 · p. 19

E.1 Environment and hyper-parameters

  • The CIFAR-10 experiments use four convolutional layers with 128, 256, 512, and 512 feature maps, max pooling, data augmentation, and SGD with momentum and weight decay.Each convolution is 3 × 3; the flattened feature vector has size 512.Source: §E.1 · p. 19

E.2 Random-sign estimate variance

  • Two of ten random-sign trials are unstable, supporting the reported high variance of the random-sign estimate.One of five initial random-sign runs collapsed to random guess; the additional trials tested how often this occurred.Source: §E.2 · p. 20, Figure 6 · p. 20, Table 4 · p. 21

E.3 Adding dropout

  • Dropout is applied by zeroing units with probability p and rescaling active units; the experiments use p = 0.1 before the classifier.The same dropped units are used within a training iteration but differ across examples in a mini-batch.Source: §E.3 · p. 21

E.4 Changing the activation function

  • The activation slope is halved to ease signal propagation and prevent saturation during the second phase.The change responds to saturating units observed in earlier ConvNet experiments.Source: §E.4 · p. 21

F Weight alignment for asymmetric connections

  • The weight-alignment analysis compares the angle between forward and backward weights for the original VF rule and the Kolen-Pollack-inspired rule during CIFAR-10 training.Source: F Weight alignment for asymmetric… · p. 21

G Layer-wise comparison of EP estimates

The section presents layer-wise comparisons of EP gradient estimates and BPTT gradients for a five-layer CNN on CIFAR-10.

G Layer-wise comparison of EP estimates

Related papers

Loading 2006.03824v1…