PaperlayerView exact revision on arXiv

Source-linked AI summary

Predictive Coding Approximates Backprop along Arbitrary Computation Graphs

Beren Millidge, Alexander Tschantz, Christopher L. Buckley

arXiv:2006.04182v5cs.LGcs.NE

TL;DR

Backpropagation is powerful but biologically implausible, motivating a local-learning alternative that works beyond multilayer perceptrons. The paper generalizes predictive coding to arbitrary computation graphs and develops CNN, RNN, and LSTM implementations, finding rapid convergence to exact gradients and comparable performance to backprop while noting scalability costs.

  • Problem

    Backpropagation's non-local and biologically implausible requirements have been addressed mainly for multilayer perceptrons, leaving arbitrary computation graphs insufficiently covered.

  • Method

    The paper extends predictive coding to arbitrary computation graphs and applies it to CNNs, RNNs, and LSTMs using local learning rules and mostly Hebbian plasticity.

  • Results

    Predictive coding converges rapidly and robustly to exact backpropagation gradients and achieves comparable performance to backpropagation in core machine-learning architectures.

  • Takeaways & Limitations

    The results provide a recipe for implementing backpropagation through arbitrary computation graphs with local, parallel predictive-coding dynamics.

  • Takeaways & Limitations

    The method uses 100–200 inference iterations, making it substantially more expensive than backpropagation and limiting scalability.

Abstract

from arXiv · show

Backpropagation of error (backprop) is a powerful algorithm for training machine learning architectures through end-to-end differentiation. However, backprop is often criticised for lacking biological plausibility. Recently, it has been shown that backprop in multilayer-perceptrons (MLPs) can be approximated using predictive coding, a biologically-plausible process theory of cortical computation which relies only on local and Hebbian updates. The power of backprop, however, lies not in its instantiation in MLPs, but rather in the concept of automatic differentiation which allows for the optimisation of any differentiable program expressed as a computation graph. Here, we demonstrate that predictive coding converges asymptotically (and in practice rapidly) to exact backprop gradients on arbitrary computation graphs using only local learning rules. We apply this result to develop a straightforward strategy to translate core machine learning architectures into their predictive coding equivalents. We construct predictive coding CNNs, RNNs, and the more complex LSTMs, which include a non-layer-like branching internal graph structure and multiplicative interactions. Our models perform equivalently to backprop on challenging machine learning benchmarks, while utilising only local and (mostly) Hebbian plasticity. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry, and may also contribute to the development of completely distributed neuromorphic architectures.

1 Introduction

The paper motivates extending predictive coding beyond layered networks because automatic differentiation trains arbitrary differentiable computation graphs, while backprop raises biological-plausibility concerns. It introduces predictive coding as a local-learning alternative and states contributions spanning arbitrary graphs and major architectures.

  • Automatic differentiation differentiates arbitrary differentiable computation graphs, extending beyond early neural networks built from matrix multiplications and nonlinearities.
  • Backpropagation is difficult to implement biologically because it appears to require non-local information and identical forward and backward weights.
  • The paper shows predictive coding converges to automatic differentiation across arbitrary computation graphs.
  • The paper implements CNNs, RNNs, and LSTMs with local learning rules and mostly Hebbian plasticity.

2 Predictive Coding on Arbitrary Computation Graphs

The paper generalizes predictive coding from layered hierarchies to supervised arbitrary computation graphs by associating local prediction errors with graph vertices. At equilibrium, these errors and parameter updates reproduce backpropagated gradients under the stated assumptions.

  • A computation graph is a directed acyclic graph whose edges apply elementary functions and whose vertices represent intermediate variables.
  • The generalized framework runs differentiable computation graphs forward to generate predictions, then defines supervised inference over vertex values between fixed data and target nodes.
  • Prediction errors are defined as ϵi = vi −ˆvi, and predictive coding minimizes a variational free-energy objective through vertex dynamics.
  • Parameter dynamics require only locally available vertex values, prediction errors, and child-vertex prediction errors.
  • The algorithm uses a feedforward sweep followed by parallel updates of vertex activities and prediction errors until convergence.
  • The fixed-prediction assumption decouples parent activities from child predictions, separating the global optimization into local vertex problems.
  • At equilibrium, prediction errors follow the same recursive structure as backpropagation's chain-rule gradients.
  • Equilibrium parameter updates exactly match backpropagated parameter gradients.

3 Related Work

Prior work developed biologically plausible approximations to backprop, while this paper extends predictive coding beyond layered hierarchies to arbitrary computation graphs and reports exponential, robust convergence to exact gradients.

  • Predictive-coding alternatives to backprop have explored random fixed feedback weights and learned backward weights.
  • Exponential convergence to exact gradients remains robust at high learning rates.
  • The method is asymptotically and practically rapidly convergent, using local information and largely Hebbian plasticity across standard machine-learning architectures.

4 Results

The experiments test predictive coding on nonlinear, branching computation graphs and translate the framework to CNNs, RNNs, and LSTMs. Across these settings, predictive coding rapidly reaches exact gradients and achieves performance largely indistinguishable from backprop, albeit at higher computational cost.

  • 4.1 Numerical Results: Predictive coding rapidly converges to exact automatic-differentiation gradients for a highly nonlinear, branching test function.With v0 = 5 and θ = 2, convergence remains stable for learning rates up to 0.5.
  • 4.1 Numerical Results: At equilibrium, predictive-coding error units exactly equal backpropagated gradients on arbitrary computation graphs.The derivation also states that this descent requires local connectivity and no separate phases or sequential backward sweep.
  • 4.2 Predictive Coding CNN, RNN, and LSTM: 100x greater computational cost accompanied predictive-coding training, while predictive-coding and backprop accuracy remained largely indistinguishable.
  • 4.2 Predictive Coding CNN, RNN, and LSTM: Predictive-coding CNNs match backprop CNN performance on SVHN, CIFAR10, and CIFAR100 using identical architectures and hyperparameters.The evaluated networks are small basic CNNs rather than state-of-the-art systems.
  • 4.2 Predictive Coding CNN, RNN, and LSTM: Predictive-coding RNNs and LSTMs extend the approach to non-parameter-linear, branching computation graphs and tasks involving name classification and Shakespeare next-character prediction.The recurrent models apply predictive coding to the unrolled computation graph; sequence length 100 still showed rapid convergence to the correct numerical gradient.

5 Discussion

The discussion presents predictive coding as a local, parallelizable approximation to backprop that extends to deep and branching architectures, while identifying scaling and computational-cost boundaries. It also connects gradient recursion to variational inference under a Gaussian generative model.

  • Predictive coding can approximate backprop on arbitrary, deep, branching graphs with rapid convergence and fully parallel updates.The algorithm does not require separate phases and can produce equivalent performance in core machine-learning architectures.
  • Both predictive-coding and backprop networks were relatively small and performed below state-of-the-art results on the presented tasks.The authors attribute this focus to demonstrating theoretical convergence and suggest scaling to more advanced architectures as future work.
  • 100–200 inference iterations make predictive coding substantially more expensive than backprop, limiting scalability despite layer-wise parallelism.One inference iteration costs about as much as a backprop backward pass.
  • Recursive gradient computation is presented as a by-product of variational inference over computation-graph vertices under a hierarchical Gaussian generative model.
  • Gaussian inverse-variance parameters weight gradient contributions by uncertainty and can themselves be learned by gradient descent on free energy.The authors note that this generalization is afforded by the Gaussian generative model.

Appendix A: Predictive Coding CNN Implementation Details

The predictive coding CNN uses convolution-specific prediction and error pathways, local inference updates, and Hebbian weight learning. Despite a relatively small architecture and suboptimal untuned parameters, its gradients closely match backprop, while CNN weight sharing remains biologically implausible.

  • CNN construction: Convolutional outputs are generated from local input patches using shared feature-map weights, and predictive coding augments intermediate variables with same-sized error units.Each output pixel depends on a relatively small input patch, while shared weights impose translational invariance.
  • CNN construction: Prediction errors are transmitted backward through a zero-padded backwards convolution, after which intermediate variables are updated during inference.The backwards convolution is defined to produce correctly sized convolutional error maps.
  • Learning rule: Parameter-linear convolutional layers permit Hebbian weight updates based on multiplication of pre- and post-synaptic potentials.
  • Biological constraint: CNN weight sharing violates locality because shared weights aggregate update contributions across image positions; separate position-specific filters are proposed as a fix.The performance and scalability of this locally connected alternative remain future-work questions.
  • Experimental setup: The experiments used a small two-convolutional-layer network followed by three fully connected layers, with parameters selected without detailed hyperparameter search.The architecture used six and sixteen filters in its convolutional layers and 200, 150, and 10 or 100 output units in its fully connected layers.
  • Results: Within 100 inference iterations, predictive coding converged very precisely to exact backprop gradients, with approximately 100x computational overhead.The divergence between true and approximate gradients remained approximately constant during training.

Appendix B: Predictive Coding RNN

The predictive coding RNN applies the method to an unrolled recurrent computation graph, using backward-through-time inference and Hebbian updates. Its training loss is effectively identical to backprop, but the backward-through-time procedure remains biologically implausible.

  • RNN structure: An RNN maintains a hidden state updated from the current input and previous hidden state, with outputs generated from the hidden state over time.
  • Training procedure: Predictive coding trains the RNN by applying the method to its unrolled feedforward computation graph, analogous to backpropagation through time.
  • Limitation: Backward-through-time updates require retaining the full sequence and delaying updates until sequence end, an implausibility not addressed here.The paper restricts its claim to predictive coding on the unrolled feedforward graph.
  • Training procedure: Inference updates proceed backward through time after true labels are presented, and weights are updated after convergence.
  • Learning rule: Because the RNN feedforward updates are parameter-linear, the proposed weight rules are Hebbian and require multiplication of pre- and post-synaptic potentials.
  • Experimental setup: The RNN was evaluated on character-level name-origin classification using one-hot character inputs, a 256-unit hidden state, and a tanh hidden nonlinearity.
  • Results: The predictive coding and backprop RNNs achieved effectively identical training losses averaged over five seeds.

Appendix C: Predictive Coding LSTM Implementation Details

The LSTM extends predictive coding to a branching, multiplicative computation graph by augmenting the unrolled cell with error units and locally updating vertices and prediction errors. It converges to exact gradients even for deep unrolled graphs, though convergence slows with sequence length and requires substantial computation.

  • LSTM structure: An LSTM provides a branching internal computation graph with hidden and cell states, whose unrolled form receives backward derivatives from later timesteps.
  • Predictive coding conversion: The predictive coding conversion rewires each prediction to the forward function of its parent and computes errors between vertices and predictions.
  • Inference: During inference, inputs and outputs are fixed while vertices and prediction errors are updated according to the predictive coding rule.The augmented graph includes the vertex update rules and can extend to more complex architectures.
  • Gradient convergence: Predictive coding rapidly converged to exact backprop gradients even for unrolled LSTMs with sequence length 100, although convergence was slower than for CNNs or shorter sequences.
  • Gradient convergence: With a fixed 200-iteration budget, gradient divergence increased roughly linearly with sequence length but remained modest and decreased with more iterations.The comparison used an inference learning rate of 0.05.
  • Experimental setup: The experiment used a single LSTM layer for next-character prediction on shuffled 50-character Shakespeare sequences.The predictive coding and backprop networks used the same weight learning rate, while predictive coding required 200 variational iterations.

Appendix D: Derivation of the Free Energy Functional

The appendix derives the variational free-energy for predictive coding on arbitrary computation graphs, using Gaussian assumptions and a Laplace approximation. Analytical treatment of the variational variance reduces optimization to the variational means, while generative precisions can remain learnable.

  • Variational formulation: The derivation starts from variational inference, replacing an intractable posterior divergence with the tractable variational free-energy objective.Minimizing free energy improves the approximate posterior's fit to the true posterior.
  • Generative model: The predictive-coding generative model mirrors the computation graph, assigning each vertex a Gaussian whose mean is predicted from its parent vertices.Inputs are fixed at the start nodes, while output vertices are clamped to the target.
  • Free-energy derivation: The free-energy functional is decomposed into energy and entropy terms, with the entropy term evaluated analytically under Gaussian variational distributions.The energy term requires a Laplace approximation when the variational density is tightly concentrated around its mean.
  • Laplace approximation: The Laplace approximation permits a closed-form optimal variational variance, so optimization can focus on the variational means vi.This removes the variational variances from the remaining optimization problem.
  • Prediction-error objective: The output likelihood acts as an additional prediction error and can be absorbed into the sum of other prediction errors.Unlike the variational variances, the generative-model precisions Σi lack an analytical form and can be optimized directly.

Derivation of Variational Update Rules and Fixed points

The variational dynamics are obtained by reducing free energy to prediction errors and assuming gradient descent on the variational means. Identity precisions make the objective a sum of equally weighted prediction errors.

  • Variational update rules: The derivation first reduces the free-energy functional to a sum of prediction errors before obtaining the variational update rule.The update rule is stated for the variational means vi.
  • Precision assumption: With Σ^-1 = I, each prediction error is ϵi = vi − f(P(vi)), implying equal variance for datapoints and graph vertices.The identity-precision assumption fixes all precisions to the identity.
  • Fixed points: The fixed-point equations follow by assuming that each vertex evolves according to gradient descent on the free energy and solving for the stationary state.The fixed point is obtained from the variational dynamics by setting their derivative to zero.
Loading 2006.04182v5…