PaperlayerView exact revision on arXiv

Source-linked AI summary

The Forward-Forward Algorithm: Some Preliminary Investigations

Geoffrey Hinton

arXiv:2212.13345v1cs.LG

TL;DR

The paper addresses whether neural networks can learn without backpropagation when forward computations are unknown or data must be processed in a pipeline. It introduces Forward-Forward, which trains layers with positive and negative forward passes, and reports promising results on small problems while leaving large-scale performance and activation choices open.

  • Problem

    Backpropagation is poorly suited to cortical learning, real-time pipelining, and networks containing unknown nonlinearities.

  • Method

    Forward-Forward replaces backpropagation’s forward and backward passes with two forward passes that increase goodness for positive data and decrease it for negative data at each layer.

  • Results

    0.64% test error was achieved on MNIST with jittered training images, similar to a convolutional neural network trained with backpropagation.

  • Takeaways & Limitations

    Forward-Forward is a promising candidate for cortical learning and low-power analog hardware, though its performance on large neural networks remains uncertain.

  • Takeaways & Limitations

    The algorithm is somewhat slower than backpropagation, generalizes less well on several toy problems, and remains untested at large neural-network scale.

Abstract

from arXiv · show

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

1 What is wrong with backpropagation

The paper argues that backpropagation is biologically implausible and poorly suited to real-time, pipelined learning when forward computations are unknown. It introduces Forward-Forward as an alternative that avoids reinforcement learning and backward error propagation.

  • 1 What is wrong with backpropagation: Backpropagation remains implausible as a model of cortical learning because there is no convincing evidence that cortex propagates derivatives or stores activities for a backward pass.The paper also notes that cortical top-down connections form loops rather than mirroring bottom-up connections.
  • 1 What is wrong with backpropagation: Backpropagation through time conflicts with the brain’s need to pipeline sensory input and learn in real time without stopping for derivative propagation.Later processing stages may influence earlier stages at later time steps, but inference and learning must continue online.
  • 1 What is wrong with backpropagation: Backpropagation requires perfect knowledge of the forward computation, so inserting an unknown nonlinear black box requires learning a differentiable model of that black box.Forward-Forward does not require backpropagation through the black box.
  • 1 What is wrong with backpropagation: Reinforcement learning avoids requiring a perfect forward model but suffers from high variance and scales badly as the number of perturbed variables grows.The paper states that its learning rate must be inversely proportional to the number of perturbed variables, limiting competitiveness on very large networks.
  • 1 What is wrong with backpropagation: Forward-Forward is presented as comparable in speed to backpropagation while supporting unknown nonlinearities and pipelined sequential learning without storing neural activities.The paper positions these properties as advantages for cortical learning and low-power analog hardware, while acknowledging weaker generalization on several toy problems.

2 The Forward-Forward Algorithm

Forward-Forward replaces backpropagation’s forward and backward passes with two local forward passes using positive and negative data. Each layer increases goodness for positive data and decreases it for negative data, with normalization preventing later layers from relying only on activity magnitude.

  • 2 The Forward-Forward Algorithm: Forward-Forward is a greedy multi-layer procedure inspired by Boltzmann machines and Noise Contrastive Estimation.It replaces backpropagation’s forward and backward passes with two forward passes using different data and opposite objectives.
  • 2 The Forward-Forward Algorithm: The positive pass uses real data to increase goodness in every hidden layer, while the negative pass decreases goodness for negative data.Negative data may be generated by the network through top-down connections or supplied externally.
  • 2 The Forward-Forward Algorithm: A layer classifies inputs as positive or negative by making its goodness exceed a threshold for real data and fall below it for negative data.With squared activities, goodness is the sum of the squared activities of rectified linear neurons.
  • 2 The Forward-Forward Algorithm: The squared activity-vector length is used because its derivatives are simple, while layer normalization removes information about goodness from the next layer’s inputs.The paper also explores the negative sum of squared activities as an alternative goodness measure.
  • 2 The Forward-Forward Algorithm: Forward-Forward normalizes hidden activity lengths so successive layers cannot trivially distinguish positive from negative data without learning new features.Without normalization, the second layer could use the first layer’s activity-vector length directly.

3 Some experiments with FF

The experiments test Forward-Forward on MNIST using unsupervised, supervised, convolution-like, and recurrent settings. It reaches competitive error rates on several variants while retaining limitations from greedy layerwise learning.

  • 3.1 The backpropagation baseline: 1.4% test error on permutation-invariant MNIST shows FF works about as well as backpropagation without complicated regularizers.The comparison uses fully connected ReLU networks and typical backpropagation performance as the reference.
  • 3.2 A simple unsupervised example of FF: 1.37% test error was obtained with four fully connected hidden layers when the normalized activity vectors of the last three layers fed a softmax.Including the first hidden layer in the classifier worsened test performance.
  • 3.2 A simple unsupervised example of FF: 1.16% test error followed from using local receptive fields without weight-sharing, with peer normalization of hidden activities.Only one such architecture was tried, and it was trained for 60 epochs.
  • 3.3 A simple supervised example of FF: 1.36% test error was achieved on supervised MNIST with labels included in the input, compared with similar backpropagation performance after about 20 epochs.FF required 60 epochs in the reported setup; doubling its learning rate and reducing training to 40 epochs produced 1.46% error.
  • 3.3 A simple supervised example of FF: 0.64% test error was achieved after jitter augmentation, similar to a convolutional neural net trained with backpropagation.Jittering used up to two pixels in each direction, producing 25 shifts per image and removing permutation invariance.
  • 3.4 Using FF to model top-down effects in perception: The recurrent FF network processes static MNIST images as repeated video input, addressing the weakness that greedy later layers cannot affect earlier learned layers.The recurrent approach uses normalized activity states from adjacent layers at the previous time step.

4 Experiments with CIFAR-10

On CIFAR-10, FF was tested in non-convolutional networks with local receptive fields and many hidden units, aiming for performance comparable to backpropagation despite variable backgrounds. FF performed slightly worse and learned more slowly, while its performance gap did not grow with additional hidden layers.

  • Experimental setup: FF used non-convolutional networks with local receptive fields, avoiding weight-sharing while retaining many hidden units.The comparison used 11 x 11 receptive fields and two or three hidden layers.
  • Experimental setup: The networks contained two or three hidden layers of 3072 ReLUs each, arranged as 32 x 32 topographic maps with three units per location.Each hidden unit had an 11 x 11 receptive field in the layer below.
  • Evaluation: Table 1 compared backpropagation and FF using weight decay to reduce overfitting.FF classification could use either one forward pass or repeated label-conditioned passes accumulating goodness over middle iterations.
  • Results: FF test performance was only slightly worse than backpropagation, even with complicated confounding backgrounds.The performance gap did not increase when more hidden layers were added.
  • Results: Backpropagation reduced the training error much more quickly than FF.

5 Sleep

The paper considers separating FF’s positive and negative learning phases so negative data could be generated and processed during sleep. However, the reported result could not be replicated, and phase separation remains unresolved.

  • Motivation: Separating positive and negative phases could make FF easier to implement in a brain by processing real data while awake and network-generated negative data during sleep.
  • Evidence: An early report that thousands of positive updates could precede thousands of negative updates with little performance loss was not replicated and may have resulted from a bug.
  • Evidence: With squared-activity goodness, alternating thousands of positive and negative updates worked only with a very low learning rate and extremely high momentum.
  • Open question: Whether another goodness function can separate the positive and negative phases remains unshown and is identified as the most important outstanding question for FF as a biological model.

6 How FF relates to other contrastive learning techniques

FF combines contrastive learning with local goodness objectives, replacing iterative Boltzmann settling and backpropagation through representations with two forward passes. Its agreement measure uses positive interference, while its one-bit local decisions create a constraint-poverty concern addressed through many blocks.

  • Relationship to Boltzmann Machines: Boltzmann Machines use stochastic binary neurons, symmetric pairwise connections, and repeated updates to sample an equilibrium distribution defined by network energy.
  • Relationship to Boltzmann Machines: Boltzmann learning obtains deep weight derivatives by contrasting data and freely running phases, but requires impractically long settling toward equilibrium.The two phases use different visible-neuron boundary conditions: clamped to data versus free.
  • Relationship to Boltzmann Machines: FF retains Boltzmann machines’ contrastive learning while using a simpler local goodness function that is more tractable than network free energy.
  • Relationship to GANs: Unlike GANs, FF makes each hidden layer greedily classify positive versus negative inputs, avoiding backpropagation for both discriminative and generative representations.The generative model reuses representations learned by the discriminative model.
  • Relationship to crop-based contrastive methods: FF measures agreement through positive interference when multiple information sources drive the same neurons, rather than comparing two representation vectors.Agreement raises squared activities, while disagreement lowers them.
  • A problem with stacked contrastive learning: Because distinguishing positive from negative cases can require only one bit, FF may receive weak domain constraints; dividing layers into blocks makes the constraint information scale linearly with block count.This is presented as better scaling than increasing the contrast set logarithmically in SimCLR-like methods.
  • A problem with stacked contrastive learning: Contrasting positive and negative data cancels structure caused solely by other network components, allowing many interconnected neuron groups to have separate objectives.

7 Learning fast and slow

FF permits rapid, simultaneous layer-wise learning because weight updates preserve the normalized activity direction for the current input. The paper notes that this property could support absorbing substantial information from individual cases, although experiments still use mini-batches.

  • Learning fast and slow: For full connectivity, updating incoming weights to change a layer’s goodness leaves that layer’s normalized output unchanged for the current input.The update changes all hidden activities by the same proportion, preserving the activity vector’s orientation.
  • Learning fast and slow: This invariance permits simultaneous online updates across many layers because earlier-layer updates do not alter later-layer activity vectors for that input.All weights can therefore be changed in one step so every layer reaches a desired goodness S*.
  • Learning fast and slow: The learning rate for reaching the desired goodness assumes the input and all layer-normalized hidden vectors have length 1.The current sum of squared activities before layer normalization is denoted S_L.
  • Learning fast and slow: Although the authors currently use mini-batches, single-case jumps to weights that handle an example perfectly may interest psychologists studying rapid learning.The paper explicitly presents this as a potentially interesting property rather than an exploited training procedure.
  • Learning fast and slow: FF can train layers separated by an unknown, possibly stochastic black-box transformation without changing the FF learning procedure.The paper suggests neural networks with slowly learning hidden layers as one possible black-box arrangement.

8 The relevance of FF to analog hardware

The paper connects FF with energy-efficient analog hardware, where vector–matrix multiplication can be implemented through voltages, conductances, and accumulating charges. Its two-forward-pass structure may avoid costly digital gradient computation and associated A-to-D conversion.

  • The relevance of FF to analog hardware: Analog multiplication can represent activities as voltages and weights as conductances, whose products accumulate as charges.The paper presents this as an energy-efficient alternative for multiplying an activity vector by a weight matrix.
  • The relevance of FF to analog hardware: Digital multiplication can require O(n^2) single-bit operations for two n-bit numbers.The passage contrasts this digital procedure with analog multiplication and discusses the cost of implementing backpropagation efficiently.
  • The relevance of FF to analog hardware: Using two forward passes instead of a forward and backward pass should make A-to-D converters unnecessary for computing gradients.The paper links this hardware advantage specifically to FF’s replacement of backpropagation’s backward pass.

9 Mortal Computation

The paper argues that future low-power computing may require hardware whose details are not faithfully shared across machines, shifting emphasis from copied parameters toward learned functions. FF is proposed as a candidate for this setting, but its scalability remains unresolved.

  • Mortal Computation: Deep learning changes how computers might be built by reducing the assumption that software must specify every operation in detail.The paper frames this as a long-term implication for the relationship between software and hardware.
  • Mortal Computation: Separating software from hardware enables program analysis, copying programs across machines, and parallel derivative computation using identical models.These are identified as benefits of the traditional software–hardware separation.
  • Mortal Computation: For image classification, the transferable object of interest is the function from pixel intensities to class labels rather than hardware-specific parameter values.This motivates transferring learned behavior across differently working hardware.
  • Mortal Computation: Distillation works best when teacher outputs reveal substantial information about the teacher’s internal representations.The passage connects this informativeness to how language may help a hearer learn similar vector representations.
  • Mortal Computation: Trillion-parameter neural networks consuming only a few watts may require computation implemented in mortal hardware whose precise details are unknown.The paper presents FF as a promising candidate for learning efficiently in that setting.
  • Mortal Computation: FF’s feasibility for unknown hardware remains uncertain because its ability to scale to large neural networks has yet to be established.This is the section’s explicit scope boundary for the proposed hardware application.
  • Mortal Computation: Sharing weights across large models offers higher-bandwidth knowledge transfer than distillation when identical hardware copies and their energy costs are acceptable.The comparison is conditioned on paying the energy costs of running identical models on multiple copies of the same hardware.

10 Future work

The paper closes by identifying open questions about FF’s data generation, objectives, activations, spatial structure, sequential processing, and constrained representations. These questions define directions for extending and evaluating the algorithm.

  • Future work: A central open question is whether FF can generate sufficiently good images or videos to create negative data for unsupervised learning.This directly concerns whether the network can supply its own negative examples.
  • Future work: The best goodness function remains unresolved, with several squared- and unsquared-activity objectives showing different reported behavior.The paper reports that minimizing squared activity for positive data and maximizing it for negative data can work slightly better, while unsquared activity minimization has also worked well.
  • Future work: Only ReLU activations have been explored so far, leaving alternative activation functions untested in FF.The paper specifically identifies a negative-log t-distribution density as an interesting possibility.
  • Future work: For spatial data, the paper asks whether many local goodness functions can accelerate learning across different image regions.The proposed arrangement assigns goodness functions to local regions rather than treating the image only globally.
  • Future work: For sequential data, an open direction is using fast weights to mimic a simplified transformer.The question is posed as a possible extension for sequence processing.
  • Future work: Another open direction combines feature detectors maximizing squared activity with constraint-violation detectors minimizing squared activity.This would extend FF beyond a single type of goodness-driven detector.
Loading 2212.13345v1…