Source-linked AI summary
Deep Learning without Weight Transport
Mohamed Akrout, Collin Wilson, Peter C. Humphreys, Timothy Lillicrap, Douglas Tweed
TL;DR
Biologically plausible deep-learning algorithms must avoid weight transport, which standard backpropagation relies on. This paper introduces weight mirrors and a modified Kolen–Pollack algorithm, which outperform feedback alignment and sign-symmetry on ImageNet while roughly matching backpropagation.
Problem
Standard backpropagation relies on transporting forward synaptic weights to a physically distinct feedback path, with no known biological mechanism for coordinating them.
Method
The paper uses weight mirrors and a modified Kolen–Pollack algorithm to learn appropriate feedback weights locally without weight transport.
Results
On ImageNet with ResNet-18 and ResNet-50, both methods outperformed feedback alignment and sign-symmetry while roughly matching backpropagation.
Takeaways & Limitations
These mechanisms may also apply to technologies such as ASICs that lack weight transport, potentially keeping forward and feedback weights local.
Takeaways & Limitations
The original Kolen–Pollack formulation is not biological because it transports weight changes, although the paper describes a local implementation.
Abstract
from arXiv · showhide
Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transport by using random feedback weights, but it performs poorly on hard visual-recognition tasks. Here we describe two mechanisms - a neural circuit called a weight mirror and a modification of an algorithm proposed by Kolen and Pollack in 1994 - both of which let the feedback path learn appropriate synaptic weights quickly and accurately even in large networks, without weight transport or complex wiring.Tested on the ImageNet visual-recognition task, these mechanisms outperform both feedback alignment and the newer sign-symmetry method, and nearly match backprop, the standard algorithm of deep learning, which uses weight transport.
1 Introduction
Deep-learning algorithms rely on weight transport, creating a biological plausibility problem, while feedback alignment fails to scale to hard visual-recognition tasks such as ImageNet. The paper proposes a weight mirror and a modified Kolen–Pollack algorithm that learn appropriate feedback weights without weight transport and perform on ImageNet about as well as backprop.
- Motivation: Backprop relies on weight transport, in which forward-path units transmit synaptic weights to a feedback path, limiting its suitability for brains.
- Prior work: Feedback alignment avoids weight transport but does not scale to hard visual-recognition problems such as ImageNet.
- Prior work: Sign-symmetry achieved good ImageNet performance, while other weight-transport-free approaches had been tested only in smaller networks and tasks.
- Contribution: The paper proposes a weight mirror and a modified Kolen–Pollack algorithm that learn appropriate feedback weights from initially random values without weight transport.
- Contribution: The proposed approaches learn ImageNet about as well as backprop without requiring forward and feedback matrices to have matching signs at initialization.
2 The weight-transport problem
Deep-learning networks use a forward path for inference and a feedback path for error signals, but backpropagation requires the feedback weights to be transposes of forward weights. This creates a biological problem because forward and feedback synapses are physically distinct, with no known coordination mechanism ensuring that relationship.
- Forward and feedback paths: Forward-path signals perform inference through multiple processing layers, while feedback signals carry errors from the output layer to guide learning.The paths may involve distinct neuron populations or the same neurons carrying signals in opposite directions.
- Forward computation: Each forward layer transforms the previous layer’s activity by multiplying it by a weight matrix, adding a bias, and applying an activation function.These quantities could correspond biologically to neuronal firing rates, synaptic weights, and biases.
- Backpropagation: Backpropagation sends feedback signals layer by layer through the transposes of the forward weight matrices.The feedback computation also uses the derivative of the activation function, which can be computed from the forward-layer output.
- The weight-transport problem: The weight-transport problem is that physically distinct forward and feedback synapses must somehow coordinate so one weight set remains the transpose of the other.The passages identify no known biological mechanism that guarantees this coordination.
3 Feedback alignment
Feedback alignment replaces transposed feedback weights with random, fixed matrices, yet learning drives forward weights toward their transposes, enabling backprop-like learning on simple tasks. The section then motivates adjusting feedback weights to improve agreement and support harder tasks without weight transport.
- 3 Feedback alignment: Feedback alignment replaces transposed W_l feedback weights with random, fixed matrices B_l.The feedback matrices do not learn in the basic formulation.
- 3 Feedback alignment: Learning drives forward matrices W_l to become roughly proportional to transposes of feedback matrices B_l.This emergent rough transposition makes feedback-alignment updates similar enough to backpropagation for simple tasks.
- 3 Feedback alignment: Rough transposition makes feedback-alignment learning perform as well as backpropagation on simple tasks.The similarity arises because the learned relationship between W_l and B_l makes the feedback update resemble the backpropagation equation.
- 3 Feedback alignment: The proposed augmentation adjusts feedback weights B_l alongside forward weights W_l to improve their agreement on harder tasks.The paper presents two mechanisms that achieve this adjustment quickly and accurately in large networks without weight transport.
4 Weight mirrors
Weight mirrors use a dedicated mirror mode and Hebbian learning to adjust initially random feedback weights toward positive scalar multiples of the corresponding forward-weight transposes. With norm control, this learning produces reasonable-sized feedback weights without weight transport, under stated activation assumptions or bias-blocking.
- Circuit and learning rule: In mirror mode, noisy forward signals are transmitted to the feedback path, whose cells adjust feedback weights by Hebbian learning.The simplest analysis assumes independently firing, zero-mean neurons with equal variance.
- Circuit and learning rule: The weight mirror is a neural circuit and learning rule that trains feedback weights to mimic forward weights.It alternates between engaged mode for inference and forward-weight learning, and mirror mode for feedback-weight adjustment.
- Convergence: The feedback matrix integrates a teaching signal that approximates, on average, a positive scalar multiple of the corresponding forward-weight transpose.This result follows when the activation operates in a roughly affine range and its derivative matrix is approximately a scalar multiple of the identity.
- Convergence: With weight decay, the feedback matrix evolves toward a reasonable-sized positive multiple of the forward-weight transpose; bias-blocking removes the need for similar activation-derivative assumptions when φ has a positive derivative around 0.Without norm control, integration may increase the feedback matrix norm.
5 The Kolen-Pollack algorithm
The Kolen–Pollack approach replaces weight transport with matched weight changes, but the original formulation is not biologically feasible. A reciprocal, locally computed variant allows forward and feedback weights to converge to transposes.
- Original Kolen–Pollack algorithm: Kolen and Pollack proposed transporting identical weight changes, rather than weights, so forward and feedback synapses converge when weight decay satisfies 0 < λ < 1.The weight difference contracts as (1 − λ)^(t+1)[W(0) − B(0)].
- Biological limitation: The original algorithm is not biologically feasible because it requires weight changes calculated at one locus to travel to distinct forward and feedback synapses.The paper notes that synaptic changes are more plausibly calculated separately within each synapse.
- Reciprocal implementation: The proposed reciprocal arrangement updates feedback synapses from their own inputs and cell-specific scalar teaching signals supplied by the forward path.This mirrors the forward rule, in which each synapse uses its scalar input and the scalar teaching signal sent to its neuron.
- Reciprocal implementation: With matched learning rates and weight decay, synapses compute adjustments locally, while only activity vectors y_l and δ_l+1 pass between cells.The resulting updates retain the Kolen–Pollack form and make the forward and feedback matrices converge to transposes.
6 Experiments
On ImageNet, weight-mirror and Kolen–Pollack networks substantially outperformed feedback alignment and sign-symmetry, nearly matching backprop, while learning feedback matrices and vectors closely aligned with backprop. Sign-symmetry improved when BatchNorm preceded ReLU, whereas the other methods changed little.
- ImageNet results: On ResNet-18 ImageNet, Kolen-Pollack reached 29.2(4)% top-1 test error and weight mirrors 30.2(7)%, versus 30.1(4)% for backprop.Plain feedback alignment reached 97.4(2)%, while sign-symmetry reached 39.2(4)%.
- ImageNet results: On ResNet-50 ImageNet, backprop reached 22.9(4)% top-1 test error, compared with 23.4(5)% for weight mirrors and 23.9(7)% for Kolen-Pollack.Feedback alignment reached 98.9(1)%, and sign-symmetry reached 33.8(3)%.
- BatchNorm condition: With BatchNorm before ReLU, sign-symmetry reached 37.8(4)% top-1 test error on ResNet-18 and 32.6(6)% on ResNet-50.The same BatchNorm change made little difference to the other four methods.
- Alignment during training: Weight mirrors kept feedback-matrix angles below 12° in every layer and feedback-vector δ angles below 25° relative to backprop.Matrix alignment improved to below 6° later in training for all layers except the final fully connected layer.
- Alignment during training: Kolen-Pollack reduced matrix and δ angles to near zero within 20 epochs and maintained them there.Feedback alignment instead kept both angles above 80° for most layers in both ResNet architectures.
7 Discussion
The weight mirror and Kolen–Pollack network outperform feedback alignment and sign-symmetry while roughly keeping pace with backprop. Their differing operating requirements also suggest applications beyond biology to systems lacking weight transport.
- Both methods outperformed feedback alignment and sign-symmetry, while roughly keeping pace with backprop.The comparison applies to both the weight mirror and Kolen–Pollack network.
- Kolen–Pollack avoids separate operating modes and layer-by-layer training, whereas weight mirrors learn feedback paths from noise without sensory input.Weight mirrors could therefore tune feedback paths during sleep or in utero.
- Both methods may apply beyond biology because other computing systems also lack weight transport.The underlying issue is that information is split between rapidly transmissible action potentials and localized, compact synaptic weights.
Appendices … C.4 Multilayer mirroring
The appendices clarify the gradient-descent interpretation and computational cost of the learning rules, while examining biological implementations and constraints on multilayer mirroring. They address neuron reuse, signal representation, projection requirements, and when multiple feedback weights can be adjusted simultaneously.
- A The transposing rule as gradient descent: With weight decay, the transposing rule descends the gradient of a loss, although its associated function alone has no minimum.The function can decrease without bound as feedback weights grow, so weight decay is required for the loss interpretation.
- B Computational costs: Weight mirroring is slightly more computationally expensive than Kolen–Pollack learning for adjusting feedback weights.For fully connected adjacent layers, both methods use n + 4n_l n_{l+1} flops for the adjustment, while weight mirroring additionally generates a random vector.
- C.1 Distinct feedback neurons?: The proposed methods can use the same neurons for inference and error signals, or separate forward and feedback neuron populations.When populations are separate, the methods require one-to-one pairing between forward-path and feedback-path cells.
- C.1 Distinct feedback neurons?: Using the same neurons creates a signal-segregation problem because inference signals y and error signals δ must flow without interfering.Possible solutions include distinct intracellular messengers, different cellular compartments, or multiplexing.
- C.2 Zero-mean signals: Neural firing rates can represent positive and negative signals by modulating around a baseline β instead of requiring zero-mean rates.A baseline can be estimated locally from average firing rates during mirroring, and push–pull cell groups offer another biological mechanism.
- C.3 Multipurpose projections?: The circuit may require distinct axon sets carrying the same forward signal y_l, because those projections can have different effects on feedback cells.One set conveys y_l for computing φ′(y_l), while another sets δ_l = y_l.
- C.4 Multilayer mirroring: Adjusting all feedback matrices simultaneously generally fails because transformed forward signals lose the zero-mean, uncorrelated, and equal-variance properties needed for subsequent mirroring.Applying noise to every second layer can permit adjustment of up to half the feedback matrices at once.
- C.4 Multilayer mirroring: Batch normalization or synaptic scaling may allow broader simultaneous adjustment by keeping forward signals approximately zero-mean and equal-variance.These mechanisms do not generally preserve uncorrelated signals, so the possibility remains conditional.
D Experimental details · D.1 Architecture and training · D.2 Mirroring
Experiments used ResNet-18 and ResNet-50 architectures with specified sign-symmetry training settings. Weight-mirror networks aligned random weights during an initial mirror phase and updated feedback weights using covariance estimates.
- D.1 Architecture and training: Experiments used 18-layer ResNet-18 and 50-layer ResNet-50 deep-residual networks.ResNet-18 sub-blocks contained two convolutional layers, whereas ResNet-50 sub-blocks contained three.
- D.1 Architecture and training: Each residual sub-block combined convolutional layers with a parallel shortcut connection whose output was added to the convolutional output.
- D.1 Architecture and training: The network output passed through a final fully-connected layer followed by a softmax.
- D.1 Architecture and training: Sign-symmetry experiments used learning rate 0.5 for both ResNet-18 and ResNet-50 after searches over [0.01, 2.0] through 140 epochs.The reported search selected 0.5 as yielding the lowest top-1 errors.
- D.2 Mirroring: Each weight-mirror network spent its first two epochs entirely in mirror mode before adding limited mirroring after each engaged-mode minibatch.Mirroring used independent Gaussian signals with zero mean and unit variance across 2048 examples.
- D.2 Mirroring: Mirror covariance estimates were averaged across connected input-output pairs to account for convolutional weight sharing.The signals’ covariance matrix was estimated by multiplying and averaging zero-mean minibatch signals.
- D.2 Mirroring: Feedback weights were trained from these covariance estimates using learning-rate factor ηB of 0.1 and weight decay λW M of 0.5.
D.3 Batch normalization · E Pseudocode
Batch normalization cannot be fully mirrored by the learned feedback matrices, so the forward-path scaling parameter σ is passed to feedback neurons while µ is omitted. The pseudocode specifies separate engaged and mirror updates for Weight Mirrors and direct feedback-weight updates for Kolen–Pollack.
- D.3 Batch normalization: Weight Mirrors match feedback matrices B_l to forward matrices W_l but cannot reproduce batch-normalization parameters µ and σ.Because convolutional weight sharing is shared by W_l and B_l while µ and σ are not, exact mirroring is impossible.
- D.3 Batch normalization: σ is passed from the forward to feedback path, while µ is omitted because it is unnecessary.The transfer uses one scalar variable per feedback neuron and could be avoided by replacing convolution.
- D.3 Batch normalization: 32.6(6)% final error was the best sign-symmetry result tested with ResNet-50 when batch normalization preceded activation.Most experiments normalized after activation, but sign-symmetry learned slightly better with normalization before activation.
- E Pseudocode: Algorithm 1 updates Weight Mirrors through forward error-driven weight learning and a separate mirror phase that trains B_l+1 from centered forward and feedback activations.The procedure samples hidden activity, propagates forward-cell activity to feedback cells, and updates B_l+1 using equation (7) and weight decay.
- E Pseudocode: Weight Mirrors compute δ_l = φ′(y_l) B_l+1 δ_l+1 to obtain error gradients using learned feedback matrices.The engaged-mode loop updates W_l+1 and b_l+1 before the mirror-mode loop updates feedback matrices.
- E Pseudocode: Algorithm 2 updates Kolen–Pollack feedback matrices B_l+1 directly alongside forward weights and biases during the backward layer loop.Its feedback update uses y_l δT_l+1, with weight decay, before computing δ_l through B_l+1.