PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Biologically Plausible Deep Learning

Yoshua Bengio, Dong-Hyun Lee, Jorg Bornschein, Thomas Mesnard, Zhouhan Lin

arXiv:1502.04156v3cs.LG

TL;DR

The paper addresses the lack of biologically plausible learning mechanisms for deep networks, especially credit assignment across many layers. It interprets STDP as approximate gradient-based learning and variational EM implemented through neural dynamics, adds gradient-free target propagation and denoising-autoencoder interpretations, and reports improved generative-model likelihood and sampling. The authors present this as an exploratory step with remaining biological implementation constraints.

  • Problem

    The paper asks how deep-network credit assignment and biological synaptic learning rules can receive a credible machine-learning interpretation compatible with neurobiology.

  • Method

    The paper interprets STDP as stochastic gradient descent and neural dynamics as approximate variational EM, using target propagation and denoising auto-encoders for inference and sampling.

  • Results

    The proposed target-propagation inference improves joint likelihood, while iterating the denoising-autoencoder Markov chain produces better samples than the directed generative model.

  • Takeaways & Limitations

    The framework provides a biologically motivated route to deep generative learning while connecting STDP, variational EM, target propagation, and denoising auto-encoders.

  • Takeaways & Limitations

    The approach remains exploratory: detailed biological connections are incomplete, and implementation may require synchronization, control mechanisms, and specific connectivity.

Abstract

from arXiv · show

Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology. We explore more biologically plausible versions of deep representation learning, focusing here mostly on unsupervised learning but developing a learning mechanism that could account for supervised, unsupervised and reinforcement learning. The starting point is that the basic learning rule believed to govern synaptic weight updates (Spike-Timing-Dependent Plasticity) arises out of a simple update rule that makes a lot of sense from a machine learning point of view and can be interpreted as gradient descent on some objective function so long as the neuronal dynamics push firing rates towards better values of the objective function (be it supervised, unsupervised, or reward-driven). The second main idea is that this corresponds to a form of the variational EM algorithm, i.e., with approximate rather than exact posteriors, implemented by neural dynamics. Another contribution of this paper is that the gradients required for updating the hidden states in the above variational interpretation can be estimated using an approximation that only requires propagating activations forward and backward, with pairs of layers learning to form a denoising auto-encoder. Finally, we extend the theory about the probabilistic interpretation of auto-encoders to justify improved sampling schemes based on the generative interpretation of denoising auto-encoders, and we validate all these ideas on generative learning tasks.

1. Introduction

The paper seeks biologically plausible deep learning by giving biological learning rules a machine-learning interpretation that supports credit assignment across layers. It interprets STDP through stochastic gradient descent and variational inference, and develops gradient-free approximations and improved sampling for generative learning.

  • Motivation: The central problem is finding a biologically plausible mechanism for credit assignment through long chains of neural connections.The paper identifies this as its main long-term question.
  • Motivation: Back-propagation is considered biologically implausible because its feedback computation is linear while biological neurons interleave linear and nonlinear operations.
  • Core framework: STDP is interpreted as stochastic gradient descent when feedback changes neuronal firing rates in proportion to an objective-function gradient.
  • Core framework: Neural dynamics are framed as approximate inference, yielding a variational EM training procedure for deep generative networks with latent variables.The inference may use MAP or MCMC approximations rather than exact posteriors.
  • Additional contributions: Gradient estimates can be obtained through ordinary forward and backward neural computation using difference target propagation and denoising auto-encoders.The approach avoids explicit derivatives for the required gradient estimates.

2. STDP as Stochastic Gradient Descent

The paper proposes an update rule whose voltage-dependent dynamics reproduce the timing dependence of STDP and interprets that rule as approximate stochastic gradient descent. The resulting relationship links spike timing and synaptic changes through the direction and magnitude of postsynaptic voltage change.

  • STDP motivation: STDP relates expected synaptic weight changes to the timing difference between postsynaptic and presynaptic spikes, but its role in deep-network learning remains unclear.
  • Update rule: The proposed update equation explains the STDP curve as a consequence of an update rule motivated by machine learning.
  • Update rule: The update rule uses the presynaptic spike, postsynaptic voltage, and temporal derivative of postsynaptic voltage to determine synaptic changes.The paper defines ˙Vj as the temporal derivative of Vj and Si as the presynaptic spike.
  • STDP mechanism: For increasing postsynaptic voltage, a longer positive spike-time difference corresponds to a smaller positive synaptic-weight change.
  • STDP mechanism: For decreasing postsynaptic voltage, negative spike-time differences correspond to negative weight changes whose magnitude decreases as the timing difference becomes more negative.
  • Machine-learning interpretation: The update rule corresponds to approximate stochastic gradient descent when postsynaptic activation changes in the direction of the objective-function gradient.Under this view, STDP implements the delta rule for a one-layer network.

3. Variational EM with Learned Approximate Inference

The paper interprets biologically plausible learning as variational inference with approximate posteriors, combining learned initialization with iterative updates that improve agreement between latent variables and observations. This framework supports local parameter and latent-state updates without requiring exact posterior inference.

  • Neural inference: Neural dynamics update latent variables toward configurations more consistent with observations, targets, or rewards, connecting inference dynamics to efficient credit assignment.This interpretation is presented as analogous to inference in Boltzmann machines while avoiding exact posterior representations.
  • Variational objective: The objective J is motivated as a variational bound on the data likelihood, with approximate inference distribution q*(H|x) standing in for the posterior p(H|x).The bound becomes tight when q*(H|x) equals p(H|x), but the paper considers variational methods where the approximation does not reach the exact posterior.
  • Learned approximate inference: The paper decomposes approximate inference into a learned parametric initialization q0(H|x) and deterministic or stochastic iterative inference defining qT(H|x).The transition operator is intended to gradually bring the inferred distribution closer to p(H|x).
  • Optimization procedure: The regularized variational MAP-EM criterion updates each example’s latent state h from q(H|x), then optimizes the average objective over examples.The regularization encourages initial and subsequent inferred states to approach the posterior using few inference steps.
  • Optimization procedure: Algorithm 1 uses q(h|x) as a learned initialization, p(h) as a prior, p(x|h) as the generative model, and learning rates δ and ϵ for latent-state and parameter optimization.Optional noise supports a stochastic MCMC variant, although the paper notes that rejection steps are not biologically plausible.

4. Training a Deep Generative Model

The deep generative model is organized as a multilayer directed graphical model with feedforward and feedback pathways. Layerwise updates are local, yet iterative inference lets all layers interact through both directions of connectivity.

  • Deep generative structure: The model decomposes latent state h into multiple layers arranged as a chain in both the generative model p and the inference model q.This organization follows the paper’s discussion of distinct cortical feedforward and feedback connectivity patterns.
  • Local learning: Layer updates are local to layer k, using feedback from neighboring layers k−1 and k+1 rather than requiring back-propagation.The updates remain coupled globally through repeated interactions across feedforward and feedback paths.
  • Local learning: The gradient for a latent variable on the left side of a conditional probability can be computed without back-propagation; for a conditional Gaussian, it is proportional to (μ−h(k))/σ^2.The paper gives this as the gradient with respect to h(k) for p(h(k)|h(k+1)).
  • Deep generative structure: Layers above h(k) provide a complex implicitly defined prior for p(h(k)) in the resulting deep model.This interpretation follows from the hierarchical conditional structure.

5. Alternative Interpretations as Denoising Auto-Encoder

The variational learning procedure can be viewed as training complementary, and with noise denoising, auto-encoders over visible and latent variables. This joint interpretation yields Markov-chain sampling procedures that can improve generated samples.

  • Complementary auto-encoders: The algorithm trains p(x|h) and q(h|x) as complementary auto-encoder pairs, with either conditional serving as encoder or decoder.In deeper models, each adjacent pair of latent layers forms a symmetric auto-encoder trained on reversed input-target roles.
  • Denoising interpretation: Noise from processes such as spike-train quantization turns the paired auto-encoders into denoising auto-encoders that map corrupted neighboring values to clean observed pairs.Both encoders and decoders are contractive near observed (x, h) pairs.
  • Joint model: A joint denoising auto-encoder treats (x, h) as visible variables and implicitly estimates an underlying p(x, h).For deeper models, middle-layer transitions can reconstruct h(k) from either above or below.
  • Sampling: Running the joint transition operator for a few Markov-chain steps can produce better samples, especially in shallow models.The motivation is that simple priors may generate latent values outside the manifold represented by the decoder’s training data.
  • Sampling: A generative procedure can alternate inference q*(h|x) with reconstruction sampling from p(x|h), producing a chain whose stationary distribution estimates the data-generating distribution.This procedure uses the same inference operations except that x is unclamped during generation.

6. Targetprop instead of Backprop

The paper replaces explicit back-propagation for hidden-state updates with target-propagation estimators derived from denoising auto-encoder theory. These estimators use forward and backward neural computations and are justified geometrically when encoder and decoder mappings are approximate inverses.

  • Gradient estimation: The target-propagation approach estimates required hidden-state gradients using ordinary neural computation rather than explicit derivatives.It is related to earlier target-propagation work and is proposed for estimating gradients in a two-layer encoder-decoder system.
  • Gradient estimation: For Gaussian p(x|h) and q(h|x), the encoder f and decoder g provide the deterministic components used by the proposed gradient estimator.The estimator depends on the noise variance injected into q(h|x).
  • Theoretical basis: Denoising auto-encoder theory supports estimating a log-score from the difference between a trained reconstruction and its input.The relation includes the variance of the injected noise and the implicitly estimated density.
  • Theoretical basis: The required hidden gradient is decomposed into terms estimated from auto-encoders associated with p(x|h) and p(h).For the p(h) term, g acts as encoder and f as decoder, so f(g(h)) reconstructs h.
  • Geometric interpretation: The geometric derivation assumes encoder-decoder compositions approximately recover nearby inputs when changes are small and the functions are smooth.This yields the formula associated with Eq. 12 and Figure 2.

7. Related Work

The paper situates its proposal among biologically motivated interpretations of STDP, target propagation, generative auto-encoders, and variational latent-variable learning. Its stated novelty is a variational-EM interpretation that supports deep generative learning without symmetric weights or back-propagation for latent variables.

  • Proposal: The proposal interprets STDP through a variational EM framework and introduces neural updates that approximately propagate gradients toward better overall activity configurations.It is presented as avoiding symmetric weights for deep generative unsupervised learning.
  • Proposal: Unlike an earlier supervised back-propagation proposal based on iterative inference, this approach avoids the need for symmetric weights.The earlier proposal required symmetric weights, whereas the present proposal addresses that constraint in its framework.
  • Relation to prior models: The framework extends Predictive Sparse Decomposition by justifying its training criterion as a variational bound and generalizing it to multiple latent layers.PSD is described as a one-layer special case of Algorithm 1.
  • Relation to prior models: The proposal differs from Wake-Sleep, variational auto-encoders, and reweighted wake-sleep within the broader theme of combining approximate inference machines with generative decoders.The supplied passage identifies these methods as important earlier instantiations of that theme.
  • Target propagation: The method builds on target propagation by using iterative inference to reconcile bottom-up and top-down pressures when finding consistent targets.Related work also addressed weight transport in supervised learning through feedback alignment.
  • Novelty: The target-propagation formula matches earlier backprop-free auto-encoder work, while this paper adds a denoising-auto-encoder justification and empirical validation for variational learning.The validation concerns the ability to climb the joint likelihood.
  • Novelty: Compared with earlier generative auto-encoder work, the paper introduces latent variables without requiring back-propagation for training.This is stated as a first contribution relative to that prior line of work.

8. Experimental Validation

The proposed procedure generates MNIST samples, increases joint likelihood during inference, and fills in randomly missing pixels using clamped-input inference.

  • The two-hidden-layer network generated MNIST samples with an estimated Parzen log-likelihood of LL=236 after 20 training epochs.The hidden layers used 1000 softplus units and 100 sigmoid units.
  • With half the pixels randomly sampled and the remaining inputs clamped, a modified INFERENCE procedure reconstructed the missing image regions.Each row used a different corruption pattern.
  • LL=236 was about the same or better than contractive auto-encoders (LL=121), deeper generative stochastic networks (LL=214), and generative adversarial networks (LL=225).
  • Targetprop updates increased log p(x, h) over 20 iterations of the INFERENCE algorithm across the 10,000-digit test set.The figure reports the average and standard error, with dashed lines showing individual datapoints.
  • The modified inference maximized p(x, h) while using visible pixels to guide reconstruction of the missing inputs.

9. Future Work and Conclusion

The paper presents target propagation and denoising-auto-encoder interpretations as exploratory steps toward biologically plausible credit assignment, while identifying unresolved biological and theoretical limitations.

  • The approach is an exploratory step toward explaining credit assignment through many layers and addresses all but one of the listed non-plausible elements of back-propagation.
  • Targetprop-based inference produced systematic improvements in joint likelihood and enabled learning a good generative model.
  • Noisy variational EM updates can be interpreted as training a denoising auto-encoder over visible and latent variables, whose Markov chain yields better samples than the directed model.
  • The framework does not explore lateral connections, although they could implement priors on the joint distribution between nearby neurons.
  • Theoretical work is still needed to determine how and how much noise should replace MAP-based inference with MCMC-like inference.
  • Applications to supervised and reinforcement learning would require greater weighting of prediction or reconstruction errors for selected observed signals.
  • Detailed biological implementation remains unresolved, including spikes, excitatory or inhibitory weight-sign constraints, synchronization, and specific connectivity.
Loading 1502.04156v3…