PaperlayerView exact revision on arXiv

Source-linked AI summary

Surrogate Gradient Learning in Spiking Neural Networks

Emre O. Neftci, Hesham Mostafa, Friedemann Zenke

arXiv:1901.09948v2cs.NEq-bio.NC

TL;DR

SNN training faces extrinsic challenges including noise and non-stationarity, motivating brain-inspired approaches to signal processing. The article studies SNNs within an RNN framework and focuses on surrogate-gradient approaches, which can train SNNs to unprecedented performance levels on real-world problems.

  • Problem

    Noise and non-stationarity make signal processing challenging, motivating brain-inspired approaches.

  • Method

    The article studies SNNs within an RNN framework and explains backpropagation through unrolled recurrent networks.

  • Results

    Surrogate-gradient approaches can train SNNs to unprecedented performance levels across a range of real-world problems.

  • Takeaways & Limitations

    Surrogate gradients provide a framework connecting machine learning, computational neuroscience, and successful approaches for training SNNs.

Abstract

from arXiv · show

Spiking neural networks are nature's versatile solution to fault-tolerant and energy efficient signal processing. To translate these benefits into hardware, a growing number of neuromorphic spiking neural network processors attempt to emulate biological neural networks. These developments have created an imminent need for methods and tools to enable such systems to solve real-world signal processing problems. Like conventional neural networks, spiking neural networks can be trained on real, domain specific data. However, their training requires overcoming a number of challenges linked to their binary and dynamical nature. This article elucidates step-by-step the problems typically encountered when training spiking neural networks, and guides the reader through the key concepts of synaptic plasticity and data-driven learning in the spiking setting. To that end, it gives an overview of existing approaches and provides an introduction to surrogate gradient methods, specifically, as a particularly flexible and efficient method to overcome the aforementioned challenges.

I. INTRODUCTION

SNNs offer efficient, biologically inspired signal processing, but their binary, dynamical outputs make training—especially for deep networks with hidden layers—difficult. The article frames SNNs as RNNs and introduces learning strategies, including surrogate-gradient methods, to address these challenges.

  • SNNs and binary RNNs inherit optimization difficulties from long-range temporal and spatial dependencies, compounded by nondifferentiable binary outputs.
  • Training deep SNNs with hidden layers remains a major obstacle, although two-layer and recurrent SNNs have been trained successfully.
  • SNNs and neuromorphic hardware avoid energetically costly floating-point multiplications, supporting their use in low-power applications.
  • The article addresses hidden-layer training by discussing learning algorithms, strategies, and approximations for implementing specific functionalities.
  • The paper maps SNNs to RNNs by discretizing LIF dynamics, showing formal equivalence to RNNs with binary activation functions.
  • LIF dynamics combine decaying synaptic currents and membrane potentials with threshold-triggered spikes, resets, and optional recurrent or feedforward communication.

III. METHODS FOR TRAINING RNNS

RNN training minimizes a loss by updating weights through gradient descent, while hidden-unit learning requires spatial and temporal credit assignment. The section reviews how backpropagation and related algorithms compute these updates across layers and time.

  • RNNs support tasks ranging from time-series prediction to language translation and automatic speech recognition.
  • Training combines a loss function measuring desired behavior with weight updates intended to minimize that loss.
  • Gradient descent modifies network parameters opposite the loss gradient, providing a standard mechanism for learning.
  • Hidden-unit updates require credit assignment through downstream activity and weights, computed efficiently using chain-rule gradients.
  • The discussion distinguishes spatial credit assignment across layers from temporal credit assignment across recurrent time steps.

A. Spatial credit assignment

Spatial credit assignment distributes error information from network outputs to upstream neurons so hidden-layer weights can be updated by gradient descent. Backpropagation provides this mechanism, but requires backward communication and retained neuron states.

  • Backpropagation assigns credit across layers by propagating errors backward from the output toward upstream neurons.
  • With a sufficiently small learning rate, hidden-layer updates are guaranteed to reduce the cost for the current training example.
  • Backpropagation requires gradients to travel backward through the network and neuron states to remain stored until errors are available.
  • Gradient descent modifies network parameters opposite the loss gradient, using neuron error and output sensitivity to parameter changes.
  • Standard backpropagation alternates forward activity evaluation with backward evaluation of error signals.

B. Temporal credit assignment

Temporal credit assignment addresses dependencies across time in recurrent networks. BPTT solves it by unrolling recurrence and propagating errors backward, while forward methods move gradient information through time with different computational and memory trade-offs.

  • Recurrent networks require temporal credit assignment because network activity has interdependencies across time.
  • Backward method: BPTT unrolls the recurrent network and back-propagates errors through the resulting network after a forward pass.
  • Forward method: Forward methods preserve recurrent structure by propagating gradient information forward in time, with simplifications reducing space complexity from O(N3) to O(N2) or O(N).
  • Backward method: The backward method is generally more computationally efficient but requires retaining inputs and activations at every time step.
  • Backward method: For each layer, the backward method has space complexity O(NT), where N is neurons per layer and T is the number of time steps.
  • Forward method: These simplifications can make forward algorithms computationally comparable or better than BPTT and compatible with synaptic-plasticity-style learning rules.

IV. CREDIT ASSIGNMENT WITH SPIKING NEURONS: CHALLENGES AND SOLUTIONS

Training SNNs involves discontinuous spiking nonlinearities and hardware constraints that complicate gradient-based optimization. The article reviews smoothing and surrogate-gradient approaches as approximate solutions, focusing on supervised methods for functional SNNs.

  • Challenges: The spiking nonlinearity is non-differentiable because its binary output has a derivative that is zero except at threshold.
  • Challenges: This all-or-nothing behavior prevents gradients from flowing and makes LIF neurons unsuitable for ordinary gradient-based optimization.
  • Solutions: Common surrogate derivatives include piece-wise linear, fast-sigmoid-derived, and exponential functions, with axes rescaled separately for illustration.
  • Challenges: SNN training also faces computation, memory, communication, locality, and scaling constraints on computers, brains, and neuromorphic devices.
  • Solutions: The article identifies four response categories: local biological learning rules, rate-network translation, continuous smoothing, and surrogate gradients.
  • Solutions: It focuses on smoothing and surrogate-gradient supervised approaches, reviewing smoothing before explaining how surrogate gradients can build functional SNNs.

A. Smoothed spiking neural networks

Smoothed SNN models make gradients directly suitable for optimization by relaxing binary spiking behavior through soft, probabilistic, rate-based, or temporal formulations. These approaches trade biological or coding properties for differentiability and may incur efficiency, temporal-precision, or noise-related limitations.

  • Smoothed SNN formulations ensure well-behaved gradients that are directly suitable for optimization.
  • Smooth-model alternatives include probabilistic, rate, and single-spike temporal formulations, while injected noise can challenge optimization.
  • Soft nonlinearity models: Soft nonlinearity models replace binary spiking with continuous-valued gating, enabling optimization with BPTT or RTRL but compromising binary behavior.
  • Probabilistic models: Probabilistic neurons smooth binary nonlinearities through stochasticity, making expected quantities such as spike-train log-likelihood differentiable.
  • Rate-based models: Rate-based coding uses smoothly varying firing rates or f-I curves to derive gradients and has achieved competitive benchmark performance.
  • Rate-based models: Rate coding requires averaging across spikes, populations, or long intervals, and operating at the rate level sacrifices temporal precision.

4) Gradients in single-spike-timing-coding networks:

Single-spike timing codes represent outputs through firing times, allowing individual spikes to carry more information than rate codes. Their gradients can be computed analytically or approximately, but assumptions such as one spike per hidden unit constrain applicability.

  • Temporal coding represents SNN outputs as firing times, so individual spikes can carry more information than rate-based spike counts.
  • SpikeProp linearized hidden-unit firing-time expressions to compute approximate hidden-layer gradients, while later work avoided that linearization.
  • Spike-timing formulations may require every hidden unit to emit exactly one spike because firing time is undefined for quiescent units.
  • The non-quiescence requirement can conflict with power efficiency, since tasks may benefit from activating only a subset of neurons.
  • Surrogate gradients replace derivatives of the discontinuous spiking nonlinearity with smooth approximations while retaining standard optimization procedures.
  • Surrogate-gradient learning can address spatial and temporal credit assignment through BPTT or forward methods such as eligibility traces.

1) Surrogate derivatives for spiking nonlinearity:

Surrogate derivatives provide a flexible alternative for training SNNs with discontinuous spiking nonlinearities. The literature spans multiple surrogate functions and architectures, while open concerns include surrogate-induced bias, vanishing gradients, and scalability constraints.

  • Surrogate-gradient methods replace each spiking derivative in standard algorithms such as BPTT with the derivative of a smooth function.
  • Different studies use surrogate functions including fast sigmoids, exponentials, piecewise-linear functions, and ReLU-like derivatives.
  • Surrogate-gradient networks have shown competitive performance on neuromorphic benchmarks, temporal datasets, and conventional benchmarks.
  • The surveyed surrogate nonlinearities are nonlinear and monotonically increasing toward the firing threshold, but systematic comparison remains pending.
  • Future challenges include vanishing gradients from sigmoidal surrogates, bias in learning dynamics, and additional difficulties in deeper architectures.
  • Surrogate gradients can reduce learning complexity and computational cost, including through approaches that make larger deviations from true gradients.

V. APPLICATIONS

Applications combine surrogate gradients with alternative error-propagation strategies to reduce nonlocality and hardware demands. Feedback-alignment methods use fixed random connections and can preserve useful error signals with limited performance loss.

  • Surrogate-gradient applications exploit continuous-time neuronal dynamics and event-driven activity, allowing networks to remain quiescent until triggered by incoming spikes.
  • Feedback alignment and random error backpropagation: Feedback alignment replaces transposed forward weights in backward propagation with fixed random matrices, making learning more local.
  • Feedback alignment and random error backpropagation: Direct random propagation replaces the entire backward pathway by propagating errors directly to each layer.
  • Feedback alignment and random error backpropagation: Random BP approaches can incur remarkably little loss in classification performance on some benchmark tasks.
  • Feedback alignment and random error backpropagation: During learning, feedforward weights partially align with random feedback weights, allowing them to convey useful error information.
  • Feedback alignment and random error backpropagation: A spike-driven random-BP adaptation using local synaptic plasticity can operate continuously and asynchronously without alternating forward and backward passes.
  • Feedback alignment and random error backpropagation: Random BP initially failed to account for neuronal and synaptic temporal dynamics, a limitation addressed by SuperSpike.

B. Supervised learning with local three factor learning rules

SuperSpike applies surrogate gradients and three-factor learning to train spiking networks using local synaptic information while avoiding backpropagation through time. It combines forward temporal credit assignment with random-feedback spatial credit assignment, but its scalability is limited in large multilayer networks.

  • SuperSpike: SuperSpike is a biologically plausible three-factor learning rule that uses surrogate gradients for supervised temporal learning.It replaces the non-differentiable spike derivative with a smooth surrogate derivative.
  • Hardware relevance: SuperSpike avoids BPTT, making the rule potentially relevant for hardware implementations and synaptic plasticity studies.The method uses synaptic eligibility traces to address temporal credit assignment.
  • Online learning: The learning dynamics can be simulated online from initial conditions to produce the derivatives needed for weight updates.The resulting updates depend only on local quantities.
  • Limitations: SuperSpike does not scale favorably for large multilayer networks because additional hidden layers or recurrent connections make the update equations more complicated and non-local.The multilayer extension addresses this by directly propagating error signals to hidden units.
  • Credit assignment: SuperSpike computes temporal credit assignment by propagating relevant quantities forward in time and spatial credit assignment through random backpropagation.For multilayer networks, output errors are propagated directly to hidden units using fixed random projections.

C. Learning using local errors

Local-error methods improve the scalability of surrogate-gradient learning by assigning layer-wise objectives through fixed random projections. DCLL retains spiking dynamics while achieving linear space complexity and performance comparable to standard backpropagation methods on DVS Gestures.

  • DCLL: DCLL uses local rate-based cost functions formed from spike-based basis functions, enabling learning that scales linearly with neuron count.The method trains feed-forward weights in a three-layer convolutional SNN using local errors from fixed random projections.
  • Local-error formulation: Local learning assigns each layer a loss using a pseudo-target and a fixed random projection to generate local errors.The auxiliary random layer is trained to match the pseudo-target, often chosen as the top-layer target.
  • Local representations: Each layer learns features that match the target after fixed random linear projection, with higher layers empirically producing higher-quality features.These features allow fixed auxiliary layers to better match the target.
  • Evaluation: DCLL performs on par with standard BP or BPTT rules on the DVS Gestures dataset.The approach was benchmarked on the event-based DVS Gestures dataset.
  • Temporal coding: Temporal XOR classification uses the identity of the first-spiking output neuron to distinguish the two classes.Four early-or-late input patterns are mapped to two output classes through a hidden layer.

VI. CONCLUSION

The paper presents surrogate-gradient methods as a framework for addressing the trainability problems of spiking neural networks. It connects machine learning, computational neuroscience, and neuromorphic hardware, while highlighting performance and implementation relevance.

  • Conclusion: Surrogate-gradient approaches can train SNNs to unprecedented performance levels across a range of real-world problems.The conclusion frames this transition as making SNNs increasingly relevant to applications previously dominated by RNNs.
  • Conclusion: Surrogate-gradient methods provide a framework connecting machine learning, computational neuroscience, and neuromorphic computing.Several presented methods are related to three-factor plasticity rules associated with synaptic plasticity.
  • Conclusion: The methods are relevant to low-power neuromorphic devices because they can learn under communication and storage constraints.The conclusion explicitly identifies this as a relevance for the neuromorphic community.
  • Conclusion: SNNs are less widely used than ANNs primarily because their trainability remains an algorithmic challenge.The conclusion identifies training issues as the central barrier to broader SNN use.
  • Conclusion: The paper surveys developments that address problems encountered when training SNNs and focuses specifically on surrogate-gradient approaches.It presents these approaches as a set of successful developments rather than a single definitive solution.
Loading 1901.09948v2…