Source-linked AI summary
SuperSpike: Supervised learning in multi-layer spiking neural networks
Friedemann Zenke, Surya Ganguli
TL;DR
The paper addresses how to train multi-layer spiking networks for nonlinear transformations of precisely timed spike patterns, despite non-differentiable neurons and difficult hidden-layer credit assignment. It derives SuperSpike, a surrogate-gradient, voltage-based three-factor rule, and evaluates uniform, symmetric, and random feedback. Simpler tasks work with all feedback types, whereas more complex tasks require symmetric feedback.
Problem
Learning and computation in biological spiking circuits remain poorly understood, while training artificial SNNs is hindered by temporal coding, non-differentiable spikes, reset dynamics, and hidden-layer credit assignment.
Method
The paper derives SuperSpike, a surrogate-gradient voltage-based three-factor rule for deterministic multi-layer LIF networks and applies it with different feedback strategies.
Results
Simpler tasks were solved with uniform, symmetric, or random feedback, but complex spatiotemporal tasks performed consistently worse with random feedback than symmetric feedback.
Takeaways & Limitations
The approach enables study of precisely timed spiking dynamics in deterministic multi-layer networks performing spatiotemporal spike-pattern transformations.
Takeaways & Limitations
The derivation neglects a self-history correction to the membrane-potential derivative, justified when firing rates are low.
Abstract
from arXiv · showhide
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico. Here we revisit the problem of supervised learning in temporally coding multi-layer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three factor learning rule capable of training multi-layer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike-time patterns.
1 Introduction
Spiking neural networks model communication through spikes, but their learning and computation remain poorly understood and difficult to train. The paper addresses this gap by targeting multi-layer networks that transform spatiotemporal spike patterns while examining biologically plausible credit-assignment strategies.
- Biological circuits primarily communicate through spikes, yet the principles by which experience sculpts their computations remain poorly understood.
- Training SNNs is difficult because spike timing matters, neuron outputs are non-differentiable, resets create self-memory, and hidden-layer credit assignment is problematic.
- Rate-based approaches largely overlook information carried by individual spike timing, which may support efficient coding and fast computation.
- The paper develops a learning rule for multi-layer deterministic LIF networks performing spatiotemporal spike-pattern transformations beyond spatial rate-based activations.
- 1.1 Prior work: Prior multi-layer approaches ported Backprop to spiking networks, while SpikeProp instead optimized differentiable firing-time expressions and was initially limited to one spike per neuron.
- 1.1 Prior work: The proposed method trains deterministic multi-layer LIF networks from initially silent hidden units without injecting noise by using presynaptic filtering and postsynaptic voltage dependence.
2 Derivation of the SuperSpike learning rule
SuperSpike derives a surrogate-gradient rule for training LIF neurons against target spike trains. Its update combines filtered presynaptic activity, postsynaptic voltage, eligibility traces, and neuron-specific error signals in a three-factor mechanism.
- The derivation formulates single-neuron training as minimizing van Rossum distance between a target spike train and the neuron's actual output.
- The method replaces the problematic spike-train derivative with a continuous auxiliary function of membrane voltage to obtain a usable surrogate gradient.
- The membrane potential is represented with a causal membrane kernel for postsynaptic potentials and a reset kernel capturing spike dynamics and self-history.
- Neglecting a low-rate self-history correction approximates the membrane-potential derivative by filtered presynaptic activity, interpreted as synaptic neurotransmitter concentration.
- SuperSpike uses an error signal, an eligibility trace, and a learning rate, with minibatch evaluation and per-parameter rates used in practice.
- The resulting rule is a voltage-based nonlinear Hebbian three-factor rule that combines multiplicative pre- and postsynaptic activity.
3 Methods
The methods train deterministic LIF spiking networks with SuperSpike and model membrane dynamics using current-based synaptic inputs, threshold-triggered spikes, refractory clamping, and axonal delays.
- Learning approach: SuperSpike trains deterministic LIF networks with supervised learning using a non-vanishing surrogate gradient.The simulations use 0.1 ms temporal resolution and the Auryn simulation library.
- Neuron dynamics: Current-based synaptic input supports an integral formulation, while membrane voltage is simulated with a differential equation.The synaptic current evolves from presynaptic spike arrivals and synaptic weights.
- Spike generation: A neuron emits a spike when its membrane voltage exceeds threshold, then remains clamped at resting potential for a 5 ms refractory period.Spikes propagate to other neurons with a 0.8 ms axonal delay.
3.2 Stimulation paradigms
The experiments use distinct stimulation paradigms for exact spike-time learning and classification, with frozen Poisson inputs repeated against consistently aligned targets in the former case.
- Exact spike-time tasks: Exact output-spike-time experiments use frozen Poisson spike trains generated from a single draw across the input units.The inputs have a specified duration and are repeated in a loop for target association.
- Model specification: The neuron-model parameter information is presented in Table 1.
- Exact spike-time tasks: Repeated frozen inputs are paired with target spike trains consistently aligned to each repetition.The stimulus and target trains are publicly available in the Supervised Spiking Benchmark Suite.
- Classification tasks: Classification experiments use different stimuli drawn as random firing-time offsets from a common onset, with stimulus order randomized.
3.3 Plasticity model
SuperSpike is organized as a nonlinear Hebbian three-factor rule that combines presynaptic activity, postsynaptic depolarization, and propagated error signals through synaptic eligibility traces.
- Plasticity rule: SuperSpike detects presynaptic–postsynaptic coincidences using a nonlinear Hebbian term based on presynaptic activity and postsynaptic depolarization.
- Eligibility traces: Temporal convolution stores each synapse’s coincidence signal transiently as a causal synaptic eligibility trace.The trace represents a temporary record of spatiotemporal coincidence at synapse w_ij.
- Learning-rule evaluation: Evaluating the learning rule involves presynaptic traces, Hebbian coincidence and eligibility computation, error propagation, and weight updates.
3.1 Presynaptic traces
Presynaptic and eligibility traces are computed online with exponential filters, while surrogate coincidence detection uses a fast-sigmoid derivative and per-synapse traces create O(n^2) scaling.
- Presynaptic traces: Double-exponential presynaptic filters are evaluated online by integrating one exponential trace and feeding it into a second filter array.
- Presynaptic traces: The effective postsynaptic-potential trace uses rise and decay time constants of 5 ms and 10 ms, respectively.
- Coincidence detection: Hebbian coincidence detection evaluates the surrogate derivative σ′(U_i) at every time step using the negative-half derivative of a fast sigmoid.
- Coincidence detection: The surrogate derivative is computed as σ′(U_i) = (1 + |h_i|)^−2, with h_i = β(U_i − ϑ) and β = 1 mV unless otherwise stated.
- Trace alignment: Delayed presynaptic traces and surrogate derivatives are combined at each time step, with delay Δ offsetting the 0.8 ms axonal delay.Presynaptic traces below 10^-7 are treated as zero to accelerate computation.
- Computational cost: Eligibility traces require filtering for every synapse, giving the algorithm O(n^2) complexity in the number of neurons.This per-synapse filtering is identified as the most obvious target for future optimization.
3.3 Error signals
The study distinguishes output error signals, tied directly to target-bearing output units, from feedback signals derived by sending those errors to hidden units. Error definitions differ between precisely timed spike generation and classification with timing slack.
- Output errors are directly tied to output units with target signals, whereas feedback signals transmit derived output errors to hidden units.
- For precisely timed output spikes, e_i = α ∗ (Ŝi − S_i) vanishes only when target and output spike trains match temporally.
- Classification errors penalize additional spikes immediately through negative feedback and provide positive feedback only at the end of trials with a missed output spike.
3.4 Feedback signals
Hidden units receive one of three feedback strategies: symmetric, random, or uniform. These range from weight-based downstream error propagation to random projections or a shared global signal.
- Hidden units were assigned symmetric, random, or uniform feedback to investigate alternative credit assignment strategies.
- Symmetric feedback propagates downstream errors using the actual feed-forward weights, without downstream activation functions.
- Random feedback uses coefficients drawn from a zero-mean, unit-variance normal distribution to form random error projections.
- Uniform feedback sets all weighting coefficients to one, approximating a single global third factor distributed across neurons.
3.5 Weight updates
Weight updates integrate error or feedback signals with synaptic eligibility traces over intervals, while a per-parameter RMax-Prop heuristic adapts learning rates for more stable training.
- The product of error or feedback signals and eligibility traces is accumulated in m_ij over chunks of t_b = 0.5257s before weight updates.
- For stimuli longer than t_b, chunked accumulation acts as a continuous-time analogue of mini-batch optimization.
- Per-parameter learning rates use an auxiliary variance estimate v_ij with slow decay, yielding r_ij ≡ r_0 √v_ij.
- The max-based RMax-Prop variant improved training stability and convergence times, with r_0 selected from (10, 5, 1, 0.5, 0.1) × 10^-3.
3.7 Regularization term
Some random-feedback experiments add a heterosynaptic regularizer to hidden-layer learning to prevent pathologically high firing rates. Its error dependence makes the penalty vanish when the task is solved perfectly, while fourth-power rate dependence strengthens high-rate regularization.
- A heterosynaptic regularization term was added to hidden-layer weights in some random-feedback experiments to avoid pathologically high firing rates.
- The regularizer depends on the square of the error signal, so it becomes zero when the task is solved perfectly.
- A fourth-power exponential synaptic trace and rate dependence were used to regularize high firing rates more strongly.
4 Numerical experiments
SuperSpike learned precisely timed spike outputs, nonlinear classification, and complex spatiotemporal transformations in deterministic spiking networks. Feedback strategy mattered little on simpler tasks but became decisive for the most challenging pattern-transformation task, where symmetric feedback performed best.
- Single-neuron learning: A single LIF neuron learned five equidistant target spikes from repeating Poisson inputs, starting from sub-threshold dynamics without initial spiking.The experiment tested whether SuperSpike could learn a predefined 500ms spike pattern from a quiescent state.
- Learning in multi-layer networks: Multi-layer networks with four hidden LIF neurons learned precisely timed output spikes using random, symmetric, or uniform feedback.Random feedback sometimes failed when all feedback connections were negative, whereas increasing hidden-layer width made this failure unlikely.
- Nonlinear classification: Random feedback quickly solved a noisy, nonlinearly separable spiking classification task with perfect accuracy, while a network without hidden units could not solve it.Symmetric feedback also learned the task quickly, and classification used output windows rather than precisely timed spikes.
- Complex spatiotemporal transformations: With 32 or more hidden units, symmetric feedback produced a visually matching 3.5-second pattern from cyclically repeating frozen Poisson input.Hidden activity after learning had intermediate firing rates of 10–20Hz and an approximately exponential inter-spike-interval distribution, although spurious output spikes remained.
- Limits of random feedback: Random feedback performed substantially worse than symmetric feedback on the complex pattern task, even falling below the no-hidden-layer baseline.Heterosynaptic regularization improved random-feedback learning but did not reach symmetric-feedback performance; disabling voltage nonlinearity degraded the output pattern.
- Limits of random feedback: The challenging transformations made learning-rule nonlinearity, firing-rate regularization, and non-random feedback more important for good performance.This pattern contrasted with simpler tasks, where networks generally reached zero error across feedback strategies.
5 Discussion
The paper presents SuperSpike as a deterministic three-factor rule for training multilayer LIF networks and examines biologically motivated feedback strategies for hidden-unit credit assignment.
- SuperSpike trains deterministic multilayer LIF spiking networks using a three-factor learning rule for precisely timed spike dynamics.The rule combines surrogate gradients with straight-through estimators and feedback-alignment ideas.
- The approach avoids the extraneous noise introduced by stochastic models while training precise spatiotemporal spike-pattern transformations.The paper contrasts its deterministic surrogate-gradient method with stochastic gradient approximations that can impede precise learning.
- The SuperSpike update is a voltage-based nonlinear Hebbian rule with individual synaptic eligibility traces.Its components have proposed biological interpretations involving voltage dependence, temporal coincidence, calcium transients, and neuromodulatory error signals.
- The discussion argues that learning may require higher-dimensional neuromodulatory or electrical feedback rather than a single global feedback signal.Such feedback may contain some knowledge of the feedforward pathway.