Source-linked AI summary
Gradient Descent for Spiking Neural Networks
Dongsung Huh, Terrence J. Sejnowski
TL;DR
Spiking networks are difficult to train because discrete spikes are non-differentiable, limiting gradient-based supervised learning despite the brain’s dynamic spike-based computation. The paper introduces a differentiable formulation with exact gradient calculation and demonstrates optimization across fast spike interactions and delayed behavioral dynamics. The method accurately optimizes both individual-spike and behavioral time scales, supporting a general-purpose supervised learning approach for spiking networks.
Problem
Spiking networks lack an efficient general supervised learning method because their discrete, all-or-none spikes are non-differentiable.
Method
The paper introduces a differentiable spiking-network formulation and derives exact gradients using backpropagation through time for gradient-based optimization.
Results
The method accurately optimized recurrent spiking dynamics on both millisecond spike-interaction and longer behavioral time scales across dynamic tasks.
Takeaways & Limitations
The approach offers a general-purpose supervised learning algorithm for spiking neural networks and supports further study of spike-based computation.
Takeaways & Limitations
Some prior methods remain limited because they target spiking activity of individual neurons at desired times, constraining their applicability.
Abstract
from arXiv · showhide
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by the lack of efficient supervised learning algorithm for spiking networks. Here, we present a gradient descent method for optimizing spiking network models by introducing a differentiable formulation of spiking networks and deriving the exact gradient calculation. For demonstration, we trained recurrent spiking networks on two dynamic tasks: one that requires optimizing fast (~millisecond) spike-based interactions for efficient encoding of information, and a delayed memory XOR task over extended duration (~second). The results show that our method indeed optimizes the spiking network dynamics on the time scale of individual spikes as well as behavioral time scales. In conclusion, our result offers a general purpose supervised learning algorithm for spiking neural networks, thus advancing further investigations on spike-based computation.
1 Introduction
Spiking neural networks capture dynamic spike-based computation but are difficult to optimize because discrete spikes are non-differentiable. The paper introduces a differentiable formulation with exact gradients for supervised optimization of recurrent spiking dynamics.
- Brain computation spans millisecond neuronal spiking and slower recurrent dynamics, whereas rate-based models use static analog outputs and miss fast spike-based dynamics.
- Discrete all-or-none spikes make standard gradient-based supervised learning incompatible with spiking neural networks.A spike is generated when membrane voltage crosses threshold and is absent otherwise.
- Earlier approaches optimized spike times or treated desired spike generation as binary classification, but spike creation and deletion remain non-differentiable.
- Targeting individual neurons’ spiking activity at desired times significantly limits the applicability of some prior methods.
- Alternative methods optimize static analog networks and replicate their solutions with spiking networks, extending applicability beyond direct spiking-network optimization.
- The paper introduces a differentiable spiking-network formulation and derives exact gradients to optimize recurrent dynamics at individual-spike time scales for general supervised learning.
2 Methods
The method replaces threshold-triggered, non-differentiable synapses with a differentiable active-zone formulation that preserves spike-like charge responses. It embeds these synapses in recurrent networks and computes exact gradients through backpropagation through time.
- Differentiable synapse model: Threshold-triggered synaptic currents generate discrete, all-or-none responses that are non-differentiable.This non-differentiability prevents direct use of standard gradient-based supervised learning methods.
- Differentiable synapse model: The gate function g(v) replaces the threshold with a non-negative, unit-integral function of narrow support called the active zone.Synaptic current is activated gradually as the presynaptic voltage traverses this zone.
- Differentiable synapse model: Including g v̇ in the synaptic dynamics preserves constant total charge for supra-threshold depolarizations regardless of depolarization rate.The derivative term also ensures dimensional consistency with the Dirac-delta formulation.
- Differentiable synapse model: Below-zone depolarizations produce no response, whereas within-zone depolarizations produce graded responses; the zero-width limit recovers threshold-triggered dynamics.Thus the formulation generalizes the threshold model while retaining equal responses for supra-threshold events.
- Network model: Network dynamics are formed by coupling differentiable neuron and synapse dynamics through recurrent, input, and tonic currents, with linear synaptic-current readout.W, U, O, and I_o are optimized against a task cost function.
- Gradient calculation: Exact gradients are calculated with backpropagation through time, which scales as O(N^2) rather than O(N^3) for real-time recurrent learning.The gradient calculation uses adjoint state dynamics and links presynaptic input with postsynaptic adjoint activity.
3 Results
The study trains recurrent spiking networks on dynamic tasks using stochastic gradient descent, demonstrating balanced and efficient spike-based coding across millisecond and longer behavioral time scales.
- Training procedure: Mini-batches of approximately 50 training examples were used to calculate average-cost gradients for stochastic gradient descent optimization.The cost penalized readout error and overall synaptic activity.
- Predictive coding task: The predictive-coding framework provides analytical solutions that serve as benchmarks and reproduces balanced excitatory-inhibitory inputs and efficient coding.The task uses linear dynamic input-output relationships and 30 NIF neurons.
- Auto-encoder task: The auto-encoder task used 30 NIF neurons to represent low-pass-filtered sinusoidal inputs with the least number of spikes.The network had 2 input and output signals, with sinusoid periods of 1200 ms.
- Auto-encoder task: The trained auto-encoder accurately tracked the desired output while fast recurrent input largely canceled external input, reducing spike outputs from 29 to 6 in the illustrated neuron.The trained fast recurrent weights approximately followed Wf ≈ −UO, while W was set to zero.
- Predictive coding task: For predictive coding, the trained network accurately tracked the desired output and learned regular recurrent weights approximately matching W ≈ UAO.Figure 3 compares the trained and predicted regular recurrent weight matrices.
- Interpretation: The results show that the algorithm optimizes millisecond spike interactions for efficient encoding and achieves efficient coding without instantaneously fast synapses.Finite time constants were assigned to fast and regular synapses to avoid unstable dynamics.
D E F
The optimized QIF network solves delayed-memory XOR by sustaining time-varying activity that preserves input history, produces go-cue-triggered outputs, and returns to background activity. This demonstrates nonlinear computation over extended time, although the precise mechanism remains unresolved.
- Task: The delayed-memory XOR task requires storing two signed input pulses and producing positive, negative, or null output when the go-cue arrives.Opposite-sign inputs require a positive pulse, equal-sign inputs a negative pulse, and the null case is also specified.
- Network: 80 QIF neurons solved the task without extra dendritic nonlinearity despite a task timescale of approximately 500 ms.The network used τv = 25 ms, τf = 5 ms, and τ = 20 ms.
- Results: 14 NIF networks failed because the memory requirement drove training toward strong recurrent connections and runaway excitation.This failure is reported as a contrast to the successful QIF network.
- Results: The trained network maintained input history through sustained, time-varying spike activity, generated correct go-cue-triggered outputs, and then returned to background activity.The authors report successful task performance in Figure 4.
- Interpretation: The result shows that the algorithm can optimize spiking networks for nonlinear computations over extended time.The exact underlying computational mechanism requires further analysis.
4 Discussion
The paper presents exact gradient-based learning for differentiable spiking networks and reports applicability across individual-spike and behavioral timescales. It also acknowledges discrepancies from biological learning and scope boundaries tied to neuron-model differentiability.
- Contribution: The method introduces a differentiable spiking-network formulation and derives a gradient calculation for supervised learning.This is presented as the paper’s core contribution.
- Scope: The method optimizes recurrent spiking dynamics for general supervised tasks at both individual-spike and behavioral timescales.The discussion contrasts this scope with previous learning methods.
- Biological relevance: Exact gradient-based learning inevitably differs from biological learning processes.The paper notes that the gradient update is sparsely compressed near spike times and resembles reward-modulated STDP.
- Scope: The formulation was coupled to differentiable single-state spiking neurons, while other neuron models may require additional gradient-calculation procedures.The paper names Hodgkin-Huxley, Morris-Lecar, FitzHugh-Nagumo, leaky integrate-and-fire, exponential integrate-and-fire, and Izhikevich models as possible extensions.
Supplementary Materials: Gradient calculation for the spiking neural network
The supplementary derivation uses Pontryagin’s minimum principle to formulate adjoint dynamics for the spiking network. It then simplifies the adjoint variables and obtains parameter gradients by integrating Hamiltonian derivatives.
- Hamiltonian: Pontryagin’s minimum principle defines a Hamiltonian for the spiking-network dynamics and cost function.The Hamiltonian includes adjoint-state terms for membrane voltage and synaptic current.
- Adjoint states: The adjoint variables correspond respectively to each neuron’s membrane voltage and synaptic current, while l(s⃗) is the cost function.These variables support the backward, or back-propagating, dynamics.
- Change of variables: A change of variables simplifies the adjoint formulation by defining pv ≡ p̄v + gp̄s/τ and ps ≡ p̄s/τ.The transformed variables are used to obtain the simplified dynamics.
- Gradient calculation: The total-cost gradient is obtained by integrating the Hamiltonian’s partial derivative with respect to each model parameter.Examples include recurrent, input, tonic-current, and output parameters.