Source-linked AI summary
Fast and energy-efficient neuromorphic deep learning with first-spike times
Julian Göltz, Laura Kriener, Andreas Baumbach, Sebastian Billaudelle, Oliver Breitwieser, Benjamin Cramer, Dominik Dold, Akos Ferenc Kungl, Walter Senn, Johannes Schemmel, Karlheinz Meier, Mihai Alexandru Petrovici
TL;DR
Spiking networks need coding and learning methods that preserve their biological dynamics while achieving fast, energy-efficient computation. The paper derives exact first-spike-time learning for leaky integrate-and-fire networks using spike-time information, implements error backpropagation, and evaluates the approach on BrainScaleS-2 and under substrate distortions. The hardware implementation combines similar classification accuracy with 48 µs and 8.4 µJ per classification, while simulations and hardware results support robustness to neuromorphic imperfections.
Problem
Spiking neural networks lag behind deep learning in performance and scalability because realistic neuronal dynamics complicate coding and error backpropagation.
Method
The paper derives analytical first-spike timing and exact gradients for leaky integrate-and-fire neurons, using pre- and postsynaptic spike times to implement hierarchical error backpropagation.
Results
BrainScaleS-2 achieved similar classification accuracies to software simulations with 48 µs and 8.4 µJ per classification, while the approach handled substrate-induced distortions.
Takeaways & Limitations
First-spike-time learning provides a spike-based error-backpropagation framework applicable to neuromorphic substrates with speed, energy, and distortion robustness demonstrated within the studied settings.
Takeaways & Limitations
Hardware networks were trained with updates calculated off-chip because the exact gradient rules are complex, although simplified rules showed promise for on-chip implementation.
Abstract
from arXiv · showhide
For a biological agent operating under environmental pressure, energy consumption and reaction times are of critical importance. Similarly, engineered systems are optimized for short time-to-solution and low energy-to-solution characteristics. At the level of neuronal implementation, this implies achieving the desired results with as few and as early spikes as possible. With time-to-first-spike coding both of these goals are inherently emerging features of learning. Here, we describe a rigorous derivation of a learning rule for such first-spike times in networks of leaky integrate-and-fire neurons, relying solely on input and output spike times, and show how this mechanism can implement error backpropagation in hierarchical spiking networks. Furthermore, we emulate our framework on the BrainScaleS-2 neuromorphic system and demonstrate its capability of harnessing the system's speed and energy characteristics. Finally, we examine how our approach generalizes to other neuromorphic platforms by studying how its performance is affected by typical distortive effects induced by neuromorphic substrates.
Introduction
Spiking neural networks offer sparse, fast, and energy-efficient processing, but finite neuronal dynamics and historically incompatible backpropagation have limited their performance and scalability. This paper develops exact first-spike-time learning for leaky neurons and evaluates it in software and neuromorphic hardware.
- Challenges: Finite response times, membrane dynamics, and spike-based communication make reliable coding and computation paradigms more difficult than in abstract neural networks.
- Motivation: Spiking neural networks retain temporal information in spike times, enabling sparse, fast, and energy-efficient processing through parallel computation.Time-to-first-spike coding further emphasizes early responses and few spikes.
- Time-to-first-spike coding: Time-to-first-spike coding represents a neuron's response by the elapsed time before its first spike, encouraging fast information processing with few spikes.The scheme is especially suited to tasks involving static information and neuromorphic systems.
- Contribution: Previous hierarchical TTFS learning used a continuous output-time function but was limited to non-leaky neurons, whereas this work targets current-based leaky integrate-and-fire neurons.
- Contribution: Analytical first-spike timing enables exact gradients and error backpropagation in hierarchical LIF networks, supporting universal classification on continuous and discrete data.
- Neuromorphic evaluation: BrainScaleS-2 achieved similar classification accuracies to software simulations with 48 µs and 8.4 µJ per classification, while the algorithm remained robust to substrate distortions.The framework adapts to hardware imperfections using information generated on the hardware during training.
Results
The derived learning rule enables exact error backpropagation in leaky integrate-and-fire networks and supports fast, accurate classification in software and on BrainScaleS-2. The approach also tolerates several substrate imperfections, although hardware comparisons and on-chip computation remain bounded by practical limitations.
- Learning rule: Exact gradients through differentiable spike-time dynamics solve credit assignment in appropriately parametrized LIF networks of arbitrary architecture.The derivation focuses on analytically tractable time-constant configurations and uses presynaptic spike times and synaptic weights.
- Software classification: 95.9±0.7% test accuracy was achieved on Yin-Yang across 20 random seeds, with errors only for samples extremely close to class borders.The trained label neurons show clear spike-time separation, with abrupt timing changes at class boundaries.
- Limitations and outlook: The method remains bounded by substrate variability, spike jitter, limited weight range and granularity, while direct numerical comparisons with other platforms are difficult because architectures and training methods differ.The authors also identify synaptic-plasticity complexity as a potential on-chip limitation and report possible future improvements of up to fourfold classification rate and threefold lower energy per classification.
- Robustness: Weight resolution preserved approximately constant performance down to 5 bit, while time-constant variability had no discernible effect when mean time constants matched learning-rule assumptions within 10-20%.The robustness to time-constant variation follows from using true output spike times during the backward pass.
Discussion
The paper presents exact first-spike-time learning for leaky integrate-and-fire networks and applies it to hierarchical error backpropagation. The framework supports fast, energy-efficient BrainScaleS-2 classification while remaining robust to substrate imperfections, though the reported hardware training uses off-chip updates.
- Contribution: Exact first-spike-time learning rules optimize spike times and implement spike-based error backpropagation in hierarchical feedforward networks.The reported experiments use networks with one and two hidden layers.
- Scope: The framework is not restricted to TTFS coding and can target other relationships between spike timings, recurrent structures, and multi-spike coding schemes.The paper identifies TTFS as the simplest, fastest, and most efficient case studied for static-pattern classification.
- Robustness: The framework incorporates output spike times into the backward pass, softening the requirement for exact membrane-to-synaptic time-constant ratios.This extends applicability to substrates including BrainScaleS-2.
- Robustness: The learning framework requires only spike times and is therefore robust to substrate-induced distortions across neuromorphic platforms.The paper connects this hardware compatibility to its minimal neuromorphic-hardware demands.
- BrainScaleS-2 performance: Including relaxation and communication, 10,000 MNIST samples are classified in less than 1 s at about 8.4 µJ per classification.The implementation achieves about 10 µs to classification after receiving the first spike.
- Limitation: The hardware networks were trained with updates calculated off-chip from emulated spike times because the exact gradient-based rules are complex.Simplified learning-rule simulations suggest a possible future on-chip implementation.
Methods
The methods derive spike-time learning and backpropagation rules for leaky integrate-and-fire neurons using causal presynaptic spikes and analytic spike-time expressions. The resulting synaptic rule implements exact error backpropagation on spike times and is compatible with neuromorphic implementation constraints.
- Learning-rule derivation: The derivation treats τm →∞, τm = τs, and τm = 2τs cases, using analytic expressions for the corresponding spike times.The τm = τs case uses l’Hôpital’s rule, while the τm = 2τs case becomes quadratic.
- Neuron model: The model assumes LIF dynamics with finite membrane and synaptic time constants, a difference-of-exponentials PSP kernel, and one relevant spike per neuron.Under TTFS, the second spike sum reduces to a single term.
- Causal inputs: The causal set C contains presynaptic spikes arriving before the output spike time T.Because T depends on C, the correct set is found by testing ordered prefixes of input spikes.
- Causal inputs: If no tested causal set yields a valid output time, the neuron is assigned T = ∞.The algorithm selects the earliest valid spike among candidate causal sets.
- Learning-rule derivation: For τm = τs, the spike-time equation is solved with the differentiable Lambert W function.The earlier threshold crossing requires selecting the real branch returning the larger W value.
- Backpropagation: Layer-wise weight updates use the chain rule and recursively propagated errors to implement exact error backpropagation on spike times.The resulting synaptic learning rule is equivalent to the equations presented in the main text.
- Hardware implementation: BrainScaleS-2 emulates analog neuron and synapse dynamics at 1000-fold accelerated time scales.The platform physically represents state variables such as membrane potentials and synaptic currents.
- Hardware implementation: A refractory period longer than τs generally ensures that neurons emit only one relevant spike for each input pattern.This operationally supports the TTFS assumption used by the method.
Data availability
The paper states that data are available from the authors upon request.
- Data are available from the authors upon request.
- The data-availability statement does not specify a public repository.
- Access therefore requires contacting the authors directly.
Code availability
The paper provides public code for the Yin-Yang dataset and states that other code is available from the authors upon request. It also documents the parameters used for software and BrainScaleS-2 experiments.
- Code availability: Code for the Yin-Yang dataset is available at the authors’ GitHub repository.
- Code availability: Other code is available from the authors upon request.
- Software experiments: Table A lists neuron, network, and training parameters for the Yin-Yang and MNIST software experiments.
- Hardware experiments: Table B lists network and training parameters for BrainScaleS-2 experiments on Yin-Yang and 16×16 MNIST.
- Hardware experiments: The BrainScaleS-2 neuron parameters are not listed in Table B because the chip determines them.
SI.A Learning with time-to-first-spike (TTFS) coding on BrainScaleS-1
The framework was tested on BrainScaleS-1, whose wafer-scale architecture and conductance-based neurons differ from BrainScaleS-2. Training adapted the synapse dynamics with global weight-scale factors to approximate current-based behavior.
- A four-class dataset with 7 × 7 input pixels was used to track accuracy, loss, label-layer spike times, and raster activity during training.The correct-class neuron was highlighted in the spike-time plots.
- BrainScaleS-1 uses wafer-scale interconnection and conductance-based rather than current-based neurons.These substrate properties differ from the BrainScaleS-2 setup and are relevant to the coding scheme and learning rules.
- Global weight-scale factors modeled reversal-to-leak potential distance and total conductance to approximate current-based synapses.The scaled weights enabled training with the existing learning rules despite conductance-based synapse dynamics.
SI.B Additional experiments
Additional MNIST simulations examined generalization under input noise and architectural changes, while also reporting baseline and training-configuration details. The experiments included larger hidden layers, deeper networks, and weight quantization.
- Additional MNIST runs compared noiseless training with training under noisy input spike times to quantify effects on generalization.The comparisons used the hyperparameters listed in Table A.
- The simulations also tested a larger hidden layer, a deeper network with two hidden layers, and weight quantization.
- Table SI.B1 reports baseline values from Table 2 and uses specified hyperparameters for 28 × 28 and 16 × 16 input resolutions.The main stated exception concerns the number of training epochs.
- Some networks were trained for 300 epochs, while another was trained for 150 epochs.
SI.C Robustness to post-training variations
The study evaluated robustness to post-training distortions that were absent during training, including parameter shifts and hidden-neuron deletion. Results were summarized across trained networks using medians, quartiles, and full ranges.
- Post-training evaluation tested distortions not present during training, including threshold and neuronal time-constant shifts.Such effects are especially relevant for analog circuits exposed to changing conditions.
- Hidden-neuron deletion was evaluated on 10 fully trained networks to model neural cell death in biological networks.Each deletion scenario was repeated because outcomes depend strongly on which neurons are removed.
- Figure SI.C1 reports medians, quartiles, and minimum-to-maximum ranges for threshold shifts, time-constant shifts, and random hidden-neuron deletion.
SI.D Simplification of the learning rule
The supplementary work replaces computationally expensive exponential and Lambert W evaluations with simpler approximations for on-chip learning. The simplified rule retained substantial task performance but reduced accuracy and became less stable during longer training.
- Computational motivation: The original τm = τs learning rule requires multiple exponential evaluations and the Lambert W function, making plasticity computationally demanding.Lambert W has no closed-form expression.
- Evaluation: 91.7 ± 1.4% test accuracy and 91.7 ± 1.2% training accuracy were achieved on Yin-Yang with the simplified learning rule across 10 seeds.The result was obtained after 150 training epochs.
- Approximation: The approximation replaces exponential functions and W with a constant, yielding updates composed only of simple differences and multiplications.This makes the rule more amenable to on-chip implementations.
- Evaluation: The simplified rule produced a small but noticeable test-accuracy drop versus the full rule and greater instability during longer training periods.The reported evaluation used the standard τm = τs setup and λ = 0.0192.
- Approximation: Eqn. (SI.D2) includes ti + τs, representing the postsynaptic-potential maximum and capturing the sign change around the output spike time.This difference encodes the major non-monotonic relationship in the time derivative.
SI.E Power consumption and execution time measurements
The measurements separate host, communication, and chip execution costs while comparing BrainScaleS-2 with a GPU reference. BrainScaleS-2 achieves 48 µs and 8.4 µJ per classification, whereas the GPU reference uses 8 µs and 852 µJ per image.
- BrainScaleS-2 measurements: BrainScaleS-2 consumed 175 mW during runtime, including communication links, digital periphery and clocking, and analog-circuit biasing.Power consumption did not significantly change between inference and an inactive-network emulation.
- Execution-time breakdown: The conservative 0.937 s round-trip time for 10,000 MNIST images includes host preprocessing, postprocessing, and communication.Once data reaches the chip, processing the test set takes 480 ms.
- Execution-time breakdown: 48 µs per classification corresponds to a rate of 20 800 images per second on BrainScaleS-2.The execution-time breakdown separates encoding, experiment, and decoding phases.
- Execution-time breakdown: Sequential sample presentation and waiting for residual activity to decay make the current 48 µs classification duration longer than the hardware time constants suggest.Artificial resets could reduce classification time to 11.8 µs while increasing power by approximately 20 mW.
- GPU reference: The GPU reference classified an image in 8 µs with 852 µJ per classification.The GPU measurements used a standard PyTorch convolutional network for MNIST.
- GPU reference: For a fully connected GPU network matching the BrainScaleS-2 hidden-layer size, energy per classification could only be estimated as a lower bound on the order of 10 µJ.The measurement resolution was insufficient because this network classified 20 to 25 times faster than the CNN.
SI.F Extended literature comparison
The supplementary comparison broadens the neuromorphic literature overview beyond studies reporting both energy and time measurements.
- Extended literature comparison: Table SI.F1 provides a more comprehensive overview of neuromorphic classifiers, including references without energy or time measurements.The comparison therefore includes studies with incomplete hardware-efficiency reporting.