Surrogate gradients for analog neuromorphic computing

Introduction

• Biological spiking neurons integrate inputs through analog membrane potentials and emit asynchronous binary spikes when they reach a firing threshold.Source: Introduction · p. 1
• The brain’s energy efficiency and fault tolerance are linked to spatiotemporal sparseness in how biological spiking neural networks process and exchange information.Source: Introduction · p. 1
• Digital neuromorphic systems simulate neuronal dynamics, whereas analog and mixed-signal systems represent neuronal or synaptic states with continuously evolving physical voltages, currents, or conductances.By leveraging substrate dynamics, neuromorphic computing is presented as a route to power-efficient, scalable SNNs.Source: Introduction · p. 1
• In-the-loop surrogate-gradient training addresses three barriers: binary spikes impede vanilla gradient descent, energy efficiency requires sparse activity, and analog devices have manufacturing-related imperfections.Source: Introduction · p. 1
• The trained networks solve challenging benchmarks with competitive accuracy comparable to software simulations, using sparse, precisely timed spikes and hardware acceleration for ultra-low-latency inference.The framework also achieves this through self-calibration, automatically correcting device mismatch without costly offline calibration.Source: Introduction · p. 1
• The framework extends surrogate gradients into general in-the-loop training and is applied to the mixed-signal BrainScaleS-2 single-chip system.Source: Introduction · p. 1

The BrainScaleS-2 analog neuromorphic substrate

• The chip routes hidden-layer spikes to output units and reads analog membrane potentials through a column-parallel converter for processing by the plasticity processing unit.Each connection is represented by a pair of excitatory and inhibitory hardware synapses holding a signed weight.Source: Figure 1 · p. 2
• 10^3 times faster than biological neurons, BrainScaleS-2’s analog membrane potentials evolve continuously according to LIF dynamics.The membrane is represented as a voltage across a capacitor, with leak conductance producing exponential decay.Source: The BrainScaleS-2 analog neuromorphic… · p. 2
• Each neuron integrates inputs from 256 synapses with 6-bit weights, while merging opposing-sign synapses enables continuous transitions between positive and negative weights during training.An event router propagates internal and external spikes to support feed-forward and recurrent topologies.Source: The BrainScaleS-2 analog neuromorphic… · p. 2
• Per-neuron adjustment of parameters, including reference potentials and time constants, supports calibration and deliberate decalibration to test learning-based self-calibration.The study actively decalibrates the system to explore the learning algorithm’s self-calibration properties.Source: The BrainScaleS-2 analog neuromorphic… · p. 2

In-the-loop learning framework

• On BrainScaleS-2, the framework records analog membrane traces and spikes, computes surrogate-gradient weight updates through backpropagation through time, and returns updated weights to the hardware.The learning algorithm also self-corrects for mismatch in analog-component parameters.Source: In-the-loop surrogate gradients on… · pp. 2–3

Recording spikes and analog membrane traces

• At about 0.6 MSample s−1 (1.7 µs intervals), BrainScaleS-2’s column-parallel CADCs digitize all neuron membrane potentials while the system continuously records and time-stamps spikes.For 256 neurons, this corresponds to a total data rate of 1.2 Gbit s−1; training samples are buffered for asynchronous transfer to the host.Source: In-the-loop surrogate gradients on… · p. 3

A computation graph for analog circuits

• The computation graph approximates substrate dynamics but inserts normalized recorded membrane traces in place of estimates, using measured values for relevant backward-pass derivatives.The graph uses model estimates for unmeasured internal variables, such as synaptic currents; spike trains are aligned to the membrane-sampling time grid, with finer resolution possible using interpolated traces.Source: In-the-loop surrogate gradients on… · p. 3

Flexible choice of a loss function

• Any differentiable loss formulated on recorded spikes or membrane voltages can be used, with regularizers for generalization, hardware constraints, or sparse firing.The framework also supports rate- and spike timing-based coding schemes.Source: In-the-loop surrogate gradients on… · pp. 2–3

Low-latency neuromorphic computation

• With an artificial reset, SNNs classified more than 85 k images per second at 8 µs latency; the full chip consumed approximately 200 mW, or 2.4 µJ per image.The reset enabled image inputs separated by 11.8 µs.Source: Figure 3 · p. 4, Results · p. 5

Efficiency through sparse spiking activity

• Test accuracy remained 97.2% with approximately 20 hidden-layer spikes per image, but declined slightly at 10 spikes, where networks relied on individual spikes and their timing.The activity-shaping term could reduce activity without imposing an upward homeostatic force, and training could start from a quiescent hidden layer.Source: Figure 3 · p. 4, Results · pp. 4–5

MNIST

• MNIST test accuracy reached (97.6 ± 0.1) % on BrainScaleS-2 with augmentation plus dropout, versus (97.5 ± 0.1) % in software and best-effort (98.7 ± 0.1) % for the ANN.The ANN result also used dropout and augmentation; the software comparison was trained without hardware-specific constraints.Source: Table 1 · p. 4, Results · p. 4

Self-calibration through ITL learning

• Learning performance was hardly affected by mismatch up to σd = 30%; beyond that, errors gradually increased and some configurations developed pathological supra-threshold resting potentials.ITL learning self-calibrated the analog SNNs despite mismatch levels far above those expected for BrainScaleS-2 and similar systems.Source: Figure 4 · p. 5, Results · pp. 5–6

Training for robustness

• With 40% dropout during training, silencing 15% of hidden units increased test error by only 10%, compared with 37% without dropout.Dropout regularization therefore improved resilience to neuron failures after deployment.Source: Figure 4 · p. 5, Results · p. 6

Speech recognition with recurrent SNNs

• On SHD, recurrent BrainScaleS-2 SNN test accuracy improved from (76.2 ± 1.3) % to (80.6 ± 1.0) % with augmentation; augmented software reached (79.9 ± 0.7) %.The test set included 81% of two speakers absent from training; intrinsic analog noise may have regularized the software-versus-hardware performance gap.Source: Results · p. 6

Discussion

• The framework combines surrogate gradients with in-the-loop training and massively parallel membrane-potential digitization for recurrent and multi-layer SNNs, achieving state-of-the-art benchmark accuracy comparable to software simulations while correcting device mismatch without explicit calibration.The resulting networks exhibited spatially and temporally sparse activity and could be optimized for resilience to neuron failure.Source: Discussion · p. 6
• The discussion identifies analog drift and manufacturing variability as sources of degraded performance for software-trained models loaded onto analog substrates, and presents ITL as a way to account for device-specific non-idealities and heterogeneity.The authors also note that on-chip training solutions are not yet mature.Source: Discussion · p. 6
• On the SHD speech task, the recurrent network developed useful representations over training and generalized to unseen data from most speakers, while two exclusive speakers contributed 81% of test samples.The caption notes that classification varied across the 20 classes, including phonemically similar digit names.Source: Figure 5 · p. 7
• BrainScaleS-2 performed competitively across metrics: only much larger or convolutional networks surpassed its accuracy, only optimized smaller-node architectures its energy footprint, and it set throughput and latency benchmarks.The authors note that neuromorphic comparisons are complicated by a lack of standardized benchmarks and heterogeneous energy-measurement conventions.Source: Table 2 · p. 7, Discussion · p. 8
• Surrogate-gradient ITL extends beyond rate-based and time-to-first-spike coding, flexibly interpolating between them on multi-layer and recurrent architectures to improve performance and energy efficiency while supporting fast inference.The passage contrasts this extension with prior ITL approaches using rate-based or time-to-first-spike coding schemes.Source: Discussion · p. 7
• Because neuromorphic comparisons lack standardized benchmarks and use inhomogeneous energy-measurement conventions, the authors included MNIST for its widespread adoption despite its limited temporal spike-based information.They attempted comparisons with earlier digital and analog systems.Source: Discussion · pp. 7–8
• The work is a first step toward on-chip learning algorithms that could exploit device heterogeneity to support complex, energy-efficient, ultra-low-latency information processing.The passage cites prior work as [39].Source: Discussion · p. 8

Software environment

• The training framework used PyTorch auto-differentiation and the BrainScaleS-2 software stack to configure the neuromorphic system and run experiments.Source: Materials and methods · p. 8

Input coding

• MNIST images were reduced to 16×16 pixels, and normalized grayscale values were converted to LIF input currents that trigger spikes when sufficiently strong.The two outermost rows were discarded before scaling; spike times depended on the input-unit time constant and threshold.Source: Materials and methods · p. 8
• For SHD, the 700 input channels were subsampled, spike times were shortened by a factor of 2000, and augmentation reassigned spikes to neighboring channels before downsampling.The time scaling accounted for the system’s acceleration factor of 1000 and reduced host computation burden.Source: Materials and methods · p. 8

Weight scaling

• Weights were scaled, rounded, and cropped to the substrate’s 7-bit signed-integer resolution, with scaling accounting for analog bias currents and other technical parameters.MNIST output weights were dynamically scaled by aligning the largest absolute software weight with the substrate maximum.Source: Materials and methods · p. 8

• B. Cramer, S. Billaudelle, and F. Zenke conceived the work; Cramer and Billaudelle implemented the software, performed experiments, and analyzed the data.Source: Contributions · p. 8
• J. Schemmel led the design and architecture of BrainScaleS-2, with system-design contributions from A. Grübl, V. Karasenko, C. Pehle, K. Schreiber, and Y. Stradmann.Source: Contributions · p. 8
• S. Kanya and A. Leibfried contributed to implementation, and J. Weis contributed calibration routines.Source: Contributions · p. 8
• B. Cramer, S. Billaudelle, and F. Zenke wrote the manuscript with input from the other authors.Source: Contributions · p. 8