Source-linked AI summary
Spiking Neural Networks Hardware Implementations and Challenges: a Survey
Maxence Bouvier, Alexandre Valentian, Thomas Mesquida, François Rummens, Marina Reyboz, Elisa Vianello, Edith Beigné
TL;DR
Whether SNN hardware is actually efficient relative to ANN accelerators remains an open question, complicated by input conversion and repeated SNN feedforward evaluation. This survey reviews SNN hardware implementations and algorithm choices; it reports that low-power SNN accelerators are slightly better than ANN ones in energy and area on MNIST, while noting the limits of that comparison.
Problem
How SNN processors compare with ANN accelerators remains an open question, complicated by SNN input conversion and repeated feedforward evaluation.
Source: §4.3 · p. 15Method
The survey synthesizes SNN hardware implementations and algorithm choices, including training approaches and hardware resource-reduction strategies.
Source: §5.2 · p. 19, §5.3 · p. 20, §8 · p. 26Results
On MNIST, current state-of-the-art low-power SNN accelerators are slightly better than ANN ones in energy and area.
Source: §8 · p. 27Takeaways & Limitations
The survey recommends benchmarks with inherent temporal information to assess SNN capabilities, while current low-power accelerator comparisons on MNIST favor SNNs slightly in energy and area.
Source: §8 · p. 27MNIST comparison is constrained by variable frame-to-event conversion, lacks temporal information, and contrasts a programmable ANN accelerator with topology-specific SNN hardware.
Source: §6.2 · p. 23, §8 · p. 27
Abstract
from arXiv · showhide
Neuromorphic computing is henceforth a major research field for both academic and industrial actors. As opposed to Von Neumann machines, brain-inspired processors aim at bringing closer the memory and the computational elements to efficiently evaluate machine-learning algorithms. Recently, Spiking Neural Networks, a generation of cognitive algorithms employing computational primitives mimicking neuron and synapse operational principles, have become an important part of deep learning. They are expected to improve the computational performance and efficiency of neural networks, but are best suited for hardware able to support their temporal dynamics. In this survey, we present the state of the art of hardware implementations of spiking neural networks and the current trends in algorithm elaboration from model selection to training mechanisms. The scope of existing solutions is extensive; we thus present the general framework and study on a case-by-case basis the relevant particularities. We describe the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level and discuss their related advantages and challenges.
1 INTRODUCTION
The growing computational demands of neural networks make their evaluation on Von Neumann architectures costly in time and power, while practical use—especially in embedded settings—remains constrained by hardware resources. The survey reviews SNN hardware and algorithms to examine their potential benefits and implementation challenges.
1 INTRODUCTION
- Neural networks’ increasing computational density makes evaluation on Von Neumann architectures require extreme time and power.This motivates dedicated accelerators, including neuromorphic processors.Source: §1 · p. 1
- Hardware resource requirements limit practical machine-learning deployment, particularly on battery-constrained devices that rely on cloud evaluation.The passage identifies low-power hardware and algorithm co-design as a need for expanding deep learning’s applicability.Source: §1 · p. 2
- Although SNN event-driven sparsity is expected to improve computation and communication efficiency, whether SNN hardware is effectively efficient remains unclear.The survey reviews SNN working principles, hardware adaptation, and strategies for leveraging performance gains.Source: §1 · p. 2
- The survey covers neural-network operation, hardware resource reduction, low-power suitability, training and on-chip learning, hardware implementations, and possible software and hardware improvements.It includes both large-scale and small-scale implementations, including those targeting potential embedded operation.Source: §1 · p. 2
2 SPIKING NEURAL NETWORKS WORKING PRINCIPLES
SNNs represent computation as weighted incoming spikes integrated over time, with neurons emitting spikes when membrane potentials cross thresholds. Their information coding and hardware mapping choices shape how these networks are evaluated on neuromorphic platforms.
2.1 Introduction to Neural Networks
- Unlike formal perceptrons, SNN neurons integrate weighted spike inputs over time and emit spikes when membrane potential exceeds a threshold.Formal perceptrons apply a nonlinear activation to a weighted sum; SNN spikes are binary signals that neurons integrate over time.Source: §2.1 · pp. 2–4
2.2 Information Representation Using Spiking Neurons
- SNN information can use rate coding, time-to-first-spike, or inter-spike-interval timing, while the appropriate representation remains under discussion.Rate coding represents information through mean firing rate over an observation period; time coding uses spike timing.Source: §2.2 · p. 4
2.3 General Hardware Implementation Strategy
- Dedicated neuromorphic hardware can exploit SNN sparsity by updating only neurons receiving spikes, unlike time-discretized simulation that evaluates every neuron at each step.The platform distributes memory and processing elements across neurocores, with cores communicating spikes through AER and a network-on-chip.Source: §2.3 · p. 5
- Depth mapping assigns one core per layer but can unevenly distribute computation, whereas spatial mapping time-multiplexes layers but can duplicate shared CNN weights.Implementations often mix both strategies to distribute neurocore activity more uniformly.Source: §2.3 · pp. 6–7
3 BRAIN INSPIRED SPIKING NEURAL NETWORKS
The survey describes brain-inspired SNN models and technologies spanning neuron and synapse dynamics, network-level mechanisms, resilient systems, and event-driven sensors. It distinguishes applications seeking low-power machine learning from those seeking biologically plausible dynamics.
3 BRAIN INSPIRED SPIKING NEURAL NETWORKS
- Brain-inspired computing serves both low-power machine learning and biologically plausible on-chip dynamics for neuroscience or interaction with biological systems.The survey treats machine-learning mechanisms separately from the brain-inspired mechanisms covered in this section.Source: §3 · p. 7
3.1 Computation in the Brain
- LIF neurons integrate inputs with leak, threshold, reset, and refractory mechanisms, while simpler IF models omit leakage and refractoriness to reduce hardware requirements.An IF implementation requires an adder, comparator, and memory for membrane-potential storage.Source: §3.1 · pp. 8–9
- Brain-inspired computation also includes WTA competition and population synchrony, while recurrent liquid state machines are used for time-dependent applications such as speech recognition.LSMs’ complex computational graphs may not map efficiently onto general neuromorphic hardware.Source: §3.1 · pp. 9–10
3.2 Brain-Inspired Technologies
- Neuroplasticity-inspired hardware can adapt and reconfigure, potentially removing redundant hardware in critical systems, while evolutionary hardware and auto-encoder networks are other resilience strategies.The passage describes neuroplasticity as adaptation and reconfiguration during lifetime operation, including connectional-structure modification and synapse adaptation.Source: §3.2 · p. 10
- Dynamic vision sensors and cochlea circuits output spikes through AER, with faster response times and greater dynamic range than frame-based sensors.These sensors use biologically plausible sensing mechanisms.Source: §3.2 · p. 10
4 NEUROMORPHIC MACHINE LEARNING ACCELERATORS
The survey reviews SNN hardware strategies for machine-learning accelerators, from neurocore organization and communication to sparse computation, fault-tolerant approximation, and crossbar arrays. It weighs their performance benefits against hardware and evaluation constraints.
4.1 Core Organization for Low Power Spiking Neural Network Evaluation
- Neuromorphic cores support parallel SNN evaluation, but their organization should match the target: embedded accelerators prioritize power and area, while very-large-scale designs prioritize flexibility and speed.Digital cores commonly time-multiplex neuron updates, which requires storing timing and neuron-state information; designs reduce costs through selective updates, parallelism, and spike scheduling.Source: §4.1 · pp. 11–13
4.2 Reduced Communication Cost
- AER sends a spiking neuron’s address asynchronously, reducing bus size, packet delay, power overhead, and network activity through SNN sparsity.The protocol scales across cores and chips, but system power grows with unit count, mean firing frequency, and chip size; rate coding sends more spikes than time coding.Source: §4.2 · pp. 13–14
4.3 Leverage High Sparsity of Dataflow
- Event-driven sensors emit data only when sensed phenomena change, while formal datasets require conversion to spikes, and the chosen conversion affects energy and accuracy.Visual inputs may be encoded as Poisson trains, continuous membrane inputs, latency-coded spikes, or sensor-generated events.Source: §4.3 · p. 14
- SNNs compare favorably with ANNs on event-driven datasets in accuracy and computation cost, whereas ANNs perform better on formal datasets.Comparisons are not straightforward because dataset conversion can bias results; on N-MNIST, one study reported 98.66% SNN accuracy versus 97.8% for an ANN, with 4× fewer operations for the SNN.Source: §4.3 · p. 15
4.4 Exploit Fault Tolerance of Neural Networks
- Quantization and stochastic synapses can reduce SNN energy or area and memory requirements, with reported accuracy trade-offs; neuron-level deactivation is another computation-skipping strategy.One reported SNN approach achieved 8× acceleration and 7.3× energy gain with a 0.25% accuracy loss on MNIST.Source: §4.4 · pp. 15–16
- Approximate processing elements using carry-skip adders reported speed and energy gains, but approximation may lose more accuracy on deeper networks and complex datasets.The survey cautions that scalability across tasks is uncertain, particularly for critical applications.Source: §4.4 · p. 16
4.5 Crossbar Array with Memristive Devices
- Crossbars place synaptic devices at intersections of orthogonal lines, using conductance-weighted currents to realize a dot product in memory.Reported gains over classical neurocores depend on network topology; a separate PCM study claimed potential power efficiency 1 to 2 orders of magnitude above standard CPUs or GPUs.Source: §4.5 · pp. 16–17
- Crossbar deployment remains constrained by device imperfections and reliability requirements, and most studies report simulations using theoretical device models.Small devices must also provide low-power reads and writes while remaining stable; crosspoint control and read-out circuitry can reduce density or add costs.Source: §4.5 · pp. 17–18
5 The Difficulty of Training a Spiking Neural Network
Training SNNs is difficult because spiking discontinuities impede gradient backpropagation, and neuron parameters require additional optimization. The survey reviews conversion, unsupervised and supervised learning, and hardware trade-offs, noting that supervised backpropagation is beginning to approach ANN accuracy while training choices depend on the target application.
5 The Difficulty of Training a Spiking Neural Network
- SNN training requires additional parameter optimization, and spiking discontinuities historically prevented direct gradient backpropagation.Thresholds and leak rates are sensitive parameters.Source: §5 · p. 18
5.1 Study of Conversion ANN to SNN
- ANN-to-SNN conversion has advanced to support techniques such as batch normalization and inception layers, but conversion accuracy can trail ANNs and rate coding reduces hardware power efficiency.Time-coding conversion can lower power, at the expense of final accuracy and applicability across ANN topologies.Source: §5.1 · p. 18
5.2 Unsupervised Learning – Autonomous Extraction of Pattern in Datasets
- STDP-based unsupervised learning extracts frequent features but may miss application-specific ones, and its layer-local learning limits multilayer accuracy.Strategies such as adding supervision, preprocessing inputs, or changing the learning rule can improve classification, but do not reach state-of-the-art supervised SNN or ANN performance. The survey positions these rules as research or preprocessing tools for extracting patterns in complex inputs.Source: §5.2 · p. 19
5.3 Supervised Learning – On the Road to Event Driven Machine Learning
- Supervised SNN methods include STDP-based training, analytical weight computation, and approximations that enable gradient calculation.Approaches to backpropagation address spike discontinuities through side functions, continuity-based models, or differentiable network models; some evaluate gradients along both depth and time, while neuron models vary in bio-plausibility and use of time.Source: §5.3 · pp. 19–20
- Supervised SNN backpropagation is starting to reach accuracies equivalent to formal neural networks, with some hardware-friendly models enabling low-power, high-accuracy inference.Gradient methods incorporating time evolution are being developed for event-driven datasets without conversion.Source: §5.3 · p. 20
5.4 On-Chip or Off-Chip Training?
- On-chip versus off-chip training depends on the application, while online adaptation entails input-memory needs and faces catastrophic forgetting.On-chip STDP can reduce inter-neurocore communication, but its circuitry or added memory may increase chip area and reduce power efficiency; supervised learning can require complex models or floating-point gradient communication.Source: §5.4 · pp. 20–21
6 Hardware Implementations: Comparisons and Discussions
The section compares SNN hardware across large-scale and embedded use cases, arguing that accelerator comparisons must account for network type, application constraints, and relevant criteria. On MNIST, SNN hardware shows favorable energy–accuracy tradeoffs, but the dataset and specialized designs qualify the comparison.
6 Hardware Implementations: Comparisons and Discussions
- Accelerator comparisons should first distinguish ANN from SNN, then account for application constraints and select criteria relevant to the target use.General machine-learning accelerators prioritize speed and flexibility, while embedded processors prioritize power, mapped-network accuracy, and die area.Source: §6 · p. 21
6.1 General Neural Network Accelerators
- Large-scale accelerators make different tradeoffs: SpiNNaker is highly reconfigurable but slower and less energy efficient, while Loihi supports flexible SNN evaluation and on-chip learning.BrainScaleS targets accelerated-time emulation of brain-sized networks with precise biological behavior; Neurogrid uses sub-threshold analog electronics for real-time emulation.Source: §6.1 · pp. 21–22
6.2 Low Power Spiking Machine Learning
- The MNIST comparison focuses on hardware-evaluated SNNs and includes a formal ANN accelerator reference, but some reported hardware results do not evaluate an SNN.Chen et al. report binary-network evaluation on neuromorphic hardware; their SNN proof of concept uses on-chip STDP for image reconstruction.Source: §6.2 · p. 23
- Simplifying a network can lower circuit cost at a substantial accuracy cost: Zheng et al. report a drop from 97.8% to around 90% on the same dataset.The changes included input down-sampling and approximate computing.Source: §6.2 · p. 23
- MNIST SNN hardware performs digit recognition at hundreds of nanojoules and above 96% accuracy, often using energy–accuracy tradeoffs that can reduce accuracy.The survey cautions that MNIST lacks temporal information and requires conversion to event-driven inputs, so it may not test SNN capabilities fully.Source: §6.2 · pp. 23–24
- With less than 0.2% accuracy reduction, Yin et al. [193] report 3× energy gain at iso-accuracy versus Whatmough et al..Yin et al. also report 99.40% accuracy in simulation; their hardware is designed for a single topology, trading flexibility for efficiency.Source: §6.2 · p. 23, Table 2 · p. 24
7 WHAT COMES NEXT?
The survey identifies algorithmic and hardware directions for improving SNN performance, while noting limits in training methods and technology maturity. Its examples include event-based coding, 3D integration, and mixed neural hardware.
7.1 Algorithm Improvements
- Temporal coding can reduce spike counts, but current training methods do not yet reach state-of-the-art accuracy across formal and event-driven algorithms.Reward-modulated STDP does not enable multilayer training; ANN-to-SNN conversion increases MNIST error from 1.04% to 1.43%, while adapting neurons can improve accuracy over traditional rate coding and reduce operations.Source: §7.1 · p. 25
7 WHAT COMES NEXT?
- SNNs can model bio-plausible dynamics and support machine-learning tasks, but no available solution stands out for machine learning, possibly because common benchmarks do not exploit SNN capabilities.The survey notes that researchers often compare SNNs with common ANNs.Source: §7.1 · p. 25
7.2 Hardware Implementations
- 3D integration offers high bandwidth, shorter interconnects, and potential parallelism, but remains immature, costly, and constrained by design requirements.Reported designs distribute memory and logic across tiers; stacking memory over logic can increase throughput, while distributed monolithic 3D memory and logic has been reported to save around 20% power versus 2D.Source: §7.2 · pp. 25–26, Figure 11 · p. 26
- Hybrid Neural Network hardware combines formal ANN processing with event-driven communication, converting between asynchronous events and frames; the cited FPGA implementation reaches 97.09% MNIST accuracy at 7uJ/frame.The authors describe it as a possible way to combine advantages of SNNs and ANNs while mitigating drawbacks.Source: §7.2 · p. 26
8 CONCLUSION
The conclusion surveys SNN hardware for both large-scale and embedded applications, emphasizing tradeoffs between resource use, accuracy, and flexibility. It finds favorable low-power SNN results on MNIST but cautions that the benchmark may not capture SNN capabilities and that emerging hardware technologies remain immature.
8 CONCLUSION
- SNN processors span deep-network evaluation and smaller embedded accelerators; weight quantization and stochastic computing can reduce memory and computation requirements, but may significantly reduce accuracy if applied carelessly.The survey observes increasing use of digital implementations and suggests bio-plausibility has not so far been associated with better algorithmic performance.Source: §8 · pp. 26–27
- On MNIST, current low-power SNN accelerators are slightly better than ANN counterparts in energy and area, but the cited SNN design targets a single topology while the ANN reference is programmable.The survey argues that MNIST may not fully exploit SNN capabilities and recommends benchmarks with inherent temporal information.Source: §8 · p. 27
- Crossbar arrays and 3D integration may improve neuromorphic hardware efficiency, but memristive crossbars do not yet meet requirements and 3D technology remains at an early research stage.Source: §8 · p. 27