Source-linked AI summary
Six networks on a universal neuromorphic computing substrate
Thomas Pfeil, Andreas Grübl, Sebastian Jeltsch, Eric Müller, Paul Müller, Mihai A. Petrovici, Michael Schmuker, Daniel Brüderle, Johannes Schemmel, Karlheinz Meier
TL;DR
The paper addresses the need for efficient, flexible neural-network emulation by presenting a user-friendly universal neuromorphic substrate. Its mixed-signal Spikey chip combines configurable analog neuron and synapse circuits with digital spike communication, and the system successfully emulates six diverse networks while retaining hardware-imposed boundaries.
Problem
Existing neuromorphic systems often restrict network topologies and neuron or synapse parameters, limiting their flexibility compared with conventional simulations.
Method
The study presents a user-friendly, mixed-signal neuromorphic substrate using configurable analog neuron and synapse circuits with digital spike transmission.
Results
The system successfully emulates six neural networks spanning substantially different structures and functions.
Takeaways & Limitations
The demonstrated substrate supports diverse cortical, preprocessing, and machine-learning network uses through different connectivity and parameter configurations.
Takeaways & Limitations
The substrate cannot readily emulate models requiring neuronal adaptation or exotic synaptic plasticity, and network size is bounded by available neurons and synapses.
Abstract
from arXiv · showhide
In this study, we present a highly configurable neuromorphic computing substrate and use it for emulating several types of neural networks. At the heart of this system lies a mixed-signal chip, with analog implementations of neurons and synapses and digital transmission of action potentials. Major advantages of this emulation device, which has been explicitly designed as a universal neural network emulator, are its inherent parallelism and high acceleration factor compared to conventional computers. Its configurability allows the realization of almost arbitrary network topologies and the use of widely varied neuronal and synaptic parameters. Fixed-pattern noise inherent to analog circuitry is reduced by calibration routines. An integrated development environment allows neuroscientists to operate the device without any prior knowledge of neuromorphic circuit design. As a showcase for the capabilities of the system, we describe the successful emulation of six different neural networks which cover a broad spectrum of both structure and functionality.
1. INTRODUCTION
Neuromorphic hardware offers parallel, accelerated neural-network emulation, but typically restricts topology and parameter flexibility. The study presents a configurable, user-friendly substrate designed to address these constraints and demonstrates it across six networks.
- 1. INTRODUCTION: Neuromorphic devices physically implement neurons and synapses, providing inherent parallelism that avoids the usual slowdown as emulated networks scale.Their time constants can also be reduced relative to biology to accelerate emulations.
- 1. INTRODUCTION: Conventional simulations offer broad model flexibility, whereas many neuromorphic systems restrict network topology and parameter space to predefined applications.Increasing configuration space consumes hardware resources and can reduce network size or precision.
- 1. INTRODUCTION: The proposed system provides almost arbitrary network topologies and a wide configuration space for neuron and synapse parameters through on-chip reconfiguration.Models may require hardware-specific modifications to accommodate limited resources and compensate for fixed-pattern noise.
- 1. INTRODUCTION: A user-friendly integrated development environment enables the substrate to emulate different neural-network types without requiring prior neuromorphic circuit-design knowledge.The study showcases six networks, each with its own topology and neuronal and synaptic parameters.
2. THE NEUROMORPHIC SYSTEM
The Spikey system combines analog neuron and synapse circuits with digital spike communication, configurable parameters, and software-controlled experiment execution. Its hardware realizes conductance-based LIF dynamics while exposing flexible but bounded connectivity and calibration-sensitive analog behavior.
- 2. THE NEUROMORPHIC SYSTEM: Spikey implements neuron and synapse electrical behavior with analog VLSI circuits, while digital signaling transmits spikes between the chip and external components.The hardware operates approximately 10^4 times faster than biological counterparts by reducing electrical time constants.
- 2. THE NEUROMORPHIC SYSTEM: The chip implements a conductance-based leaky integrate-and-fire neuron model whose synaptic conductances drive membrane potential toward excitatory or inhibitory reversal potentials.Synaptic activation is represented by exponentially decaying conductance dynamics derived from current pulses.
- 2. THE NEUROMORPHIC SYSTEM: Experiments are configured, played back, triggered, and recorded through host software, FPGA logic, and network-module memory, with PyNN providing the user-facing modeling interface.The network module stores configuration and stimulus commands, triggers emulation, and records spike times in parallel.
- 2. THE NEUROMORPHIC SYSTEM: Spikey exposes many individually adjustable analog parameters and individually weighted synapses, supporting varied neuron types and connectivity patterns.Its synapse array supports 4-bit weights and a maximum fan-in of 256 synapses per neuron.
- 2. THE NEUROMORPHIC SYSTEM: Connectivity remains bounded: the current revision uses one synapse array and limits experiments to a maximum of 192 neurons, while all-to-all connectivity is unavailable.The maximum fan-in is 256 synapses per neuron, with different limits for on-chip and external sources.
- 2. THE NEUROMORPHIC SYSTEM: Production variability and electronic noise create neuron-to-neuron and trial-to-trial variability, while calibration is used to manage fixed-pattern differences.Some synaptic plasticity parameters also lack direct model equivalents and are translated using averages across circuits because of nonlinear saturation.
3. HARDWARE EMULATION OF NEURAL NETWORKS
The study evaluates six neural-network emulations on the Spikey hardware, comparing most results with software simulations to verify network functionality and performance. Calibration substantially narrows the membrane-time-constant distribution and moves its median toward the target.
- Six neural network models were emulated on the Spikey chip, with most results compared against software simulations for verification.
- After calibration, the membrane-time-constant distribution had a median of 11.2 ms and was significantly narrower.
- Two neurons were discarded because the automated calibration algorithm did not converge.
3.1. SYNFIRE CHAIN WITH FEEDFORWARD INHIBITION
The paper adapts a feedforward synfire-chain model with excitatory RS and inhibitory FS populations to test stable propagation and stimulus filtering on neuromorphic hardware. Hardware emulation reproduces persistent propagation and apparent filtering, while differing from software simulations because of reduced populations, fixed-pattern noise, and altered excitation–inhibition balance.
- Network Topology: The synfire chain contains groups of 100 excitatory RS and 25 inhibitory FS neurons, with RS projections to subsequent RS and FS populations and local FS-to-RS inhibition.
- Network Topology: Functionality was assessed by stable propagation of synchronous activity and responses to stimuli parameterized by amplitude a and temporal spread σ.
- Network Topology: Software simulations show a separatrix dividing stimuli that produce stable propagation from those whose activity becomes extinguished.
- Hardware Emulation: Hardware constraints required modifying the original model, including smaller groups, omitted background input, and approximately 1 ms fixed synaptic delays.
- Hardware Emulation: After parameter adjustments, persistent synfire propagation was observed for 2 h wall-clock time, corresponding to more than 2 years in biological real-time.
- Hardware Emulation: Hardware responses differed from software because reduced populations and fixed-pattern noise softened parameter-space boundaries and altered excitation–inhibition balance.
- Hardware Emulation: Hardware filtering remained apparent, but its quality could improve with the original group size, requiring a large-scale neuromorphic device.
- Hardware Emulation: The synfire chain could serve as an autonomous source of periodic input because it supports extremely long-lasting experiments without sustained external stimulation.
3.2. BALANCED RANDOM NETWORK
The balanced random network combines recurrent excitation and inhibition with Poisson drive to produce asynchronous irregular activity. Calibration and short-term depression enable the hardware emulation to reproduce software-like firing statistics despite hardware noise.
- Network topology: Balanced random networks use recurrently connected excitatory and inhibitory populations driven by separate Poisson-process populations through random sparse connections.Inhibitory synaptic weights are selected four times stronger than excitatory weights.
- Calibration: Calibration adjusts synaptic conductances, synaptic weights, and neuron excitability toward target firing behavior before hardware emulation.The procedure calibrates excitatory and inhibitory drivers and then scales afferent weights neuron by neuron.
- Hardware adaptation: Short-term depression is added to excitatory efferent connections to avoid self-reinforcement of network activity on hardware.The modified network nevertheless retains an asynchronous irregular state.
- Results: The calibrated chip produces irregular and asynchronous firing, with CV = 1.02 ± 0.16 and CC = 0.014 ± 0.019 in the recorded emulation.The corresponding software emulation reports CV = 0.96 ± 0.09 and CC = 0.010 ± 0.017.
- Hardware emulation: Hardware and software show comparable membrane dynamics and firing statistics across 125 neurons, although individual spike times differ because of hardware noise sources.This statistical agreement supports use of the emulation for models relying on asynchronous irregular activity.
3.3. SOFT WINNER-TAKE-ALL NETWORK
The soft winner-take-all network uses recurrent excitation and shared inhibition in a ring to compare competing inputs. Hardware reproduces suppression and amplification behavior, though its excitation profile is broader and inhibition less efficient than in software.
- Network architecture: The sWTA circuit consists of a ring of recurrently connected excitatory neurons and a common inhibitory pool receiving feedback from the excitatory layer.Recurrent excitation and external stimulation follow Gaussian strength profiles, with recurrent spread σrec = 5 neurons.
- Evaluation: The evaluation stimulates one side of the ring constantly and varies the firing rate delivered to the opposite side to measure competitive suppression.Hardware stimulation is distributed across multiple line drivers to equalize strength among neurons.
- Results: The reference-side firing rate decreases as opposite-side stimulation increases in both software and hardware, with average rates crossing at approximately r2 = 50 Hz.Hardware firing rates are less distinctive and higher overall than software rates.
- Hardware effects: Hardware neurons exhibit a broader and slightly asymmetric excitation profile, indicating less efficient inhibition than in software.The asymmetry is attributed to inhomogeneous excitability caused by fixed-pattern noise.
- Functionality: The stronger-stimulated side is amplified while the opposing side is suppressed, qualifying the hardware system for applications using similar sWTA topologies.
3.4. CORTICAL LAYER 2/3 ATTRACTOR MODEL
The cortical layer 2/3 attractor model organizes modular cortical populations into competing patterns with attractor dynamics, adaptation, and synaptic depression. A 192-neuron hardware implementation reproduces UP states and pattern completion, while fixed-pattern noise and hardware constraints reduce fidelity.
- Architecture: The model is modular, comprising cortical hypercolumns subdivided into minicolumns containing pyramidal, basket, and RSNP cell populations.
- Attractor dynamics: Within hypercolumns, basket cells mediate soft-WTA competition, while long-range RSNP inhibition governs competition among patterns.
- Pattern organization: A pattern contains one minicolumn from each hypercolumn, with within-pattern pyramidal connectivity reinforcing the corresponding attractor.Adaptation and short-term depressing pyramidal synapses weaken active patterns and facilitate switching.
- Scaling: Scaling the original 2673-neuron model to 192 neurons preserves the essential architecture by reducing pyramidal cells and consolidating basket-cell populations where connectivity is identical.
- Results: Hardware emulation shows UP states and pattern completion, including activation of a full attractor from stimulation of only a subset of its pyramidal cells.Short-term depression prevents the completed pattern from remaining active indefinitely.
- Limitations: The implementation showcases platform configurability and functionality, but hardware fixed-pattern noise and limited adaptation, parameter ranges, and configurability require additional tuning.These effects make hardware attractor dwell times less stable than in software.
3.5. INSECT ANTENNAL LOBE MODEL
The insect antennal lobe model uses lateral inhibition among glomeruli to decorrelate input channels. On Spikey, increasing inhibition drives output correlations toward zero, providing an olfaction-inspired preprocessing module for data analysis.
- Architecture: The network mirrors insect antennal-lobe connectivity, with receptor neurons driving projection neurons and local inhibitory neurons mediating lateral inhibition across glomeruli.
- Input and implementation: The experiment maps ten-dimensional firing-rate vectors onto ten glomeruli and presents each vector for one second using Gamma-process spike trains.Each glomerulus receives six receptor-neuron input streams projecting to seven projection neurons and three local inhibitory neurons.
- Evaluation: Channel correlation is assessed by comparing 10 × 10 input and output correlation matrices computed from receptor-neuron and projection-neuron rates.The matrices use Pearson correlation between glomerular rate vectors.
- Results: With increasing lateral inhibition, off-diagonal output correlations approach zero and output-channel correlation is virtually eliminated.Inhibition strength is scaled by q, with q = 0 representing none and q = 1 representing maximal strength.
- Application: The output’s residual correlation can be controlled by adjusting lateral-inhibition strength, enabling the network to function as a preprocessing module for Spikey-based data analysis.
3.6. LIQUID STATE MACHINE
The study evaluates a liquid state machine emulated on configurable neuromorphic hardware, using a recurrent liquid and an on-chip-trained spike-based tempotron to classify temporal spike patterns. Hardware and software implementations achieve comparable performance, retaining information over roughly 50–100 ms but losing classification ability for older inputs.
- Learning: On-chip iterative learning trains the tempotron readout and compensates for fixed-pattern noise in the hardware.The learning rule weakens causally contributing excitatory afferents and strengthens inhibitory ones after erroneous spikes, with the reverse adjustment when a required spike is absent.
- Architecture: The liquid state machine combines a recurrent liquid network with a spike-based tempotron classifier.The liquid projects continuous input streams into a high-dimensional space for readout, while the tempotron distinguishes two classes by emitting one or no spike.
- Network Topology: The liquid uses recurrent and feedforward excitatory and inhibitory populations, with an 80:20 excitatory-to-inhibitory neuron ratio.Each neuron receives four inputs from both the excitatory and inhibitory source populations, while other connection probabilities define the topology.
- Task: The task classifies 50 ms spike-train segments drawn from two templates with identical rates, X and Y, embedded in a continuous stream.Segments are randomly selected from either template to form the input sequence.
- Results: Around 90% classification correctness was achieved for segments 50–100 ms before stimulus end in both software and hardware implementations.For inputs farther back in time, performance dropped to chance level, 50% for the binary task, independently of simulation back-end.
- Scope: The implementation supports a broad variety of classification tasks, including work toward handwritten-digit recognition using the same Spikey setup.The tempotron can also function without a liquid as a sparse-response classifier robust to fixed-pattern noise.
4. DISCUSSION
The discussion presents the substrate as a flexible platform that reproduces diverse neural microcircuits and machine-learning functions with varied parameters and connectivity. It also identifies practical limits: model translation, fixed-pattern noise, hardware capacity, supported dynamics, and the scalability of calibration procedures.
- Overall Capability: A single universal substrate successfully implemented diverse neural microcircuits with activity patterns qualitatively and partly quantitatively similar to software simulations.Reference models were generally not significantly modified, and hardware and software used identical topologies when possible.
- Demonstrated Networks: The demonstrated networks span synfire chains, asynchronous irregular firing, soft winner-take-all circuits, cortical attractor memory, biological preprocessing, and machine learning.These examples cover signal propagation, stochastic cortical activity, attractor states, working memory, multivariate data preprocessing, and classification-related functions.
- Configurability: Different neuron and synapse parameters and vastly different connectivity patterns emphasize the substrate’s configurability across networks.Most networks used the full available neuron allocation receiving input from one synapse array on the Spikey chip.
- Translation and Noise: Model translation requires hardware-specific modifications, with fixed-pattern noise especially challenging after substantial downscaling.Noise is addressed through population averaging, on-chip weight training, and, for the cortical attractor memory, heuristic activity fitting.
- Usability: The integrated development environment is intended to make the system usable by neuroscientists without neuromorphic engineering backgrounds.The authors envision applications including accelerated simulation and studying robustness to parameter variability.
- Boundaries: The substrate cannot readily emulate models requiring neuronal adaptation or exotic synaptic plasticity, and neuron and synapse counts impose a hard network-size limit.A planned next generation is described as increasing available neurons and synapses by 10^3 while extending unit configurability.
- Scaling: Scaling calibration and fitting procedures to brain-sized emulations appears impractical when only a small subset of components can be individually calibrated.The discussion argues that simulation equivalence alone will not suffice as a quality criterion at such scales.