Biologically Inspired Spiking Neurons : Piecewise Linear Models and Digital Implementation

Hamid Soleimani; Arash Ahmadi; Mohammad Bavandpour

Source-linked AI summary

Biologically Inspired Spiking Neurons : Piecewise Linear Models and Digital Implementation

Hamid Soleimani, Arash Ahmadi, Mohammad Bavandpour

arXiv:1212.3765v1Dec 16, 2012cs.LGcs.NEq-bio.NC

TL;DR

Large-scale spiking-neuron hardware must balance biological behavior with implementation cost, especially when FPGA multipliers limit network size. This paper develops multiplier-less piecewise-linear modifications of the Izhikevich model and evaluates their dynamics, networks, and learning applications. The models retain similar neuron behavior while reducing hardware cost and increasing implementation speed, with a character-recognition case study reaching 91.7% accuracy.

Problem
Large-scale spiking-neuron implementations face computational complexity and FPGA multiplier limitations that constrain hardware cost and network size.
Method
The paper develops multiplier-less piecewise-linear modifications of the Izhikevich model and evaluates them in neuron dynamics, networks, FPGA hardware, and rate-based character recognition.
Results
The PWL models show similar dynamical behavior with lower hardware cost, while reaching approximately 200 MHz and 91.7% recognition accuracy in the reported implementations.
Takeaways & Limitations
Simple arithmetic PWL neuron models can support high-speed, lower-area FPGA networks without embedded multipliers while retaining biologically inspired responses.

Abstract

from arXiv · show

There has been a strong push recently to examine biological scale simulations of neuromorphic algorithms to achieve stronger inference capabilities. This paper presents a set of piecewise linear spiking neuron models, which can reproduce different behaviors, similar to the biological neuron, both for a single neuron as well as a network of neurons. The proposed models are investigated, in terms of digital implementation feasibility and costs, targeting large scale hardware implementation. Hardware synthesis and physical implementations on FPGA show that the proposed models can produce precise neural behaviors with higher performance and considerably lower implementation costs compared with the original model. Accordingly, a compact structure of the models which can be trained with supervised and unsupervised learning algorithms has been developed. Using this structure and based on a spike rate coding, a character recognition case study has been implemented and tested.

I. INTRODUCTION

Spiking-neuron models balance biological fidelity against computational complexity, motivating piecewise-linear alternatives for large-scale FPGA implementations. The paper replaces costly operations in the Izhikevich model and evaluates the resulting models in neuron dynamics, networks, hardware, and character recognition.

Motivation: Detailed neuron models can reproduce biological measurements accurately but are computationally complex, while spike-based models target temporal behavior with lower complexity.The paper frames neuron-model selection as a tradeoff between accuracy and computational cost.
Hardware Motivation: FPGA implementations of large networks are constrained by the limited number of fast multipliers, which consume substantial area, latency, and power.Earlier FPGA arrays were limited by the number of embedded multipliers available on the target chips.
Contribution: The proposed multiplier-less PWL models replace the Izhikevich square operation with comparison or absolute-value operations, simplifying multiplication into addition and logic shifts.The stated goal is lower hardware cost while retaining similar dynamic behavior.
Paper Scope: The paper develops FPGA-oriented neuron models and a rate-based training approach, then applies them to a pattern-recognition network and character-recognition case study.The reported workflow covers model analysis, hardware design, training, and recognition experiments.

II. BACKGROUND

The Izhikevich model uses coupled membrane and recovery dynamics with reset rules, but its quadratic voltage term complicates circuit implementation. The paper therefore introduces piecewise-linear approximations to improve computational efficiency.

Original Model: The Izhikevich neuron uses membrane potential v and recovery variable u, with reset equations applied after the voltage reaches its spike threshold.The recovery variable represents ionic-current effects and provides negative feedback to v.
Implementation Challenge: The quadratic term in the Izhikevich dynamics creates a parabolic characteristic that makes digital and analog circuit realization difficult.This quadratic component is identified as the main implementation challenge in the background discussion.
Proposed Direction: Three piecewise-linear approximations are proposed to improve the computational efficiency of the Izhikevich model.The subsequent models approximate the nonlinear quadratic component with simpler piecewise functions.
Second-Order Model: The second-order PWL model approximates the quadratic component using two crossed lines, providing two degrees of freedom for fitting the original behavior.Its formulation is presented as the first of the proposed piecewise approximations.

B. Third order piecewise linear model

The third-order PWL model introduces a nonlinear piecewise approximation with three degrees of freedom. It costs more to implement than the second-order model but can more closely match the original behavior through coefficient selection.

Model Formulation: The 3PWL model uses a third-order piecewise function with three degrees of freedom to approximate the original nonlinear behavior.The approximation is depicted as the third-order model in Fig. 1.
Tradeoff: Compared with 2PWL, 3PWL is more expensive to implement but can achieve behavior closer to the original model with appropriate coefficients.The comparison presents a direct accuracy–implementation-cost tradeoff between the two approximations.

C. Forth order piecewise linear model

The fourth-order PWL model adds a piecewise approximation controlled by constant coefficients. It has more complex circuitry than the other PWL models but exhibits behavior very close to them.

Model Formulation: The 4PWL approximation uses constant coefficients k1, k2, and k3 and provides three degrees of freedom for fitting the nonlinear behavior.Its equilibrium locus is presented as the fourth PWL model in Fig. 1.
Implementation Tradeoff: The 4PWL model requires more complex circuit implementation than the other PWL models but has behavior very close to the other proposed models.Coefficient values are selected according to model accuracy and implementation simplicity.

IV. FINDING K COEFFICIENTS FOR THE PWL MODELS

The paper selects PWL coefficients by balancing neuron-dynamics accuracy against fixed-point implementation constraints. Error measures and a search procedure identify stable low-error coefficient regions for each model.

Dynamic criteria: The evaluation examines single-neuron dynamics because network behavior depends on both the network structure and the accuracy of the individual neuron model.The analysis focuses on equilibrium loci, refractory response, return-path shape, and firing-rate effects.
Error assessment: The coefficient search compares PWL and original neuron behavior using slope and peak-position errors that capture refractory and excitation differences.ERRS measures slope mismatch along the excitation path, while ERRP measures displacement at the curve minimum and its effect on required input stimulus.
Coefficient search: The search algorithm evaluates feasible k-coefficient combinations through nested loops while removing unstable or noisy signal portions before comparison.Each candidate combination resets the membrane potential and recovery variable, and the procedure excludes unstable transients from sampling.
Hardware-aware selection: The optimized coefficients are not unique, so hardware simplicity further narrows the candidates after error minimization.The color plots visualize cost-function values for 2PWL tonic-spiking and tonic-bursting cases across coefficient ranges.
Hardware-aware selection: Coefficient selection targets a low-error stable zone rather than a single minimum because fixed-point precision and digital simplifications can amplify local error variation.The target area combines consistently low cost-function error with coefficients suitable for digital arithmetic.

V. VARIOUS NEURON-LIKE RESPONSES

The PWL models reproduce transitions among resting, tonic bursting, and tonic spiking as input current increases. Across ten tested response types, the 4PWL model most closely matches the original Izhikevich model.

Transition behavior: The PWL models reproduce the resting ↔ tonic bursting ↔ tonic spiking transition as input current increases, similarly to the original model.The transition is attributed to bifurcation phenomena and is evaluated using time waveforms.
Response comparison: Across ten neuron-response types, the 4PWL model has the closest behavior to the original Izhikevich model.The comparisons include spiking, bursting, mixed-mode, adaptation, excitability, latency, and sub-threshold oscillations.

VI. NETWORK BEHAVIOR

A randomly connected 2000-neuron network produces similar overall activity for the original and PWL models. The models share a 5 Hz rhythm, while precise spike timing differs across variants.

Network dynamics: The 2000-neuron simulations show similar network activity structure across the original and PWL models, with all producing the same 5 Hz rhythm.The network uses a 4:1 excitatory-to-inhibitory ratio, stronger inhibitory synapses, and noisy thalamic input.
Spike-timing comparison: The 4PWL model has the best mean relative-error performance, followed by the 3PWL and 2PWL models.Mean relative error is computed from spike-by-spike timing differences over the randomly connected 2000-neuron network.
Spike-timing comparison: Differences from the original model are linked mainly to ERRP, especially in the 2PWL model, where excitation shifts can alter firing-rate frequency.The authors state that more accurate regulation of the initial input I(t) would improve the proposed models’ network results.

VII. DESIGN AND HARDWARE IMPLEMENTATION

The hardware architecture implements the PWL neuron, synaptic weights, and training mechanism as coordinated sub-blocks. It supports full-shape signaling for evaluation and can be reduced for spike-timing communication.

Implementation modes: The implementation uses full-shape neuron signaling to evaluate model accuracy, while lower-cost versions can target spike timing over AER communication.AER-based implementations do not require signal shaping.
Learning support: The architecture supports both supervised and unsupervised training implementations.The training mechanism is presented as part of the generalized neural-network structure.
Architecture: The hardware structure contains neuron-model, synaptic-weight, and training sub-blocks that combine weighted inputs into each neuron’s total current.Weights are updated by the learning mechanism, while output firing time is determined from the neuron input.

1. W Unit

The W and N units organize synaptic weights, input-current computation, and pipelined neuron-state updates for digital PWL neuron implementation. Their arithmetic pipelines replace costly operations with hardware-suitable stages while preserving synchronization across neurons.

W Unit: The W Unit stores synaptic weights and computes each neuron's input current from weighted input values.It supports weight changes during training and can accommodate more sophisticated synaptic dynamics such as STDP.
Weights Bank: The Weights Bank uses M buffers of N digital weights, shifting each buffer's values and applying training updates before reinsertion.Each weight is represented with a configurable wb-bit fixed-point value.
Input computation unit: Input computation conditionally passes or complements weights according to binary input neurons, pipelines their accumulation, and adds i_bias to produce I_in.The resulting input current determines the neuron's spike rate, with I_S computation stages and D_S synchronization delays.
N Unit: The N Unit combines V_pipeline, U_pipeline, V_buffer, and U_buffer blocks to update fixed-point neuron states and send membrane potential to threshold control.V and U use configurable vb-bit and ub-bit representations, while the control unit generates firing signals when the threshold is reached.
Arithmetic pipelines: Replacing multiplication with shift-and-add operations reduces digital cost, and PWL models are expected to run at higher frequency than the original model.Pipeline and delay-stage choices are used to satisfy neuron-count and synchronization requirements.

3. C Unit

The C Unit coordinates training, threshold timing, neuron selection, and output reporting through counter storage, control logic, and an output provider.

Counter buffer: The counter buffer stores each neuron's spike timing in clock pulses, resetting on firing and supplying timing values for weight-update evaluation.It stores N counter values and shifts them each clock pulse.
Control Unit: The control unit receives neuron outputs, counters, targets, validity signals, and training inputs, then issues reset, update, output, and weight-change commands.The weight-change value is calculated by the training mechanism according to the selected learning algorithm.
Control and training interface: The C Unit's design supports supervised and unsupervised training inputs while synchronizing target firing times and selected-neuron outputs.Its interfaces include target values, valid signals, input neurons, and a neuron-selection code.
Output provider: The output provider uses an encoder and shift register to select a neuron's output and updates its register when that neuron's membrane potential reaches the N-unit output.The selected output register updates once every N clock pulses.

VIII. CHARACTER RECOGNITION AND TRAINING ALGORITHM

The paper evaluates a two-layer spiking network for handwritten character recognition using rate-coded inputs and a supervised weight-adjustment algorithm. Output neurons learn target spike frequencies through an error defined over target and observed firing periods.

Network and coding: A two-layer spiking network uses rate-based coding, with each character pixel mapped to an input neuron and tonic-spiking output neurons used for recognition.Numerical values use a 20-bit fixed-point 8.12 representation.
Training objective: The training error is the squared difference between each output neuron's target spike period and its observed counter period.The counter represents the output neuron's firing period, while the target specifies the desired frequency period.
Training rule: Because tonic-spiking neurons have an inverse linear relation between input current and firing-period counter, gradient-based weight updates can reduce spike-rate error.The update derivation uses the counter-current relation and a positive normalization coefficient.
Teacher and student neurons: The case study distinguishes valid and invalid responses using high and low output frequencies, respectively, with 80 Hz and 10 Hz used as the two frequency levels.The network trains student neurons to trace the teacher neuron's spiking frequency across Izhikevich and 2PWL, 3PWL, and 4PWL models.
Supervised training: The supervised algorithm presents valid target values through a target register and applies training control signals during the selected neuron's firing process.The target register shifts logically after the valid input is reset.

IX. IMPLEMENTATION RESULTS

FPGA experiments show that the PWL neurons reproduce the tested dynamical behaviors with substantially better implementation efficiency than the original Izhikevich model. The hardware network achieves 91.7% handwritten alpha-digit recognition accuracy.

Dynamical behavior: Oscilloscope measurements on the FPGA compare tonic-spiking and tonic-bursting outputs from Izhikevich, 2PWL, 3PWL, and 4PWL implementations.The physical measurements were performed on a XILINX Virtex-II Pro XC2VP30 platform.
Speed comparison: The implemented PWL models were approximately 9.2 times faster than Izhikevich's model with a simple combinational multiplier.The original model's v^2 term creates a longer critical path and requires high-performance multipliers.
Resource trade-offs: A fully pipelined multiplier brings Izhikevich frequency close to the PWL models but consumes considerably more area and resources.A 20-bit Booth multiplier reduces area and resources but makes the overall system 11 times slower than the FPGA clock.
Character recognition: 91.7% recognition accuracy was obtained on hardware for MNIST-style handwritten alpha-digits, versus 95% reported for a traditional back-propagation network with 300 hidden neurons.Training used 29 handwritings per character repeated 25 times, while testing used 10 remaining handwritings per character.
Hardware performance: Approximately 200 MHz was reached by the PWL models on a Virtex-II Pro, nearly 9.2 times the original Izhikevich model's rate.The paper reports comparable area or a 2-4× area improvement over a fully pipelined multiplier model at comparable frequency.