Source-linked AI summary
BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity
Amirhossein Tavanaei, Anthony S. Maida
TL;DR
The paper addresses how to train supervised multi-layer SNNs despite their non-differentiable spike activations and the biological and computational limitations of gradient descent. It proposes BP-STDP, a temporally local STDP/anti-STDP rule derived from backpropagation for integrate-and-fire neurons. Across XOR, Iris, and MNIST, the model achieves performance comparable to conventional backpropagation while retaining efficient, bio-inspired learning.
Problem
Supervised training of multi-layer SNNs is difficult because spike-based neurons lack differentiable continuous-valued activation functions, while gradient descent is computationally expensive and generally not biologically plausible.
Method
BP-STDP derives temporally local STDP and anti-STDP weight updates from backpropagation rules and applies them at each time step in multi-layer integrate-and-fire SNNs.
Results
On XOR, Iris, and MNIST, BP-STDP successfully performs supervised classification, with reported accuracies of 96.0% on Iris and 97.2% on MNIST comparable to conventional backpropagation and recent SNNs.
Takeaways & Limitations
BP-STDP combines accurate gradient-descent-derived updates with efficient, biologically plausible, temporally local STDP for supervised learning in multi-layer SNNs.
Takeaways & Limitations
The authors identify extending BP-STDP to deep SNNs with regularization modules and larger pattern-recognition tasks as future work.
Abstract
from arXiv · showhide
The problem of training spiking neural networks (SNNs) is a necessary precondition to understanding computations within the brain, a field still in its infancy. Previous work has shown that supervised learning in multi-layer SNNs enables bio-inspired networks to recognize patterns of stimuli through hierarchical feature acquisition. Although gradient descent has shown impressive performance in multi-layer (and deep) SNNs, it is generally not considered biologically plausible and is also computationally expensive. This paper proposes a novel supervised learning approach based on an event-based spike-timing-dependent plasticity (STDP) rule embedded in a network of integrate-and-fire (IF) neurons. The proposed temporally local learning rule follows the backpropagation weight change updates applied at each time step. This approach enjoys benefits of both accurate gradient descent and temporally local, efficient STDP. Thus, this method is able to address some open questions regarding accurate and efficient computations that occur in the brain. The experimental results on the XOR problem, the Iris data, and the MNIST dataset demonstrate that the proposed SNN performs as successfully as the traditional NNs. Our approach also compares favorably with the state-of-the-art multi-layer SNNs.
1. Introduction
SNNs offer biologically plausible, event-based computation, but supervised training of multi-layer networks remains difficult because spikes lack differentiable continuous activations. Prior methods trade accuracy against efficiency, motivating BP-STDP’s combination of backpropagation updates with temporally local STDP in integrate-and-fire networks.
- SNN neurons communicate through discrete spike events, creating a supervised-training challenge because they lack differentiable continuous-valued activation functions.
- Multi-layer supervised learning is crucial for demonstrating that SNNs can predict and classify stimulus patterns.
- SpikeProp and related methods apply gradient-descent-style learning to multi-layer SNNs by relating spike timing to membrane potential.
- Widrow-Hoff, perceptron, STDP, and anti-STDP approaches improve efficiency, but many are less accurate than gradient-descent methods or limited to single-layer supervision.
- BP-STDP trains multi-layer integrate-and-fire SNNs with supervised rules that combine efficient STDP and high-performance backpropagation.
2. Method
The method converts ReLU-based backpropagation into temporally local, event-based STDP rules for multi-layer SNNs of integrate-and-fire neurons.
- IF neuron and ReLU approximation: IF neurons approximate ReLU neurons because their accumulated membrane potential and postsynaptic spike count represent a scaled, threshold-shifted activation.The membrane potential integrates input spike trains, fires at threshold θ, and resets after firing.
- IF neuron and ReLU approximation: Input values are encoded as presynaptic spike trains over T ms, while output values are represented by postsynaptic spike counts.The spike count is proportional to the IF neuron's activity during the interval.
- Backpropagation conversion: The conventional gradient-descent weight update is re-expressed for SNNs by treating desired, output, and hidden activities as spike counts.This conversion relies on the approximation of spiking IF neurons to ReLU neurons.
- Temporally local learning: Because the interval-wide update is not temporally local, T is divided into sub-intervals containing zero or one spike.This decomposition produces updates that can be implemented with event-based STDP and anti-STDP.
- Output-layer learning: A teacher signal selects STDP for target neurons and anti-STDP for non-target neurons at desired spike times.Target spike trains use maximum frequency β, whereas non-target neurons are silent.
- Hidden-layer learning: For hidden layers, weight changes depend on presynaptic and postsynaptic spike timing, with hidden-neuron firing paralleling the ReLU derivative condition.The resulting multi-layer algorithm is named BP-STDP.
3. Results
BP-STDP was evaluated on XOR, Iris, and MNIST, where it learned selective output responses and achieved strong classification accuracy across SNN configurations.
- Experimental setup: BP-STDP was evaluated on XOR, Iris, and MNIST using multi-layer integrate-and-fire SNNs.The experiments varied network sizes and included 2-layer and 3-layer SNN evaluations.
- XOR problem: After around 150 training iterations, XOR output neurons became selective to the four input categories, with suitable learning rates in [0.01, 0.0005].The XOR labels were {0, 1, 1, 0}.
- Iris dataset: 96% Iris accuracy was achieved with BP-STDP versus 96.7% for a traditional backpropagation neural network.The result used 5-fold cross validation, and the model matched or outperformed the other listed multi-spiking supervised methods except Lin et al.’s computationally expensive gradient-descent method.
- MNIST dataset: On MNIST, a 1000-hidden-neuron SNN reached 90% accuracy after 100 epochs and 96% after 900 epochs.The target neuron reached threshold in under 9 ms, and the best accuracy rates were obtained with more than 500 hidden neurons.
- MNIST dataset: 2-layer and 3-layer MNIST SNNs achieved 96.6 ± 0.1% and 97.2 ± 0.07% accuracy, respectively.These results were reported as comparable to traditional backpropagation neural networks.
4. Discussion
BP-STDP translates backpropagation updates into biologically plausible, temporally local STDP-based learning for multi-layer SNNs. Experiments report supervised classification success while avoiding computationally expensive gradient descent.
- Learning rule: BP-STDP develops spatio-temporally local STDP and anti-STDP rules from backpropagation updates for multi-layer SNNs.The rules use error signals, presynaptic spike times, and postsynaptic activity to update weights temporally.
- Learning rule: The method combines biologically plausible, efficient STDP with the high performance of gradient-descent-based backpropagation.The approach is designed for spike-based communication while retaining backpropagation-derived weight changes.
- Experimental findings: 96.0% Iris and 97.2% MNIST classification performances were comparable to conventional backpropagation and recent SNNs.The reported results support supervised learning in multi-layer SNNs across these classification tasks.
- Experimental findings: Table 3 compares MNIST performance across conventional backpropagation, spiking gradient descent, and recent STDP-based SNNs.The table also distinguishes hidden-layer sizes and training-set distortion used for data augmentation.
- Significance: BP-STDP is presented as high-performance STDP-based supervised learning that avoids computationally expensive gradient descent.The paper positions this conversion from gradient-descent updates to spike-based STDP rules as computationally inexpensive.
5. Conclusion
BP-STDP combines backpropagation-derived updates with temporally local STDP in integrate-and-fire SNNs. It classifies XOR, Iris, and MNIST patterns successfully, while deeper networks and regularization remain future work.
- Contribution: BP-STDP combines bio-inspired STDP with backpropagation-derived learning in multi-layer integrate-and-fire SNNs.The method uses temporally local STDP and anti-STDP rules derived from traditional backpropagation updates.
- Results: The XOR experiments showed that BP-STDP can classify non-linearly separable patterns represented by spike trains.The conclusion also reports comparable classification performances on Iris and MNIST.
- Future work: Developing a deep SNN with BP-STDP and regularization modules is identified as future work.The paper suggests this direction for larger pattern-recognition tasks while preserving efficient, brain-like computations.