Source-linked AI summary
A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
Yunzhe Hao, Xuhui Huang, Meng Dong, Bo Xu
TL;DR
SNN supervised training is challenging because spike activity is non-differentiable, while some alternatives rely on energy-inefficient communication. The paper proposes a biologically plausible SNN model combining symmetric STDP with dynamic-threshold intrinsic plasticity and reports good MNIST performance, including 81.65% with two hidden layers.
Problem
SNN supervised training is challenging because spike activity is non-differentiable, and communication among neurons can be energy inefficient.
Method
The model combines a biologically plausible symmetric STDP rule with intrinsic plasticity of the dynamic threshold.
Results
81.65% was achieved with two hidden layers, compared with 57.74% for one hidden layer, while the model also showed good MNIST performance.
Takeaways & Limitations
The proposed SNN model is presented as a biologically plausible neural network model for supervised learning.
Takeaways & Limitations
The model lacks the ability to process complex temporal information.
Abstract
from arXiv · showhide
Spiking neural networks (SNNs) possess energy-efficient potential due to event-based computation. However, supervised training of SNNs remains a challenge as spike activities are non-differentiable. Previous SNNs training methods can be generally categorized into two basic classes, i.e., backpropagation-like training methods and plasticity-based learning methods. The former methods are dependent on energy-inefficient real-valued computation and non-local transmission, as also required in artificial neural networks (ANNs), whereas the latter are either considered to be biologically implausible or exhibit poor performance. Hence, biologically plausible (bio-plausible) high-performance supervised learning (SL) methods for SNNs remain deficient. In this paper, we proposed a novel bio-plausible SNN model for SL based on the symmetric spike-timing dependent plasticity (sym-STDP) rule found in neuroscience. By combining the sym-STDP rule with bio-plausible synaptic scaling and intrinsic plasticity of the dynamic threshold, our SNN model implemented SL well and achieved good performance in the benchmark recognition task (MNIST dataset). To reveal the underlying mechanism of our SL model, we visualized both layer-based activities and synaptic weights using the t-distributed stochastic neighbor embedding (t-SNE) method after training and found that they were well clustered, thereby demonstrating excellent classification ability. Furthermore, to verify the robustness of our model, we trained it on another more realistic dataset (Fashion-MNIST), which also showed good performance. As the learning rules were bio-plausible and based purely on local spike events, our model could be easily applied to neuromorphic hardware for online training and may be helpful for understanding SL information processing at the synaptic level in biological neural systems.
1. Introduction
SNNs offer energy-efficient, event-based computation, but supervised training is difficult because spikes are non-differentiable and existing approaches compromise biological plausibility, efficiency, or performance. The paper proposes a dopamine-modulated symmetric STDP method with homeostatic mechanisms and reports effective recognition on MNIST and Fashion-MNIST.
- SNN training is challenging because spike-type activity is non-differentiable, motivating efficient supervised-learning algorithms.
- Backpropagation-based methods require real-valued computations and non-local communications, limiting their energy efficiency and biological plausibility.
- Plasticity-based approaches commonly use unsupervised clustering that requires an additional supervised recognition module, while existing supervised variants may be artificially modified or perform poorly.
- The paper introduces a bio-plausible supervised SNN method using dopamine-modulated symmetric STDP, combined with synaptic scaling and a dynamic threshold.
- The model achieved high performance on MNIST under two training methods and good classification performance on Fashion-MNIST, highlighting robustness.
2. Network architecture and neuronal dynamics
The model uses a three-layer feedforward SNN with LIF neurons, dynamic thresholds, lateral inhibition, and biologically motivated synaptic plasticity. Supervised signals train the output layer while local spike dynamics and homeostatic mechanisms shape responses.
- The architecture is a three-layer feedforward SNN comprising input, hidden, and supervised-learning layers.
- Supervised signaling: During supervised training, one of 10 output neurons receives a teacher-driven Poisson spike signal while the others remain at rest; during testing, all respond to hidden-layer inputs.
- Neuronal dynamics: LIF neurons use excitatory and inhibitory conductances to update membrane potential, reset after threshold crossing, and remain refractory for 2 ms.The membrane potential integrates leak, excitatory, and inhibitory terms before threshold-triggered reset.
- Neuronal dynamics: Dynamic thresholds implement intrinsic plasticity by increasing after spikes and decaying otherwise, helping prevent response dominance by individual neurons.A dynamical increment slows threshold growth as the threshold increases.
- Synaptic plasticity: Synaptic weights combine DA-STDP with synaptic scaling, whose normalization creates competition among incoming synapses because DA-STDP only increases synaptic strength.Scaling is applied after pattern training using a factor β ∈ (0, 1).
3. Recognition performance for the MNIST task
The MNIST experiments evaluated simultaneous and layer-by-layer supervised training across network sizes and compared them with Label Statistics. Accuracy converged quickly, the proposed methods generally outperformed the comparator, and the best reported accuracy was 96.73%.
- Experimental setup: The model was trained and evaluated on MNIST using 60,000 training samples and 10,000 test samples.
- Convergence: Classification accuracies converged quickly for networks with Nhidden = 400 and Nhidden = 6400 under both proposed training methods.
- Classification behavior: The test confusion matrix showed very high consistency between desired labels and inferred outputs in the supervised-learning layer.Darker pixels indicate stronger consistency between desired and inferred outputs.
- Performance comparison: The two proposed supervised-learning methods outperformed Label Statistics for all small-scale networks with Nhidden = 100, 400, and 1600.
- Performance comparison: Layer-by-layer training outperformed Label Statistics at all network scales, achieving the model’s best accuracy of 96.73% in the largest network.
4. Visualization of model clustering ability
The model’s layer activities become increasingly separable from input through hidden to supervised-learning layers. t-SNE visualizations also show that excitatory neurons’ output synaptic weights cluster consistently with their labels.
- Clustering visualization: t-SNE was used to reduce high-dimensional activity or synaptic-weight representations for clustering visualization.The method is widely used to visualize high-dimensional data in low-dimensional space.
- Layer-based activity visualization: t-SNE visualized original digit patterns, hidden-layer spike activities, and supervised-layer spike activities for all MNIST test samples.Each dot represents a digit sample and is colored by its corresponding label.
- Layer-based activity visualization: Separability of output information increased from the input layer to the supervised-learning layer.
- Clustering visualization: The supervised-learning layer served as a good classifier after training.
- Clustering visualization: Output synaptic weights of excitatory neurons clustered highly consistently with their labels and with supervised-layer spike activities.The hidden layer contained 6400 excitatory neurons in the referenced visualization.
5. Comparison with other SNN models
Existing supervised SNN methods often rely on backpropagation, external supervised modules, or specially modified plasticity rules, creating biological-plausibility or performance concerns. The proposed model replaces outside-network computations with spike-based processing and combines sym-STDP with additional plasticity mechanisms.
- Comparison with other SNN models: Backpropagation-based SNN training uses continuous variables or non-local computation, limiting biological plausibility and insight into spike-based learning.Examples approximate spike activities using membrane potentials, currents, or activity rates.
- Comparison with other SNN models: Several STDP-based models require external classifiers, artificial label statistics, or additional supervised modules for classification.These modules can leave the SNN without a direct relationship to supervised learning.
- Comparison with other SNN models: Some prior modified-STDP approaches reported relatively poor performance, including less than 90% accuracy.
- Comparison with other SNN models: The proposed model uses an additional spiking-neuron layer to reproduce algebraic computation previously performed outside the network.This enables completely spike-based computations in the comparison with Diehl and Cook’s model.
- Comparison with other SNN models: The model introduces a neuroscience-rooted sym-STDP rule and a dynamic threshold rule with a decay factor for α.The decay factor was reported to significantly improve performance.
- Comparison with other SNN models: 91.41% accuracy was achieved with Nhidden = 400 on the full MNIST dataset, compared with 88% on a smaller MNIST subset for another CSNN.The result indicates effective performance with a small network size.
6. Robustness of our SL model
The model was evaluated on Fashion-MNIST, a more realistic and challenging dataset than MNIST, using different network sizes and supervised-learning methods. It performed well across the tested settings, with its best result comparable to conventional machine-learning methods.
- Robustness evaluation: Fashion-MNIST was selected to test robustness because it is considered more realistic and more challenging than MNIST.Its images are generated from fashion-product thumbnails and retain MNIST’s image size and train/test structure.
- Robustness evaluation: Parameter adjustments included decreasing β to reduce weights and offset the impact of excessive spike quantity.
- Robustness evaluation: The model was trained with network sizes Nhidden = 400 and 6400 under the same evaluation criteria.
- Robustness evaluation: Layer-by-Layer training achieved 78.68% and 85.31% accuracy for network sizes 400 and 6400, respectively.
- Robustness evaluation: The Fashion-MNIST results were reported to further confirm the robustness of the supervised-learning model.
- Robustness evaluation: The best Fashion-MNIST performance was comparable with an SVM using a linear kernel at 83.9% and a multilayer perceptron with highest reported accuracy of 87.1%.
7. Discussion
The discussion attributes the model’s supervised-learning performance to the combined operation of three biologically plausible plasticity rules and examines how architecture and dopamine-related assumptions shape its scope. It also identifies limits in temporal processing and convolutional generality while emphasizing local spike-based learning.
- Learning mechanism: Sym-STDP, synaptic scaling, and dynamic threshold plasticity jointly connect inputs to supervised signals and support effective MNIST learning.Sym-STDP links spike events, synaptic scaling stabilizes weights and creates competition, and dynamic thresholds promote selective firing.
- Training procedure: The model achieved good performance while synchronously training two layers, unlike methods requiring layer-by-layer or multi-phase training.The discussion contrasts this training procedure with several previous SNN models.
- Network architecture: Global receptive fields let hidden neurons respond selectively to pattern categories, enabling clustering with one hidden layer and direct supervised classification.Sparse hidden activity also permits a simple linear mapping to the classification layer.
- Biological interpretation: Dopamine was identified as a key factor for supervised learning, but the proposed dopamine-involved sym-STDP mechanism remains an assumption lacking direct experimental evidence.The authors note that future evidence could challenge the biological validity of this assumption.
- Limitations: The model’s ability to process complex temporal information is limited, motivating future architectures that combine its advantages with stronger spatiotemporal processing.The authors specifically propose exploring convolutional SNNs to test broader applicability.