Source-linked AI summary
Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
Milad Mozafari, Mohammad Ganjtabesh, Abbas Nowzari-Dalini, Simon J. Thorpe, Timothée Masquelier
TL;DR
The paper addresses the energy and data demands of deep neural networks while seeking more biologically plausible spiking alternatives. It develops a latency-coded DCSNN combining STDP and reward-modulated STDP, achieving 97.2% accuracy on MNIST without an external classifier and extracting task-diagnostic features. The authors also report that multiple R-STDP layers help ignore frequent distractors, while decision efficiency and training stability remain bounded by design limitations.
Problem
Deep convolutional networks are less energy-efficient than biological systems and typically use data-hungry backpropagation, while spiking networks have not matched their recognition accuracy.
Method
The paper uses a latency-coded deep convolutional spiking network trained with STDP in lower layers and reward-modulated STDP in higher layers, with neuron-based decisions instead of an external classifier.
Results
97.2% recognition accuracy was achieved on MNIST without an external classifier, and multiple R-STDP layers helped ignore frequent distractors and extract diagnostic features.
Takeaways & Limitations
STDP extracted frequent intermediate features, whereas R-STDP guided higher layers toward task-specific diagnostic features and could reduce the training-data requirement to approximately 20%.
Takeaways & Limitations
Potential-based decision-making degrades computational efficiency, and simultaneously training multiple layers can destabilize learning when the ultimate layer changes as quickly as the penultimate layer.
Abstract
from arXiv · showhide
The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (DCSNN) and a latency-coding scheme. We trained it using a combination of spike-timing-dependent plasticity (STDP) for the lower layers and reward-modulated STDP (R-STDP) for the higher ones. In short, with R-STDP a correct (resp. incorrect) decision leads to STDP (resp. anti-STDP). This approach led to an accuracy of $97.2\%$ on MNIST, without requiring an external classifier. In addition, we demonstrated that R-STDP extracts features that are diagnostic for the task at hand, and discards the other ones, whereas STDP extracts any feature that repeats. Finally, our approach is biologically plausible, hardware friendly, and energy-efficient.
1 Introduction
The paper addresses the gap between accurate but energy- and data-intensive deep convolutional networks and more biologically plausible spiking networks. It proposes a three-layer DCSNN using STDP and R-STDP for classifier-free digit recognition, achieving strong MNIST performance and improved handling of distractors.
- Motivation: Deep convolutional networks achieve high recognition accuracy but are less energy-efficient than biological systems and typically rely on data-hungry backpropagation.Spiking neural networks are investigated as more brain-inspired alternatives, although they have not matched deep convolutional networks in recognition accuracy.
- Motivation: STDP-based networks can extract recurring visual features, but unsupervised STDP alone generally requires an external classifier or supervised plasticity for decision-making.Earlier MNIST systems using STDP-based learning reported accuracies from 91.6% to 96.5%.
- Approach: R-STDP extends STDP by incorporating reward or punishment feedback, encouraging or discouraging synaptic changes according to the network’s decision.The approach combines STDP in lower layers with R-STDP in higher layers.
- Approach: The proposed three-layer DCSNN uses DoG filtering, intensity-to-latency encoding, convolutional and pooling layers, and a final neuron-based decision layer without an external classifier.Decision neurons are pre-assigned to digits, and the earliest spike or maximum potential determines the output.
2 Methods
The method is a six-layer spiking convolutional architecture that converts images into latency-coded spikes, learns convolutional filters with STDP or R-STDP, and makes decisions through labeled global-pooling neurons. Reward or punishment feedback changes the plasticity polarity in R-STDP-enabled layers.
- Architecture: The DCSNN replaces an external SVM classifier with a neuron-based decision-making layer and applies R-STDP to higher layers.The architecture contains three convolutional layers and three corresponding pooling layers.
- Input encoding: MNIST images are filtered with six DoG filters and converted into spike times using intensity-to-latency encoding.The filters use on- and off-center responses at three scales; low filter outputs are ignored and remaining values are ordered for spike propagation.
- Network layers: Three spiking convolution-then-pooling stages process the input, with pooling used to introduce position invariance and reduce redundancy.Convolutional layers use integrate-and-fire neurons that emit at most one spike per input image, while pooling can represent earliest spike time or maximum potential.
- Decision-making: The final C3 layer globally pools S3 activity, assigns neurons to digit categories, and selects the neuron with the earliest spike or maximum potential.Each category may be assigned to multiple neurons, depending on the decision mechanism.
- Reinforcement signal: A correct decision generates reward and an incorrect decision generates punishment for layers using R-STDP.The reinforcement signal is generated after comparison with the input label and received globally by R-STDP-enabled layers.
- Plasticity: STDP changes synapses according to pre- and postsynaptic spike ordering, whereas R-STDP uses reinforcement signals to determine the polarity of weight changes.Reward sets α = 1 and β = 0, punishment sets α = 0 and β = 1, and a neutral signal sets both to zero.
3 Task 1: Solving 10-class Digit Recognition
The proposed three-layer DCSNN was evaluated on MNIST using STDP in early layers and R-STDP in the decision layer, achieving strong recognition without an external classifier. Its performance was also tested under reduced training data and alternative plasticity placement.
- The network was tested on 60,000 MNIST training images and 10,000 unseen test samples.
- C1 and C2 perform local pooling, while the decision selects the C3 neuron with maximum potential.
- 97.2% recognition performance was achieved with STDP in S1 and S2 and R-STDP governing S3.
- The proposed network ranked second among compared STDP-based MNIST SNNs without using an external classifier.
- Applying R-STDP to the penultimate S-layer did not improve performance on this digit-recognition task.
- With 20% of the training samples, the network regained almost all of its generalization power, unlike the compared supervised SNN.
4 Task 2: R-STDP Can Discard Non-Diagnostic Features
The handmade and MNIST tasks test whether R-STDP helps deeper spiking networks retain diagnostic features while discarding distractors. Across both tasks, R-STDP improves target-aware feature extraction, especially when neuronal resources are limited.
- 4.1 Handmade Problem: The handmade task uses three target bar combinations and a horizontal distractor, requiring S1 to learn all three diagnostic orientations.The three-class network is fully solvable only when its three S1 feature maps learn the target bars rather than the nondiagnostic horizontal bar.
- 4.1 Handmade Problem: R-STDP enabled the handmade network to solve the task without failure, whereas STDP usually failed to extract the correct orientations.Because all four bar orientations appeared equally often, STDP had only a 25% chance of discarding the nondiagnostic feature and retaining all targets.
- 4.2 Network Configuration: The MNIST task distinguishes two target digits from distractor digits using a smaller network and tests STDP versus R-STDP in the penultimate layer.S1 is trained with STDP, while S2 is trained with either STDP or R-STDP and S3 always uses R-STDP.
- 4.3 Training and Evaluation: R-STDP is more beneficial than STDP when S2 has fewer feature maps, while increasing feature-map capacity allows STDP to narrow the performance gap.R-STDP also makes S2 features target-aware, whereas STDP extracts the same features across digit pairs.
- 4.3 Training and Evaluation: The reported accuracies may be conservative because the experiments did not tune parameters for each digit pair and used limited iterations with faster learning rates.The authors state that these choices were made to keep the large number of simulations feasible.
5 Discussion
The discussion positions spiking networks and biologically inspired plasticity as hardware-friendly alternatives for online learning, while showing that R-STDP improves task-specific feature extraction under constrained resources. The proposed network achieves strong MNIST recognition without an external classifier, but some design choices trade computational efficiency for accuracy or stability.
- Motivation: SNNs offer spatio-temporal processing, energy efficiency, and hardware-friendly learning, making them suitable for online on-chip learning.Exact backpropagation is described as high-precision and time-consuming, whereas biologically inspired rules are more compatible with real-time hardware.
- Feature extraction: R-STDP is most beneficial with few S2 feature maps or frequent distractors, whereas increasing feature-map capacity lets STDP approach its performance.R-STDP extracts target-aware features, while STDP can fill the performance gap when more computational resources are available.
- Learning approach: R-STDP combines local plasticity with environmental feedback, allowing the network to learn task-relevant behavior through reward or punishment.The paper relates this mechanism to neuromodulation of synaptic plasticity and uses it in higher network layers.
- Recognition results: 97.2% recognition performance was achieved on MNIST using R-STDP in the last trainable layer and STDP in the first two layers.The network makes decisions with a neuron-based output layer rather than an external classifier.
- Feature extraction: Applying R-STDP to multiple layers helps ignore frequent distractors and extract diagnostic features using fewer computational resources than an STDP-enabled layer.This benefit is reported specifically when distractors are frequent and computational resources are limited.
- Trade-offs: Potential-based decision-making increases computation time, while simultaneous learning in multiple layers can destabilize training if upper-layer changes outpace lower-layer adaptation.The paper reports that waiting for the last spike solves false alarms from shared partial features, but increases computation time.
6 Conclusion
The paper concludes that a bio-realistic DCSNN can combine STDP for lower-level regularities with R-STDP for higher-level diagnostic features and make decisions without an external classifier. It reaches 97.2% on MNIST, uses substantially less training data than supervised learning, and remains open to improvements in decision-making and intermediate-layer training.
- Conclusion: STDP extracts frequent simple features in lower layers, while R-STDP guides higher layers toward complex diagnostic feature combinations.The resulting decision is made by neurons in the ultimate layer.
- Conclusion: 97.2% recognition performance was reached on MNIST with three trainable layers and at most one spike per neuron.The network ranked second among the cited STDP-based networks.
- Conclusion: Approximately 20% of the training samples were sufficient to reach the network’s ultimate performance, unlike the supervised counterpart.The paper also reports that multi-layer R-STDP avoids frequent non-diagnostic features and extracts task-specific ones.
- Limitations and future work: Potential-based decisions reduce computational efficiency, and future work includes inhibitory neurons, population-based decisions, and layer-wise R-STDP objectives.The proposed directions target false alarms, performance, and intermediate-layer learning.