Source-linked AI summary
STDP-based spiking deep convolutional neural networks for object recognition
Saeed Reza Kheradpisheh, Mohammad Ganjtabesh, Simon J Thorpe, Timothée Masquelier
TL;DR
The paper asks whether deep SNNs can combine biologically grounded spike processing and unsupervised learning with robust object recognition. It proposes a temporal-coding SDNN with STDP-trained convolutional layers and pooling, achieving strong recognition results with sparse activity and few unlabeled examples. The findings support STDP with latency coding as a model for efficient visual learning within the tested scope.
Problem
DCNNs achieve strong recognition but use floating-point activations, supervised back-propagation, and large labeled datasets, unlike biological visual processing and primate learning.
Method
The paper proposes a deep SNN combining latency-based temporal coding, STDP-trained convolutional layers, pooling, and competitive learning.
Results
The SDNN outperformed the compared unsupervised methods and reached 99.1% on Caltech face/motorbike, 82.8% on ETH-80, and 98.4% on MNIST.
Takeaways & Limitations
The results suggest that combining STDP with latency coding can produce sparse, efficient, discriminative object representations from few unlabeled examples.
Takeaways & Limitations
DoG filtering loses some texture and color information, motivating future input-layer encodings for these additional visual signals.
Abstract
from arXiv · showhide
Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and pooling layers. We used a temporal coding scheme where the most strongly activated neurons fire first, and less activated neurons fire later or not at all. The network was exposed to natural images. Thanks to STDP, neurons progressively learned features corresponding to prototypical patterns that were both salient and frequent. Only a few tens of examples per category were required and no label was needed. After learning, the complexity of the extracted features increased along the hierarchy, from edge detectors in the first layer to object prototypes in the last layer. Coding was very sparse, with only a few thousands spikes per image, and in some cases the object category could be reasonably well inferred from the activity of a single higher-order neuron. More generally, the activity of a few hundreds of such neurons contained robust category information, as demonstrated using a classifier on Caltech 101, ETH-80, and MNIST databases. We also demonstrate the superiority of STDP over other unsupervised techniques such as random crops (HMAX) or auto-encoders. Taken together, our results suggest that the combination of STDP with latency coding may be a key to understanding the way that the primate visual system learns, its remarkable processing speed and its low energy consumption.
Introduction
The paper addresses biological limitations of deep convolutional networks by combining hierarchical spiking computation with unsupervised STDP learning. It proposes and evaluates a deep spiking network designed to learn robust object representations from few, unlabeled examples.
- Motivation: DCNNs use biologically implausible floating-point activations and supervised back-propagation, despite their hierarchical visual-processing architecture.Biological neurons communicate with spikes, whose timing can encode information, whereas DCNNs neglect cortical processing and learning mechanisms.
- Motivation: Millions of labeled examples may be needed to train deep networks, whereas primates can learn from far fewer examples, often without labels.The paper identifies unsupervised STDP as a possible mechanism supporting this difference.
- Related work: Existing spiking networks either used shallow architectures or converted rate-based DCNNs, inheriting many-spike and long-processing-time drawbacks from spike-rate coding.The proposed direction targets deeper trainable SNNs with temporal rather than rate coding.
- Contribution: The proposed SDNN combines a temporal-coding input layer with consecutive STDP-trained convolutional layers and pooling layers that compress visual information.The network encodes images as asynchronous spike trains and uses pooling for translation invariance.
- Contribution: Features become more complex through the hierarchy, progressing from simple edge features to object prototypes while producing discriminative object representations.The network was evaluated on several image datasets and achieved high recognition accuracies.
- Results: 99.1% accuracy was reached on the Caltech face/motorbike task, 82.8% on ETH-80, and 98.4% on MNIST.The authors also report low spike counts and suitability for neuromorphic hardware implementation.
Proposed Spiking Deep Neural Network
The SDNN uses rank-order temporal coding, hierarchical convolution and pooling, and competitive STDP to learn increasingly complex visual features. Its sparse event processing combines feature selectivity, translation invariance, and unsupervised layer-wise learning.
- Architecture: The sample architecture contains three convolutional layers and three pooling layers, ending with a classifier operating on globally pooled features.Convolutional layers learn feature combinations, while pooling uses local maxima to compress information and provide translation invariance.
- DoG and temporal coding: DoG filters encode contrast strength as spike latency, with stronger contrasts producing earlier spikes under rank-order coding.Separate ON- and OFF-center maps respond to positive and negative contrasts.
- Convolutional layers: Convolutional neurons integrate presynaptic spikes and emit when their membrane potential reaches threshold, then reset after firing.The neurons are nonleaky integrate-and-fire units whose synaptic weights determine feature selectivity.
- Convolutional layers: Lateral inhibition and one-spike-per-neuron constraints create sparse coding in which each location can signal one selected visual feature.When a neuron fires, competing maps at that location are inhibited until the next image.
- Pooling layers: Pooling performs a nonlinear maximum over neighboring neurons and propagates the first arriving spike, providing invariance while compressing visual data.Pooling neurons do not learn; their unit weights and thresholds make the first input spike trigger the output.
- STDP-based learning: STDP learning is restricted to convolutional layers and proceeds layer by layer, with weight changes determined by the temporal order of pre- and postsynaptic spikes.The multiplicative weight term keeps synaptic weights within [0,1].
- STDP-based learning: Within-map competition selects the first neuron to fire for learning, while local inter-map competition encourages different maps to learn different features.The winning neuron's updated weights are copied across locations in its map.
Results
Across the hierarchy, STDP learned increasingly complex and category-specific features, from oriented edges to object prototypes. The network achieved strong recognition with few training examples and limited labeled data, while learned features outperformed random alternatives and tolerated moderate noise.
- Hierarchical feature learning: STDP progressively learned oriented edges, intermediate object features, and whole-object prototypes across the convolutional hierarchy.Learning competition favored salient, common, and diverse features while discouraging redundant maps and background features.
- Hierarchical feature learning: Higher-layer neurons became category-specific: face or motorbike inputs activated corresponding prototype maps, and complementary neurons formed robust object representations.Some individual global-pooling neurons reached 93% accuracy, while their mean accuracy was 89.8%.
- Ablations and robustness: Replacing learned features with random ones in lower convolutional layers reduced recognition accuracy, showing that learned hierarchical features contributed to performance.The experiment evaluated random features in the third layer, the second and third layers, or all three convolutional layers.
- Recognition performance: 78.2% accuracy was reached with 5 training images per category, and 95.1% with 40 images per category.Increasing the number of training samples improved recognition, but reasonable accuracy was obtained with only a few tens of images.
- Recognition performance: 93.8% average accuracy was achieved with only one labeled sample per category after STDP learned features from 200 images per category.The result indicates that unsupervised feature learning reduced the number of labeled examples needed by the classifier.
Discussion
The proposed SDNN combines deep hierarchical processing with spike-time coding and unsupervised STDP, targeting biologically grounded recognition with sparse activity. It outperformed the cited unsupervised methods and addressed limitations of supervised deep networks on smaller datasets, while remaining compatible with parallel hardware and future event-based inputs.
- Architecture and learning: The SDNN uses spike-time coding with STDP-based learning across multiple convolutional layers, rather than rate coding or conventional supervised back-propagation.Its temporal coding represents visual information in first-spike timing, while STDP learns features in successive trainable layers.
- Biological and computational advantages: The SDNN was designed as a multi-layer learnable spiking network for large-scale natural object images, unlike prior converted DCNNs that rely on rate coding.Rate-coded conversions require many spikes and long processing times, while this network uses temporal coding and STDP.
- Comparison with other methods: The proposed SDNN outperformed the cited unsupervised methods and was better than the supervised deep network on the evaluated smaller datasets.The supervised deep network was reported to suffer from overfitting and insufficient training data, whereas state-of-the-art supervised DCNNs performed strongly on large datasets such as Imagenet.
- Implementation: The SDNN can be efficiently implemented on parallel hardware using address event representation, potentially allowing simulations much faster than biological real time.AER represents spike events through sending and receiving neuron addresses, with time encoded by asynchronous event occurrence.
- Limitations and extensions: The input layer can be replaced by spiking retinal models or dynamic vision sensors, but DoG filtering currently loses texture and color information.Future work should encode these additional visual features in the input layer.
Supporting Information
The supplementary video visualizes learning on the Caltech face-and-motorbike task, from input presentation through propagated spike waves and STDP updates. It also displays the evolving neural activity at successive processing stages.
- Learning visualization: The video presents face and motorbike training examples, propagates their spike waves, and applies the STDP learning rule.The input image appears in the top-left panel.
- Input coding: The top-middle panel shows output spikes from the input DoG layer at each time step.This panel visualizes the temporal spike representation generated from the input image.
- Task: The video follows learning progress and neural activity over the Caltech face-and-motorbike task.