First-spike based visual categorization using reward-modulated STDP

• Unsupervised STDP strengthens synapses according to spike timing and extracts frequent input patterns, but may miss rare diagnostic features needed for decision-making.STDP-enabled visual networks also require an external readout for classification.Source: §1 · p. 2
• R-STDP modulates or reverses STDP weight changes using reward or punishment signals, linking synaptic plasticity to reinforcement learning.The rule stores eligibility traces and applies modulated weight changes when the signal arrives.Source: §1 · p. 2
• The four-layer network classifies from the earliest spikes of class-specific neurons, applies reward or punishment based on correctness, and uses no external classifier.Only the third layer is trainable, and each neuron can spike at most once per image for computational efficiency and biological realism.Source: §1 · p. 3
• Toy experiments showed that R-STDP found informative features with fewer computational resources than STDP and could encourage a neuron to unlearn previously learned behavior.Source: §1 · p. 3
• R-STDP outperformed STDP in task-specific feature discovery, reaching 98.9% accuracy on Caltech face/motorbike, 89.5% on ETH-80, and 88.4% on NORB.Source: §1 · p. 3

2.1 Overall Structure

• The network alternates two simple and two complex layers in a feedforward structure, following Masquelier and Thorpe.Figure 1 depicts the four retinotopically organized layers.Source: §2.1 · p. 3, Figure 1 · p. 4
• C1 applies local pooling for some position invariance and propagates the earliest spike in each input window.The figure describes this as local max-pooling.Source: §2.1 · p. 3, Figure 1 · p. 4
• S2 uses integrate-and-fire neurons and a plasticity rule combining pre- and postsynaptic spike times with reward or punishment.The rule controls neuron selectivity to input patterns.Source: §2.1 · p. 4
• C2 assigns neurons to categories and decides by the first spike, then generates reward or punishment signals that drive S2 plasticity.C2 globally pools S2 neurons within a particular grid.Source: §2.1 · p. 4

2.2 Layer S1

• S1 convolves grayscale input with four orientation-specific Gabor filters to produce feature maps representing oriented-edge saliency.The filters use a 5 × 5 window, wavelength 2.5, and effective width 2.Source: §2.2 · p. 4
• S1 uses absolute convolution to introduce invariance to image negatives and applies a π/8 offset to relax bias toward common vertical and horizontal edges.Source: §2.2 · p. 4
• S1 converts edge saliency into spike latency, so more salient edges propagate earlier, and spikes are processed sequentially in ascending latency order.Source: §2.2 · p. 4

Layer C1

• Each C1 neuron pools over a local window in one orientation grid and spikes immediately after its earliest input, reducing S1 redundancy and neuron count while adding local edge-position invariance.The pooling stride is set to the window size minus one.Source: §2.3 · p. 5
• When a neuron fires, neurons at the same position in other orientation grids are prevented from firing.Source: §2.3 · p. 5
• Firing also increases nearby neurons’ latencies according to distance, using inhibition factors of 15%, 12%, 10%, 7%, and 5% for distances 1–5 pixels, respectively.Floating-point distances are truncated to integers.Source: §2.3 · p. 5

2.4 Layer S2

• S2 has n two-dimensional IF-neuron grids, each receiving plastic inputs from a ωs2 × ωs2 × 4 C1 window, with shared weights enabling feature detection across positions.Source: §2.4 · p. 5
• An S2 neuron fires when its membrane potential reaches threshold T; it has no leakage and may fire at most once per image.Source: §2.4 · p. 5
• S2 synapses update from pre- and postsynaptic spike order under the next layer’s reward/punishment signal, with initial weights drawn at mean 0.8 and standard deviation 0.05.Small or midrange means can leave neurons inactive, while large variance increases the impact of the network’s initial state; a high mean and small variance are suitable.Source: §2.4 · p. 5

2.5 Layer C2

• Each C2 neuron propagates only the first spike received from its corresponding S2 grid.There are exactly n C2 neurons, one assigned to each S2 grid.Source: §2.5 · p. 5
• C2 neurons are grouped by stimulus category, and the network predicts the category whose group propagates the earliest spike.Source: §2.5 · p. 6
• A correct C2 decision triggers reward and an incorrect one punishment; if none fire, weights remain unchanged, while tied earliest spikes go to the lowest-index neuron.Source: §2.5 · p. 6

2.6 Reward-Modulated STDP (R-STDP)

• Weight-change magnitude is modulated by a reward or punishment signal determined by the correctness of the network’s decision.The mechanism updates pre-synaptic weights of S2 neurons.Source: §2.6 · p. 6
• Only the earliest-spiking S2 neuron wins the one-winner-takes-all competition, updates its synaptic weights, and determines the network’s decision.Source: §2.6 · p. 6
• Punishment reverses STDP polarity, swapping long-term depression with potentiation; the rule ignores exact spike-time differences and uses an infinite time window.Source: §2.6 · p. 6

2.7.1 Adaptive learning rate

• Overfitting can arise when punishment or reward dominates, producing dead or overly selective neurons that miss inputs or exclude misclassified samples.The method balances the impact of correct and incorrect samples across training iterations; samples with no active S2 neuron may remain unclassified.Source: §2.7.1 · p. 7

2.7.2 Dropout

• Dropout temporarily disables C2 neurons with probability pdrop, increasing neuron involvement and the chance of finding discriminative features while reducing blind firings.The paper motivates dropout by noting that training-only reward maximization can leave neurons untrained, lowering test hit rates when they fire indiscriminately.Source: §2.7.2 · p. 7

2.8 Classification

• The earliest spike in C2 supplies the network’s final classification decision, eliminating the need for an external classifier.The network uses only last-layer activity, particularly the earliest C2 spike, to decide on an input stimulus.Source: §2.8 · p. 7
• For m categories, the network uses n = k × m neuronal grids in S2 and assigns each C2 neuron to a category.Here, k is the number of features associated with each category.Source: §2.8 · p. 7
• During training, the network compares its classification decision with the stimulus label and generates reward for a match or punishment for a mismatch.The network classifies input stimuli using equation (5).Source: §2.8 · p. 7

2.9 Comparison of R-STDP and STDP

• STDP is trained unsupervised to extract S2 features for comparison with the R-STDP model.Feature vectors are extracted for both training and testing sets.Source: §2.9 · p. 7
• The three S2 feature encodings are a first-spike vector, a spike-count vector, and a potential vector.The first-spike vector marks only the earliest-spiking neuronal grid; the spike-count vector records spikes per grid, while the potential vector records each grid’s maximum membrane potential without applying the threshold.Source: §2.9 · pp. 7–8
• KNN and SVM evaluate performance, with the same learning strategy and STDP formula and parameter values used in both models.Only the STDP LTP and LTD magnitudes are explored.Source: §2.9 · p. 8

3.1 R-STDP Increases Computational Efficiency

• On two spatially similar patterns distinguished by temporal order, R-STDP learned discriminative features in 98% of 100 runs; STDP did not, unless larger receptive fields and higher thresholds enabled success in 80%.With the standard STDP setup, neurons selected the shared early pattern, so the group classified both stimuli alike; the alternative STDP configuration used more computational resources.Source: §3.1 · pp. 8–9, Figure 2 · p. 8

3.2 Plastic Neurons

• After target stimuli were swapped at iteration 200, punishment reversed synaptic strengthening and weakening; neurons found the new target after iteration 300, whereas STDP could not adapt to the change.During roughly 80 iterations of punishment, previously weakened synapses strengthened and previously strengthened ones weakened, reducing old sensitivity and allowing new learning.Source: §3.2 · p. 9

3.3.1 Datasets

• The experiments assessed object recognition on Caltech face/motorbike, ETH-80, and small NORB, with the latter two presenting objects from different viewpoints and posing harder tasks.Caltech face/motorbike was described as the easiest benchmark and used mainly for demonstration.Source: §3.3.1 · p. 10

3.3.2 Reinforced Selectivity

• On Caltech face/motorbike, reward and punishment produced robust category selectivity on training samples after 200 iterations, while test-sample selectivity took 300 additional iterations to emerge.Early training with 50% dropout yielded unstable responses to random stimuli; the network was faster to find features for seen samples than to converge on features applicable to testing samples.Source: §3.3.2 · p. 11

3.3.3 Performance

• R-STDP achieved 98.9 ± 0.4% on Caltech face/motorbike, 89.5 ± 1.9% on ETH-80, and 88.4±0.5% on NORB, versus STDP’s bests of 97.2%, 84.5%, and 66%, respectively.On NORB, both methods struggled to distinguish four-leg animals from humans and cars from trucks; R-STDP produced a more balanced error distribution.Source: §3.3.2 · p. 12, §3.3.3 · p. 12

3.3.4 Overfitting Problem

• The proposed network tended to overfit; adaptive learning rates improved test performance, and the dropout probability needed to be higher for the more complex NORB dataset than for ETH-80.The need for dropout increased further with steady learning rates, where faster convergence and greater sample complexity intensified overfitting.Source: §3.3.4 · p. 13

Discussion

• R-STDP trains class-assigned neurons to respond early to their preferred category; classification uses the first-spiking neuron's latency, with at most one spike per neuron per image.Inputs are encoded as latency-coded spikes after convolution with oriented Gabor filters.Source: §4 · pp. 14–15
• R-STDP solved a hand-designed spike-order discrimination task with fewer neurons, synapses, and lower thresholds than STDP, which tends to learn frequent rather than diagnostic features.STDP could compensate by using more neurons and tuned thresholds, while R-STDP was more efficient in computational resources.Source: §4 · pp. 14–15
• R-STDP outperformed STDP and the same-structure CNN on Caltech Face/Motorbike, ETH-80, and NORB, using first-spike decisions without benefit from powerful external classifiers.ETH-80 and NORB contain objects shown from different viewpoints, making recognition harder.Source: §4 · p. 14
• The one-spike limit and event-driven processing make the network potentially energy-efficient and hardware-friendly, while R-STDP is more suitable for online on-chip learning than deep-network error backpropagation.Hardware implementation and efficiency were outside the paper's scope.Source: §4 · p. 15
• Despite outperforming STDP with external classifiers, the network remains below state-of-the-art deep learning; its single trainable layer and large receptive fields limit robustness to object variation.Future work could add layers, though deeper networks require more parameter tuning and a suitable multilayer plasticity rule.Source: §4 · p. 15
• Future work could use reinforcement signals to learn semantic associations between diagnostic features, linking instances such as different animals or face views into broader categories.R-STDP could support learning connections between categories and forming super-categories.Source: §4 · pp. 15–16

Evaluation protocol

• Across benchmarks, training-set images drove per-image synaptic plasticity, while testing-set images assessed categorization of unseen samples; parameters were exhaustively searched.Input-window sizes were chosen according to the datasets’ average target-object sizes.Source: §1 · p. 20

Caltech face/motorbike

• Caltech face/motorbike formed a two-class benchmark with 435 images per category, including 200 training images per category, and images scaled by 0.5.The remaining images were used for testing.Source: §1 · p. 20

ETH-80

• ETH-80 tested view-invariant recognition across eight object categories using leave-one-instance-out evaluation, with 2,952 training and 328 testing images per trial.Each category contains 10 instances, with 41 viewpoints per instance; each trial trains on nine instances and tests on the remaining one.Source: §1 · pp. 20–21

NORB

• NORB covered five object classes under varied viewpoints and lighting, using the authors’ train/test sets subsampled to 3,600 images split evenly between training and testing.The subset used the left camera, lighting level 4, four odd-numbered elevations, and all 18 azimuths.Source: §1 · p. 21

2.1.1 Feature Extraction

• R-STDP extracted discriminative category features that distinguished similar apples and tomatoes by including the tomato’s head.Figure S2 shows examples of features extracted for ETH-80 categories.Source: §2.1.1 · p. 21, Figure S2 · p. 21

2.1.2 Confusion Matrix

• R-STDP produced more balanced, symmetric confusion matrices than STDP, whose frequent-feature learning caused cross-category firing and classifier bias between similar objects.The asymmetry measure is closer to 0 for a more symmetric matrix.Source: §2.1.2 · pp. 21–22, Figure S3 · p. 22

2.2 Dropout

• Dropout improved performance by increasing the chance of finding discriminative features and decreasing blind firings.In each training iteration, dropout randomly turns off some neurons; Figure S4 compares neuron involvement rates with and without dropout on ETH-80 and NORB.Source: §2.2 · p. 22, Figure S4 · p. 23

2.3 Shallow CNN

• The shallow CNN comparison model used convolutional and pooling layers followed by dense hidden and output layers.It took grayscale images, used four feature maps in its first convolutional layer, ReLU activations for convolutional and hidden layers, and Softmax for output; the authors tuned the hidden-layer size and explored dense-layer dropout and kernel regularizers.Source: §2.3 · pp. 22–23

2.4 Comparison with Deep Convolutional Neural Network

• Fine-tuned VGG16 always outperformed the proposed network on ETH-80 and NORB, while training from scratch on NORB caused serious overfitting.The authors note that performance depends on fine-tuning strategy and parameter values, so these results are not guaranteed to be optimal.Source: §2.3 · p. 23, §2.4 · p. 24