PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Depth Temporal Fusion for Feedforward, Locally Trained Spiking Neural Networks

Aidin Attar, Eleonora Cicciarella, Michele Rossi

arXiv:2609.37047v2cs.NEcs.CVcs.LG

TL;DR

Deep SNNs trained with local STDP/R-STDP leave open how to route temporal evidence as processing depth increases. The paper combines a four-layer locally trained spiking pipeline with MDTF, which preserves early evidence and selectively adds deeper events. It reports gains over traditional STDP/R-STDP baselines on higher-variability visual tasks.

  • Problem

    How to route intermediate temporal representations in deeper SNNs trained with local plasticity remains poorly understood, although suppressed information cannot be recovered by later stages through a global error signal.

    Source: §2 · p. 4

  • Method

    A label-free TTFS front end and four-layer STDP-trained convolutional backbone feed MDTF, which preserves early evidence and selectively incorporates residual and temporally agreeing deeper events before a multi-prototype R-STDP readout.

    Source: §1 · p. 3, §3 · p. 5

  • Results

    +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10 versus traditional STDP/R-STDP baselines, with high accuracy retained after removing many late or weak spike events.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Preserving and refining the early temporal code is preferable to progressively replacing it with deeper representations, with refinements increasingly useful on more complex data.

    Source: §7 · p. 18

    Extension to larger event-based tasks, recurrent architectures, and neuromorphic hardware, along with direct activity and energy measurements, remains future work.

    Source: §7 · p. 18

Abstract

from arXiv · show

We propose a new spiking neural network (SNN) design to process static images and event streams using time-to-first-spike (TTFS) latencies. Our key research question is which architectural choices best accommodate local and online learning in multi-layer convolutional SNNs. This question is addressed via an original framework combining residual-like connections with multi-depth feature aggregation and consensus. The full SNN pipeline features an early-vision front end, to convert raw visual data into sparse spike latencies, a four-layer convolutional backbone trained layerwise with unsupervised spike-timing-dependent plasticity (STDP), a deterministic Multi-Depth Temporal Fusion (MDTF) and a final classifier trained with reward-modulated spike-timing-dependent plasticity (R-STDP). Rather than replacing early features in deeper layers, the proposed MDTF preserves early temporal evidence, adding sparse residual events from intermediate layers, and incorporating deeper features only when they agree in time with earlier representations. The resulting architecture is experimentally validated across MNIST, Fashion-MNIST, CIFAR-10, and N-MNIST, delivering strong classification performance under a fully local learning regime. Selective multi-depth fusion significantly outperforms traditional STDP/R-STDP baselines on higher-variability visual tasks (achieving +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10). Furthermore, activity-budget analyses show that the network retains high accuracy even when removing a large fraction of late or weak spike events, confirming its high data efficiency and reduced event-processing requirements. The codebase is publicly available at github.com/aidinattar/multi-depth-temporal-fusion-snn.

1 Introduction

The paper asks how deep convolutional SNNs can preserve task-relevant temporal evidence when trained with local, online plasticity rather than global error signals. It proposes a multi-depth routing architecture and locally trained pipeline, reporting stronger results on higher-variability visual tasks than traditional STDP/R-STDP baselines.

1 Introduction

  • The paper situates its study within interest in event-driven SNNs, which use discrete spikes and explicit time under efficiency constraints.The introduction also outlines the paper's progression from related work and pipeline to representations, learning, experiments, and conclusions.Source: §1 · pp. 1, 3
  • Deeper SNNs trained with STDP/R-STDP face unclear trade-offs: added complexity may improve representations but exacerbate spike sparsity, unstable dynamics, and temporal credit assignment.Local STDP alone strengthens recurring patterns but does not directly favor features useful for a specific task.Source: §1 · p. 2
  • The design couples network topology to local plasticity by determining which spike-time events reach the classifier and remain available for subsequent synaptic adaptation.Its objective is to preserve informative temporal evidence as processing depth increases.Source: §1 · p. 2
  • The pipeline combines a label-free latency encoder, a four-layer convolutional backbone trained with unsupervised STDP, deterministic MDTF, and a multi-prototype R-STDP readout.The readout reinforces target-class prototypes and selectively punishes competitive non-target prototypes.Source: §1 · pp. 2–3
  • The proposed MDTF preserves early spike-time evidence while routing sparse residual and temporally agreeing deeper events as corrections to the initial code.The architecture treats depth as temporal evidence routing, because suppressed information cannot be directly recovered by later local plasticity.Source: §1 · pp. 2–3
  • +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10 versus traditional STDP/R-STDP baselines, with high accuracy retained after removing many late or weak spikes.The framework was evaluated on MNIST, Fashion-MNIST, CIFAR-10, and N-MNIST.Source: §1 · p. 3

2 Related Work

Prior work establishes local STDP and reward-modulated learning for spiking vision, but leaves open how intermediate spike-time representations should be routed as locally trained SNNs deepen. The paper frames architectural paths and branches as choices that affect which events reach later plasticity and classification.

2 Related Work

  • Prior studies span STDP feature learning, reward-modulated task learning, and event-based applications, but differing input statistics and objectives make cross-domain comparisons difficult.The paper positions its experiments within a common visual-recognition setting.Source: §2 · p. 3
  • The schematic combines a four-layer convolutional backbone with MDTF, preserving the early code and adding intermediate residual and temporally consistent deep contributions before an R-STDP readout.The readout uses earliest output spike timing for class scores; labels define reward and punishment signals at the decision stage.Source: Figure 1 · p. 4
  • Reward can modulate only spike-time features that reach the decision layer, so information suppressed by earlier locally trained stages cannot be recovered through global error propagation.This makes preservation and selective enrichment of temporal evidence more consequential as inputs and architectures become richer.Source: §2 · p. 4
  • Local STDP and R-STDP support spiking visual learning, but their effectiveness and feature routing in deeper locally trained SNNs remain unclear.The paper restricts experiments to visual classification across static and event-based inputs to examine routing without conflating task formulations.Source: §2 · pp. 3–4

3 Processing Pipeline Overview

The processing pipeline is a fully spiking architecture trained with local rules from sensory latency coding through classification. Its stages comprise a label-free front end, an STDP-trained convolutional backbone, MDTF, and an R-STDP readout.

3 Processing Pipeline Overview

  • The pipeline consists of a label-free TTFS front end, stage-wise convolutional STDP, residual- and agreement-based MDTF, and a multi-prototype R-STDP classifier.STDP trains convolutional layers S1–S4; R-STDP trains the readout.Source: §3 · p. 5
  • The front end establishes the latency code, and the backbone learns local feature maps with unsupervised STDP for subsequent classification.The front end is inspired by functional early-vision processing.Source: §3 · p. 5
  • MDTF preserves the initial code while adding residual and inception-like refinements, thereby determining which spike-time events remain available to later stages under local learning.In this setting, MDTF governs event availability rather than facilitating gradient flow.Source: §3 · p. 5
  • The final spiking readout uses reward- and punishment-modulated local synaptic updates with multiple neurons per class.Multiple neurons per class provide population coding.Source: §3 · p. 5

4 Signals and Features

The model represents activations as first-spike latencies, with silence marking missing spikes. A deterministic front end encodes static images or event streams, and Multi-Depth Temporal Fusion preserves shallow evidence while adding sparse intermediate and temporally agreeing deep events.

4.1 Spike-Time Representation

  • First-spike latency represents each activation; earlier spikes indicate stronger temporal evidence, while missing spikes are silent features.Concatenation preserves evidence from separate sources, top-k selection retains earliest or most informative latency changes, and branch agreement is temporal.Source: §4.1 · p. 5

4.2 Visual Front End and Latency Encoding

  • The label-free front end converts static or event-based visual input into sparse latency maps, using early-vision transformations for static images and a dedicated event stream procedure.Static processing includes decorrelation, polarity handling, gain control, and response-to-latency mapping; event inputs are denoised, temporally binned, and normalized per sample.Source: §4.2 · pp. 5, 7, 8

4.3 Design Principles for Multi-Depth Temporal Fusion

  • MDTF preserves the shallow representation P and adds sparse intermediate residual events plus deep events only when temporally consistent with the intermediate representation.The residual contribution retains the earliest finite intermediate events; the agreement contribution is sparsified, suppressing isolated or inconsistent deep events.Source: §4.3 · pp. 8–9

4.4 Instantiation of the Multi-Depth Temporal Fusion Mechanism

  • The instantiated backbone processes H(0) through S1/C1 and S2/C2, then sends H(2) through S3 and S4 to produce H(4) for temporal agreement with H(2).The final fusion combines H(1), the sparse residual from H(2), and agreement events from H(2) and H(4); H(3) is not included directly.Source: §4.4 · pp. 9–10

5 Local Learning Framework

The framework uses local, timing-based synaptic adaptation, with unsupervised STDP in the backbone and reward-modulated updates at the readout. Training proceeds layerwise without end-to-end backpropagation, followed by samplewise readout learning on a frozen backbone.

5.1 Learning Rules

  • Backbone convolutional STDP updates depend exponentially on current weights, whereas the R-STDP readout uses additive updates and relies on clipping and per-neuron ℓ1 normalization for stability.Unsupervised STDP layers use local winner selection and layer-specific learning-rate scheduling, without labels.Source: §5.1 · p. 10
  • The readout uses a local temporal objective: target prototypes firing too late are rewarded, while the K most violating non-target prototypes are selected as hard negatives for anti-STDP updates.The readout remains within the R-STDP family, and updates are based on pre/post spike timing with class-level modulation.Source: §5.1 · p. 11

5.2 Training Schedule

  • Each trainable layer is exposed to the preceding layer’s spike-time representation, updates only its assigned synapses, and is frozen before the next layer is trained.After the backbone is fixed, the readout is trained samplewise with R-STDP; the fused latency representation concatenates preserved shallow, residual, and agreement contributions.Source: §5.2 · pp. 11–12

6 Experiments and Results

Across four visual benchmarks, the locally trained pipeline remains effective on static and event-based inputs, while front-end ablations show that usable latency representations depend on the task. The results also report sparse readout codes and stable readout dynamics.

6 Experiments and Results

  • The experiments use label-free preprocessing, local STDP backbone training, and a readout trained on frozen spike-time features with reward-modulated STDP.Labels are used only by the readout reward signal, not to train the backbone.Source: §6 · p. 12

6.2 Main Results

  • The four-layer locally trained pipeline remains effective across MNIST, Fashion-MNIST, CIFAR-10, and event-based N-MNIST, with CIFAR-10 challenged by weak color and texture evidence.On N-MNIST, asynchronous polarity events are converted into locally normalized event-map latency representations.Source: §6.2 · p. 13
  • The reported accuracies come from sparse frozen representations, and test accuracy remains close to its best value during the final readout-training phase.The selected reporting epoch is based on validation performance; test trajectories assess stability rather than model selection.Source: §6.1 · p. 12, §6.2 · p. 13, Table 2 · p. 13

6.3 Front-End Configuration Ablations

  • Direct latency coding from raw intensity is close to chance on static benchmarks, while the best tested N-MNIST configuration uses the full event-map representation.On N-MNIST, raw event frames exceed 93% accuracy, but removing denoising or log compression causes modest, consistent losses.Source: Table 3 · p. 13, §6.3 · p. 14

6.4 Comparison against State-of-the-Art STDP/R-STDP Baselines

Against a closely matched local STDP/R-STDP baseline, the proposed architecture shows its advantage on more complex datasets, while the locally trained pipeline benefits from additional training examples across all four benchmarks.

6.4 Comparison against State-of-the-Art STDP/R-STDP Baselines

  • The proposed architecture gives major gains over the local STDP/R-STDP baseline on Fashion-MNIST and CIFAR-10, but no noticeable improvement on MNIST.The baseline underperforms on N-MNIST because it lacks a front end that converts dynamic inputs into a suitable latency format.Source: §6.4 · pp. 14–15, Table 4 · p. 15

6.5 Data Efficiency

  • The data-efficiency experiment reduces examples for both unsupervised STDP layers and the reward-modulated readout, probing the full local-learning pipeline.The readout training protocol is kept fixed as training budgets increase.Source: §6.5 · p. 15
  • Accuracy increases monotonically with training examples on all four datasets, with early saturation on grayscale benchmarks and CIFAR-10 remaining the most data-demanding static task.N-MNIST improves sharply once event-map representation leaves the few-sample regime.Source: §6.5 · p. 15, Table 5 · p. 15

6.6 Spike-Budget Pareto

Activity-budget tests show a gradual accuracy tradeoff as weak or late events are removed, while routing ablations find that preserving the early representation and adding sparse deeper contributions is more effective than replacing it.

6.6 Spike-Budget Pareto

  • Accuracy declines gradually as activity is reduced; simpler static benchmarks retain most full-code performance at smaller budgets, while CIFAR-10 and N-MNIST need more events for high accuracy.The test retains top-scoring latency features and retrains the readout with the backbone fixed; earlier spikes are treated as stronger evidence.Source: §6.6 · p. 15, Figure 4 · p. 16

6.7 Residual and Agreement Ablations

  • The preserved representation P remains dominant, and intermediate and deep representations alone are insufficient, whereas residual routing keeps P available as sparse contributions are added.Additional contributions provide selective refinements, especially on higher-variability datasets; incorporating them increases accuracy.Source: §6.7 · pp. 16–17, Figure 5 · p. 17

6.8 Decision-Level Confusion Repair

The full fused representation changes only a subset of P-only decisions, repairing more errors than it introduces on Fashion-MNIST, CIFAR-10, and N-MNIST. The routing analysis finds that preserving P while adding sparse residual and agreement evidence is more effective than replacing early representations with deeper ones.

6.9 Interpretation of Representation and Routing

  • Static-image tasks benefit from the structured front end rather than raw-intensity latency coding, while binned event-count latency performs best among tested N-MNIST configurations.The input-specific transformations provide the backbone with a common TTFS representation.Source: §6.9 · p. 17
  • Preserving P is more effective than replacing it with deeper representations, while sparse residual and agreement contributions add evidence without discarding the early code.These contributions modify only a subset of P-based decisions, repairing some remaining errors rather than broadly changing classifier output.Source: §6.9 · pp. 17–18

6.8 Decision-Level Confusion Repair

  • On Fashion-MNIST, CIFAR-10, and N-MNIST, repaired errors outnumber new errors; MNIST shows little net change because P-only performance is near saturation.Most P-only errors remain unchanged, and CIFAR-10 has the largest amount of decision change.Source: §6.8 · p. 17, Figure 6 · p. 18

7 Concluding Remarks

The work presents a locally trained multilayer SNN for image and visual-stream classification, with a final R-STDP readout. Its MDTF preserves early latency codes while refining them, and future work is proposed to test the principle in broader settings.

7 Concluding Remarks

  • The model processes image and visual-stream latency representations through multilayer convolutional SNNs trained online, with final classification by an R-STDP readout.The readout is trained using reward-modulated STDP.Source: §7 · p. 18
  • The MDTF combines residual connections, inception, and consensus, preserving early latency codes as deeper layers refine them.Across considered datasets, early codes remain the main source of discriminative evidence; refinements become more useful on complex data, and higher-variability tasks outperform existing STDP/R-STDP baselines.Source: §7 · p. 18
  • Future work can test temporal routing on larger event-based tasks, recurrent architectures, and neuromorphic hardware, with direct activity and energy measurements.Source: §7 · p. 18

CRediT authorship contribution statement

The authors’ contributions span conceptualization, methodology, software, investigation, supervision, and writing.

CRediT authorship contribution statement

A Reproducibility Details

The reproducibility details describe settings for the main four-layer models, covering the front end, backbone and fusion, and temporal R-STDP readout. Static and event inputs use different encoders before sharing the downstream pipeline.

A Reproducibility Details

  • The main four-layer models use different deterministic encoders for static images and event streams, followed by the same layer-wise backbone, sparse MDTF, and temporal R-STDP readout.Top-kmap and Top-kfus refer to map-wise and fusion sparsity caps; k/s/p denotes kernel size, stride, and padding.Source: A Reproducibility Details · p. 22
  • Table 6 lists front-end and shared optimization settings; whitening stride samples fitting patches, while the learned operator is applied densely with unit stride.Static calibration is fitted on the training split and held fixed thereafter.Source: Table 6 · p. 22
  • Table 7 reports backbone and fusion parameters, including the latency representation passed to the next stage and the adaptive deep path used by the final model.C2 representations use pooling kernel/stride 2/1.Source: Table 7 · p. 22
  • Table 8 gives temporal R-STDP readout operating points, including shared normalization, decoding, decay, target, and annealing settings.Updated-prototype counts specify selections from the target population and each selected hard-negative class.Source: Table 8 · p. 23

Related papers

Loading 2609.37047v2…