PaperlayerView exact revision on arXiv

Source-linked AI summary

Saliency-Guided Replay for Continual Learning in Deep Spiking Neural Networks

Francesco Panetta, Federico Di Valerio, Michela Proietti

arXiv:2610.17998v1cs.LGcs.AIcs.CVcs.NE

TL;DR

Continual learning aims to preserve earlier knowledge as tasks change, but replay buffers require memory that can be impractical for constrained systems. This paper uses spike-based saliency to select compact replay patches for deep SNNs, improving accuracy at equal budgets in low-memory regimes while reducing storage.

  • Problem

    Replay can preserve prior task knowledge, but larger buffers improve retention while imposing memory costs that challenge resource-constrained systems.

    Source: §1 · p. 1

  • Method

    The paper adapts EPR to deep SNNs by using forward-only SAM explanations from spike activity to select image patches for replay.

    Source: §1 · p. 2, §2 · p. 3, §3.1 · p. 3

  • Results

    Across Split CIFAR10 and CIFAR100, SAM-guided replay improves equal-budget performance and reduces forgetting in low-memory regimes; at MB ≈10 on CIFAR100/10, CIL AvgAcc increases by 5.8 points.

    Source: §1 · p. 2, §5.1 · p. 7

  • Takeaways & Limitations

    EPR-SP retains competitive TIL and CIL accuracy while reducing storage cost by 44%, supporting a favorable trade-off in highly memory-constrained settings.

    Source: §5.1 · p. 7, §5.2 · p. 8

    The approach depends on SAM quality and a fixed patch size, and is evaluated on small-scale image-classification benchmarks.

    Source: §6 · p. 9

Abstract

from arXiv · show

Spiking Neural Networks (SNNs) are promising for energy-efficient neuromorphic computing, but they still suffer from catastrophic forgetting when learning tasks sequentially. This work investigates the integration of Experience Packing and Replay (EPR) as a memory-efficient continual learning strategy for deep SNNs under low-latency constraints. The proposed framework combines Batch Normalization Through Time stabilized VGG9 SNNs with Spike Activation Maps (SAM), which provide forward-only explanations based on spike activity. These explanations are temporally aggregated into spatial attention maps and used to select the most informative image patches for replay, instead of storing full images. Experiments on Split CIFAR10 and Split CIFAR100 show that SAM-guided EPR improves the accuracy-memory trade-off under tight memory budgets, and remains competitive while requiring substantially less storage.

1 Introduction

Continual learning must retain prior knowledge as tasks evolve, but replay buffers can be costly under resource constraints. This work adapts saliency-guided replay to deep SNNs using spike-based explanations.

1 Introduction

  • Replay methods preserve prior samples during training, but larger buffers improve retention while imposing costs that challenge resource-constrained systems.EPR reduces storage by retaining saliency-guided patches rather than full samples.Source: §1 · p. 1
  • SNNs offer sparse, event-driven computation for energy-efficient machine intelligence, but continual learning in SNNs remains underexplored.Prior replay adaptations include standard experience replay and compact latent replay.Source: §1 · pp. 1–2
  • The work adapts EPR to deep SNNs with forward-only Spike Activation Maps, evaluating BNTT-trained VGG9 models at 25 timesteps on Split CIFAR10 and CIFAR100.The reported results show improved equal-budget performance and reduced forgetting in low-memory regimes.Source: §1 · p. 2

2 Related Work

Replay-based continual learning for SNNs remains underexplored, and saliency-guided methods developed for ANNs do not account for spiking computation’s temporal dynamics. This work uses spike-compatible saliency to select compact replay information.

2 Related Work

  • Replay-based continual learning for SNNs remains underexplored, while ANN continual-learning methods have begun using XAI to preserve task-relevant information.EPR selects discriminative image patches from saliency maps to reduce memory requirements while maintaining competitive performance.Source: §2 · p. 2
  • Applying ANN explanation mechanisms directly to SNNs overlooks sparse event-driven activity and temporal dynamics in spiking computation.The paper identifies spike-compatible explanation strategies as a way to use temporal spiking activity.Source: §2 · pp. 2–3
  • The method replaces EPR’s Grad-CAM-based patches with SAM-based spatio-temporal saliency, making explanation part of replay rather than only a diagnostic tool.SAM is used to select compact replay information.Source: §2 · p. 3

3 Methods

The framework combines SAM-derived saliency with EPR to select compact replay patches for deep SNNs. It uses forward spike activity to guide selection, replays reconstructed inputs alongside current-task data, and evaluates final average accuracy and forgetting.

3.1 Spike Activation Maps

  • SAM provides forward-only explanations from spike-train activity, avoiding gradient-based methods that are difficult to apply to discrete, non-differentiable SNN dynamics.A causal temporal kernel weights recent spikes while restricting each timestep to past activity; accumulated history is combined with current spikes to form heatmaps.Source: §3.1 · p. 3
  • For replay selection, SAM heatmaps are aggregated across time by averaging all timesteps or using only the final heatmap.The deepest convolutional layer is chosen for its richer semantic representations and more discriminative saliency maps, despite lower spatial resolution.Source: §3.1 · pp. 3–4

3.2 Experience Packing and Replay

  • EPR-SP upsamples an aggregated SAM heatmap, scans the image with a 24 × 24 window, and stores the candidate patch with the maximum saliency score.Each exemplar can retain its coordinates, class label, selection heatmap, and optional SAM history.Source: §3.2 · p. 4
  • During replay, stored patches are restored at their original coordinates on zero-valued canvases and trained alongside current-task minibatches.EPR-SP stores samples only when prediction is correct or the ground-truth class is among the top three; BASE-EPR instead draws samples uniformly at random.Source: §3.2 · p. 4
  • Explanations guide exemplar selection rather than an explicit explanation-matching loss, while λ_replay controls the stability–plasticity trade-off.This formulation is described as compatible with SAM’s class-agnostic nature.Source: §3.2 · p. 5

3.3 Evaluation Metrics

  • Final performance is evaluated using average accuracy and forgetting across tasks, with a_j,i denoting accuracy on task j after training through task i.The number of tasks is denoted by N.Source: §3.3 · p. 5

4 Experimental Setup

Experiments use Split CIFAR benchmarks and compare replay variants under task- and class-incremental learning. Hyperparameter ablations select EPR-SP with mean temporal aggregation as the default configuration.

4 Experimental Setup

  • The experiments use a VGG-9-inspired SNN with per-timestep batch normalization, 25-timestep Poisson inputs, and Split-CIFAR10 and Split-CIFAR100 in both TIL and CIL settings.The benchmarks include CIFAR10 with five tasks and CIFAR100 with ten- and twenty-task versions; BASELINE stores full 32 × 32 images.Source: §4 · p. 5

4.1 Hyperparameter Search

  • Across the three benchmarks, EPR-SP consistently outperforms or ties BASE-EPR, while mean aggregation yields consistently higher or comparable accuracy.The selected defaults are EPR-SP with mean aggregation; replay weight is 1.0 for CIFAR10 and tight-budget CIFAR100, and 0.3 for larger-budget CIFAR100.Source: §4.1 · p. 6, Figure 1 · p. 6
  • Across benchmarks, EPR-SP stores more patches than full-image BASELINE at equal memory budgets and achieves higher accuracy at tight and moderate budgets, with the advantage diminishing as memory grows.The figure reports TIL and CIL accuracy versus memory budget.Source: Figure 2 · p. 7
  • SAMs are computed from layer 7 with temporal decay γ = 0.7, and learning rates are selected from 0.1 or 0.3 before cosine annealing within each task.The grid search varies membrane leak, learning rate, and dropout; early stopping uses validation accuracy.Source: §4.1 · pp. 6–7

5 Results

Under tight memory budgets, EPR-SP outperforms standard replay; with equal sample counts, it retains competitive accuracy while using less storage. SAM maps also show largely preserved salient regions across continual learning.

5.1 Comparison Under Equal Memory Budgets

  • At MB ≈10, AvgAcc increases by 3.1 and 2.4 points in TIL and 5.8 and 3.2 points in CIL on CIFAR100/10 and CIFAR100/20, while CIL AvgForg decreases by 17.8 and 13.7 points.EPR-SP outperforms BASELINE at the tightest memory budget across datasets; the gap narrows as the budget grows.Source: §5.1 · pp. 7–8

5.2 Comparison Under Equal Sample Budgets

  • At the same number of exemplars per class, EPR-SP uses 44% less memory than BASELINE while maintaining competitive TIL and CIL accuracy.At the tightest CIFAR10 budget, EPR-SP outperforms BASELINE; across equal sample budgets, its storage cost is 44% lower.Source: §5.1 · p. 7, §5.2 · p. 8, Figure 3 · p. 8

5.3 Explanation Drift Analysis

  • Across tasks, salient regions in SAMs remain largely consistent, and final maps for earlier tasks are often more concentrated on target objects.Figure 4 compares each task’s SAM immediately after learning with its SAM after all 20 tasks.Source: §5.3 · p. 9, Figure 4 · p. 9

6 Conclusions

The paper presents SAM-guided replay for continual learning in deep SNNs and reports favorable accuracy–memory trade-offs, particularly under tight budgets. Its evidence is limited to small-scale image classification benchmarks, and the approach depends on SAM quality and a fixed patch size.

6 Conclusions

  • SNN-adapted EPR uses SAM-guided replay for continual learning in deep SNNs and shows favorable accuracy–memory trade-offs, particularly under tight memory budgets.Qualitative analysis indicates that task-relevant representations are preserved and may become more localized over time.Source: §6 · p. 9
  • The evaluation covers small-scale image-classification benchmarks, uses a fixed patch size, and depends on SAM quality.The authors suggest pretrained backbones to improve early explanation quality and class-sensitive SAM as future directions.Source: §6 · p. 9

Related papers

Loading 2610.17998v1…