PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning While Inferring: Local and Parallel Learning for Edge SNNs across Sensing Modalities

Yanxun Zhang, Yifei Wang, Changze Lv, Jingwen Xu, Yiyang Lu, Xiaohua Wang, Di Yu, Xin Du, Xiaoqing Zheng

arXiv:2610.03149v1cs.NE

TL;DR

Always-on edge sensors must adapt without pausing inference, but standard surrogate-gradient backpropagation blocks incoming data during its global backward pass. BSD pairs locally aligned forward and reverse spiking branches so inference and learning can proceed concurrently. Across 25 SOUL benchmarks, it stays within 3.8 percentage points of matched BP on average and its representations transfer to few-shot class-incremental learning.

  • Problem

    Always-on edge devices need to adapt to changing conditions while sensing continuously, but standard surrogate-gradient BP requires a global backward pass that blocks new inputs.

    Source: §1 · pp. 1–2

  • Method

    BSD couples a stimulus-driven forward spiking branch with an independent target-driven reverse branch, aligning intermediate representations through local objectives.

    Source: §4.1 · p. 4

  • Results

    Across 25 SOUL tasks, BSD stays within 3.8 percentage points of matched BP on average (81.12% vs. 84.89%), and its representations transfer to few-shot class-incremental learning.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    BSD’s disjoint branches permit concurrent inference and updates, and deployment requires only the forward branch.

    Source: §6 · p. 10

    Efficiency figures are operation-level estimates and projections rather than measurements on specific hardware.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Edge intelligence requires models to sense continuously in real time and to keep adapting on-device, all under tight compute, energy, and memory budgets. Although spiking neural networks (SNNs) enable efficient event-driven inference, standard surrogate-gradient backpropagation (BP) serializes updates and blocks ongoing inference. We investigate Bidirectional Spike-Based Distillation (BSD) as an on-device learning principle that lets edge SNNs learn while inferring. BSD couples a stimulus-driven forward network with an independent target-driven reverse network and aligns their intermediate representations through local objectives. Because the two pathways have disjoint computation graphs until alignment, forward inference, reverse inference, and stage-wise updates can run concurrently. On 25 benchmarks spanning the five sensing modalities of SOUL, BSD stays within 3.8 percentage points of matched BP baselines on average, while only its forward branch is needed at deployment. The learned representations also transfer well to few-shot class-incremental learning without replaying past data. By removing the dense floating-point backward chain, BSD reduces the projected training latency to $0.72\times$ and the estimated training energy to $0.36\times$ that of BP.

1 INTRODUCTION

Always-on edge sensors must adapt to changing conditions without interrupting real-time sensing, but standard surrogate-gradient backpropagation blocks incoming data during its sequential global update. BSD uses locally aligned forward and reverse spiking branches to overlap learning with inference; the paper reports competitive accuracy across sensing modalities and transfer to few-shot incremental learning.

Motivation

  • Standard surrogate-gradient BP propagates floating-point errors across layers and time steps, requiring a global sequential pass that blocks ongoing inference.Sparse spiking inference alone does not remove this training bottleneck.Source: §1 · pp. 1–2

Contributions

  • BSD stays within 3.8 percentage points of BP on average across five sensing modalities, while its training pipeline overlaps work that BP performs sequentially.Figure 1 summarizes the modality scope and contrasts BSD’s concurrent pipeline with BP’s sequential one.Source: Figure 1 · p. 2, §1 · p. 3

BSD overview

  • BSD couples stimulus-driven forward and target-driven reverse spiking branches through local ReCo objectives, allowing inference, reverse inference, and updates to run concurrently.The branches operate independently until their local losses are computed, avoiding a global error-backpropagation chain.Source: §1 · p. 2, Figure 1 · p. 2

Contributions

  • Independent local gradient computations expose parallelism and shorten the training critical path, while spike-based reverse inference reduces operation-level energy relative to dense floating-point BP.The paper presents these as resource advantages for on-device adaptation.Source: §1 · pp. 2–3
  • BSD’s local learning is evaluated for few-shot class-incremental learning, where representations support adaptation to novel classes from only a few labels.The setting reflects novel concepts arriving with few labels and constraints against replaying historical data.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work addresses edge efficiency, online adaptation, and SNN training, but the cited literature leaves BSD’s cross-modal viability, concurrency, and few-shot incremental generalization unstudied systematically.

Edge computing and online learning

Training spiking neural networks

  • SNN training work reduces latency, memory, or update costs, but surrogate-gradient approaches remain tied to a global cross-layer error chain that serializes execution and blocks inference.Examples include temporal re-weighting (Deng et al., 2022), online updates (Xiao et al., 2022), and sparsity-promoting objectives.Source: §2 · p. 3
  • Decoupled learning breaks global dependencies; for SNNs, prior examples include DECOLLE and EC-SNN (Di Yu et al., 2024), while BSD uses opposing pathways aligned locally.Whether BSD works across edge modalities, supports concurrent learning and inference, and generalizes in few-shot incremental learning had not been systematically studied.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries specify the discrete-time LIF dynamics used by the SNNs and define FSCIL as sequential class expansion under few-shot supervision without retraining on earlier data.

3.1 SPIKING NEURAL NETWORKS

  • The SNNs use the standard discrete-time leaky integrate-and-fire neuron model, with membrane potentials defined before and after the spike-triggered reset.Incoming synaptic current, membrane time constant, firing threshold, reset potential, and emitted binary spike are specified; learning uses an ATan surrogate derivative with scale α = 2.Source: §3.1 · pp. 3–4

3.2 FEW-SHOT CLASS-INCREMENTAL LEARNING

  • FSCIL starts with a base class session, then adds disjoint new classes in ordered sessions with K labeled support samples per new class, without retraining on earlier sessions.After each session, the model classifies examples from all classes encountered so far.Source: §3.2 · p. 4
  • FSCIL evaluation reports seen-class accuracy and an H score based on the harmonic mean of per-class accuracies, which reflects weak performance on poorly recognized classes.A high H score requires balanced accuracy across base and novel classes.Source: §3.2 · p. 4

4 METHODS

BSD trains feedforward and temporal SNNs by aligning forward and target-driven reverse pathways locally, then uses learned features for prototype-based few-shot class-incremental learning. Its independent branches and stage-local updates expose concurrency, while spike-based reverse inference avoids dense backward operations.

4.1 GENERALIZED BSD FOR FEEDFORWARD MODELS

  • Independent forward and reverse branches align intermediate representations with local ReCo objectives; each stage’s gradients remain local, and the task loss applies only to the final forward readout.The input sequence enters the forward branch, while the reverse branch receives a spike-encoded target label.Source: §4.1 · pp. 4–5

4.2 BSD FOR TEMPORAL MODELS

  • The forward RNN processes inputs chronologically, while the reverse RNN processes the target backward in time; their hidden states are aligned at every time step.For spiking LSTMs, alignment covers hidden states, not cell states; state detachment can restrict temporal credit assignment or permit truncated backpropagation through time.Source: §4.2 · p. 5

4.3 PROTOTYPE-BASED FSCIL

  • A frozen BSD backbone supplies features for class prototypes; incremental sessions build novel prototypes from support samples while keeping base prototypes fixed.Classification fuses cosine similarities from intermediate layers and adds a class-dependent bias to compensate for score gaps between base and novel prototypes.Source: §4.3 · p. 5

4.4 TIME- AND ENERGY-AWARE BSD EXECUTION

  • Independent forward and reverse inference and stage-local gradients and updates enable overlap; latency depends on available parallel resources and synchronization, while reverse spike accumulations replace dense BP MACs.BP propagates floating-point errors through layers and time steps, whereas BSD uses event-driven reverse inference and stage-confined local gradients.Source: §4.4 · p. 6

5 EXPERIMENTS

Across 25 SOUL tasks, BSD remains close to matched BP baselines while reducing projected training time and estimated energy. Its CIFAR-100 FSCIL results track BP, and ablations identify contributions from the adaptation components.

5.1 DATASET PROCESSING

  • The 25 SOUL tasks span five sensing modalities, and BSD and BP use identical preprocessed tensors and train/test splits.Images, motion, audio, wireless measurements, and event streams are converted to regular-shaped inputs; matched models use the same forward architecture and input pipeline. Simulations use T = 4 steps, or T = 10 for the event-camera datasets.Source: §5.1 · p. 7, §5.2 · p. 7

5.2 EXPERIMENTAL SETTINGS

  • BSD's matched models use modality-appropriate convolutional or recurrent SNNs, with the same forward architecture and input pipeline as BP.The experiments profile accuracy, firing sparsity, latency, and operation-level energy; the BSD objective uses ReCo penalty weight γ = 0.6 and layer-wise loss weight 25.0.Source: §5.2 · pp. 7–8

5.4 TRAINING TIME AND ENERGY ANALYSIS

  • BSD/BP cost ratios have geometric means of 0.72× for projected latency and 0.36× for training energy.Projected latency is below BP on 20 of 25 benchmarks; energy is below BP on all 25. Figure 3 covers ten representative benchmarks, two per modality.Source: Figure 3 · p. 8, §5.4 · p. 9

5.3 MAIN EXPERIMENTAL RESULTS

  • BSD stays within 3.8 percentage points of BP on average across 25 tasks (81.12% vs. 84.89%).The gap is within 3 points on 11 benchmarks and within 5 points on 19.Source: §5.3 · p. 8
  • Wireless results are partly constrained by memory-driven minibatches of 4 to 16, because ReCo needs enough instance pairs to shape intermediate representations.The authors frame the accuracy level as a trade-off given that deployment uses only the forward branch and BSD reduces training latency and energy.Source: §5.3 · p. 8

5.5 FSCIL PERFORMANCE

  • BSD tracks BP across FSCIL sessions; final seen accuracy is 42.18% vs. 44.62%, and H score is 32.54% vs. 32.85%, with slightly higher H scores in the first two incremental sessions.Figure 4 reports the two metrics per session as mean ± std over three runs.Source: Figure 4 · p. 9, §5.5 · p. 9

5.6 ABLATION STUDIES

  • Removing any adaptation component degrades both metrics; multi-layer fusion most affects seen accuracy (3.7 points), while prototype TTA contributes most to H score (+3.4 points).The components provide a training-free adaptation strategy without backbone retraining.Source: §5.6 · pp. 9–10, Table 2

6 CONCLUSION

BSD supports on-device edge-SNN learning while inference continues, with accuracy near matched BP baselines across five sensing modalities. Its representations transfer to few-shot class-incremental learning, while projected training latency and estimated energy are lower than BP.

6 CONCLUSION

  • Across 25 benchmarks in five sensing modalities, BSD stays within 3.8 percentage points of matched BP baselines on average.Only the forward branch is needed at deployment.Source: §6 · p. 10
  • Projected training latency is 0.72× that of BP, and estimated training energy is 0.36× on average across datasets.The energy ratio is a geometric mean; energy is lower on all 25 datasets.Source: §6 · p. 10
  • BSD couples stimulus-driven forward and target-driven reverse branches with local objectives, allowing branch inference and stage-wise updates to run concurrently.The branches have disjoint computation graphs until alignment.Source: §6 · p. 10
  • BSD-trained representations transfer to 5-shot class-incremental learning.The passage characterizes this transfer as strong.Source: §6 · p. 10

REPRODUCIBILITY STATEMENT

The paper documents dataset preprocessing, implementation, and evaluation procedures to support reproducibility. Accuracy results are averaged across three runs, and code and related materials are slated for release upon publication.

REPRODUCIBILITY STATEMENT

  • Dataset characteristics and preprocessing procedures are documented in Appendix C.The paper uses public benchmark datasets and reports dataset-specific procedures.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Appendices C and D provide network architectures, hyperparameters, training procedures, and evaluation protocols.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Accuracy results are averaged over three runs, and source code, configurations, and plotting scripts are slated for release upon publication.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

The included material states ethical and privacy cautions, explains surrogate-gradient and temporal BSD choices, and outlines training procedures, datasets, and model architectures.

ETHICS STATEMENT

  • Wireless and camera-free monitoring or identification can occur without subjects’ awareness, so the authors call for informed consent and appropriate access control.On-device processing reduces raw-data transmission but does not itself guarantee privacy or prevent misuse.Source: ETHICS STATEMENT · p. 10

A.1 SURROGATE GRADIENT AND HEAVISIDE STEP FUNCTION

  • The Heaviside threshold has a zero-almost-everywhere derivative, so training uses a smooth surrogate derivative during backward computation while forward activations remain binary.The experiments use the arctangent surrogate with α = 2.Source: §A.1 · p. 14

B.1 TEMPORAL BSD WITH SPIKING LSTMS

  • Temporal BSD aligns forward and reverse hidden states at each time step, but leaves LSTM cell states unaligned and limits temporal credit assignment to segments of length n.n = 1 gives time-local updates; n > 1 permits truncated backpropagation within each segment.Source: §B.1 · pp. 14–15, Figure 5 · p. 14

B.2 PSEUDOCODE

  • BSD’s feedforward and temporal training steps run branches concurrently, detach aligned representations, and parallelize independent per-stage gradient tasks and parameter updates.The pseudocode covers CNN and RNN/LSTM models.Source: §B.2 · p. 15

C.1 DATASET COMPOSITION AND PREPROCESSING

  • The benchmarks cover vision, motion, acoustic, wireless, and neuromorphic sensing, with dataset-specific inputs and preprocessing across these modalities.Examples include resized or native-resolution images, windowed motion sequences, log-mel spectrograms, wireless tensors, and accumulated or rasterized event data.Source: §C.1 · pp. 15–16

C.2 MODEL ARCHITECTURES

  • Feedforward tasks use convolutional SNNs with mirrored reverse branches, while temporal tasks use BSD RNN, spiking LSTM, or GRU variants with hidden-state alignment.BP baselines reuse the corresponding BSD forward architecture; BSD’s reverse branch is training-only and discarded at deployment.Source: §C.2 · p. 16

C.3 TRAINING CONFIGURATIONS

The paper specifies shared training configurations, an FSCIL protocol using a frozen BSD forward backbone, and an operation-level energy-counting model whose values are estimates rather than hardware measurements.

C.3 TRAINING CONFIGURATIONS

  • Common training uses a cosine schedule, at most 200 epochs, gradient clipping at 0.3, and three seeds, with dataset-specific settings reported in Table 3.The common schedule has a 30-epoch warm-up over a 120-epoch horizon; BSD uses ReCo weight γ = 0.6 and layer-loss weight 25.0.Source: §C.3 · pp. 16–17, Table 3 · p. 17

C.4 FSCIL CONFIGURATION

  • FSCIL uses 75 base classes and five 5-way incremental sessions with five labeled examples per novel class, after which the forward backbone is frozen.Novel prototypes use view-averaged support features and are projected toward the base-prototype subspace; Figure 6 summarizes the pipeline.Source: §C.4 · p. 17, Figure 6 · p. 18

D.1 ENERGY COUNTING RULES

  • The energy model counts binary-input synaptic operations as ACs and continuous-input operations as MACs, but its 45 nm-based estimates are proxies, not measurements on specific hardware.The model assigns 4.6 pJ per MAC and 0.9 pJ per AC; BSD’s reverse branch is mostly ACs and its local gradients avoid a cross-layer error chain.Source: §D.1 · p. 18

D.2 PER-DATASET MEASUREMENTS

Per-dataset measurements compare matched BP and BSD operation counts, estimated training energy, and profiled training-step times. Across the 25 datasets, BSD's geometric-mean optimized-time and energy ratios are below one, though five datasets have optimized-time ratios above one.

D.2 PER-DATASET MEASUREMENTS

  • Table 4 reports per-sample MACs, ACs, and estimated training energy, while Table 5 profiles training-step phase times and optimized step times.Energy estimates use 4.6 pJ per MAC and 0.9 pJ per AC; optimized BSD time is max(TF, TR) + (TL + TU)/P.Source: §D.2 · pp. 18–19, Table 4 · p. 19
  • 0.72× for optimized time and 0.36× for energy are the geometric-mean BSD-to-BP ratios across datasets.Energy ratios are below one on all 25 datasets, while optimized-time ratios exceed one on five.Source: §D.2 · p. 19
  • 2.11 is the highest optimized-time ratio, on BullyDetect; the other four exceptions are SVHN, Widar3, AOPHand, and MMActivity.The passage attributes most exceptions to shallow or small-scale models with few parallel gradient tasks.Source: §D.2 · p. 19

E BATCH-SIZE SENSITIVITY

This section tests how BSD accuracy changes with training batch size, motivated by ReCo's reliance on negative sample pairs. Accuracy is largely preserved at batch size 32 but degrades noticeably below 16.

E BATCH-SIZE SENSITIVITY

  • ReCo's off-diagonal objective relies on negative sample pairs within a batch, so its contrastive signal weakens as batch size shrinks.Source: E BATCH-SIZE SENSITIVITY · p. 19
  • The evaluation varies batch size over 64, 32, 16, 8, and 4 while using each dataset's BSD configuration from Table 3.It tests one representative dataset per modality: CIFAR-10, MotionSense, GTZAN, AOPHand, and SHD.Source: E BATCH-SIZE SENSITIVITY · pp. 19–20
  • 32: accuracy is largely preserved, while it degrades noticeably below batch size 16.The authors state that a batch of 32 remains relatively small for on-device training and yields performance at most a few points below the large-batch setting.Source: E BATCH-SIZE SENSITIVITY · p. 20, Table 6 · p. 20

F LIMITATIONS AND FUTURE DIRECTIONS

The paper identifies hardware validation and architectural breadth as limitations, and discloses the use of large language models during manuscript preparation. Its evaluation covers convolutional, recurrent, and gated-recurrent models, while larger families remain to be explored.

F LIMITATIONS AND FUTURE DIRECTIONS

  • Efficiency results are operation-level estimates and GPU timing profiles; the models have not yet been deployed on actual edge devices.The projected training-time reduction assumes concurrent execution of both branches and local-gradient tasks, and approaching the theoretical optimum may require specialized hardware exposing that parallelism.Source: F LIMITATIONS AND FUTURE DIRECTIONS · p. 20
  • The evaluated architecture families are convolutional, recurrent, and gated-recurrent models; whether BSD extends to Transformers remains unexplored.The passage says Transformers would require a dedicated design for the attention-based reverse branch and alignment points.Source: F LIMITATIONS AND FUTURE DIRECTIONS · pp. 20–21

G THE USE OF LARGE LANGUAGE MODELS

  • Large language models assisted with literature retrieval, text editing, and plotting scripts; the authors reviewed and verified generated content and take responsibility for the manuscript.Source: G THE USE OF LARGE LANGUAGE MODELS · p. 21

Related papers

Loading 2610.03149v1…