PaperlayerView exact revision on arXiv

Source-linked AI summary

Online Training Through Time for Spiking Neural Networks

Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, Zhouchen Lin

arXiv:2210.04195v2cs.NEcs.AIcs.LG

TL;DR

SNN training methods face trade-offs among low latency, memory use, theoretical clarity, and online learning. OTTT derives forward-in-time updates from BPTT while connecting its gradients to spike-representation gradients. The paper reports superior results across static and neuromorphic datasets using few time steps.

  • Problem

    BPTT with surrogate gradients can train SNNs at low latency but has high training-memory costs and unclear optimization guarantees, while spike-representation approaches require more time steps and both are inconsistent with online biological learning.

    Source: §1 · p. 2

  • Method

    OTTT tracks presynaptic activities and uses instantaneous losses to decouple temporal dependencies for forward-in-time learning, with gradients analyzed against spike-representation methods.

    Source: §1 · p. 2, §4.2 · p. 6

  • Results

    OTTT reports superior results on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS in a small number of time steps.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    OTTT combines constant training memory independent of time steps with a three-factor Hebbian update and a proved descent direction under feedforward and recurrent conditions.

    Source: §1 · p. 2, §4.2 · p. 6, §4.3 · p. 7

    The descent analysis assumes surrogate derivatives are equivalent to the well-defined mapping derivative, an approximation that is exact except in a rare stated case.

    Source: §4.2 · p. 6

Abstract

from arXiv · show

Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. Particularly, backpropagation through time (BPTT) with surrogate gradients (SG) is popularly used to achieve high performance in a very small number of time steps. However, it is at the cost of large memory consumption for training, lack of theoretical clarity for optimization, and inconsistency with the online property of biological learning and rules on neuromorphic hardware. Other works connect spike representations of SNNs with equivalent artificial neural network formulation and train SNNs by gradients from equivalent mappings to ensure descent directions. But they fail to achieve low latency and are also not online. In this work, we propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. Meanwhile, we theoretically analyze and prove that gradients of OTTT can provide a similar descent direction for optimization as gradients based on spike representations under both feedforward and recurrent conditions. OTTT only requires constant training memory costs agnostic to time steps, avoiding the significant memory costs of BPTT for GPU training. Furthermore, the update rule of OTTT is in the form of three-factor Hebbian learning, which could pave a path for online on-chip learning. With OTTT, it is the first time that two mainstream supervised SNN training methods, BPTT with SG and spike representation-based training, are connected, and meanwhile in a biologically plausible form. Experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS demonstrate the superior performance of our method on large-scale static and neuromorphic datasets in small time steps.

1 Introduction

Supervised SNN training must contend with discrete, non-differentiable spike generation. Existing methods trade low latency against memory, theoretical clarity, and online learning; OTTT is proposed to address these concerns.

1 Introduction

  • BPTT with surrogate gradients achieves high performance at low latency, but uses time-proportional training memory and lacks a clear theoretical optimization guarantee.Spike-representation methods offer clearer descent directions but require more time steps and are also inconsistent with biological online learning.Source: §1 · p. 2
  • OTTT decouples temporal dependencies by tracking presynaptic activities and using instantaneous losses, enabling forward-in-time learning derived from BPTT with surrogate gradients.The method is proposed to maintain online learning while targeting high performance and low latency.Source: §1 · p. 2
  • OTTT gradients are shown to provide a similar descent direction to spike-representation gradients under feedforward and recurrent conditions.For recurrent networks, the analysis uses convergent weighted firing rates and implicit differentiation.Source: §1 · p. 2
  • OTTT takes the form of a three-factor Hebbian rule, connecting BPTT with surrogate gradients, spike-representation methods, and biological learning rules.The paper presents this form as a path toward online learning on neuromorphic chips.Source: §1 · p. 2
  • Experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS report superior results on large-scale static and neuromorphic datasets with few time steps.The contribution summary names these four evaluation datasets.Source: §1 · p. 2

2 Related Work

Prior supervised SNN training methods fall mainly into spike-representation-based approaches and direct-gradient approaches. Online SNN methods also exist, but the cited approaches have theoretical or scalability constraints.

SNN Training Methods

  • Spike-representation methods use ANN-like mappings to convert trained ANNs or optimize SNNs from equivalent gradients, but generally require more time steps and may consume more energy with rate-based representations.Feedback SNNs may instead use equilibrium-state fixed-point equations.Source: §2 · p. 3
  • BPTT with surrogate gradients can achieve very low latency, but needs substantial memory and lacks a theoretical explanation for why surrogate gradients work well.Spiking-time gradient methods face a dead-neuron problem and are combined with surrogate gradients in practice.Source: §2 · p. 3

Online Training of Neural Networks

  • Existing online SNN approaches have constraints including feedback alignment, single-layer scope, large eligibility-trace memory, or specialized neuron and regularization requirements.The cited approaches also lack theoretical explanations for optimization or do not consider large tasks.Source: §2 · p. 3
  • The paper distinguishes its approach by explaining descent directions for feedforward and recurrent conditions with convergent inputs, while targeting large-scale tasks including ImageNet.The related-work passage contrasts this with online methods that cannot scale to large-scale tasks.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe the leaky integrate-and-fire neuron and two prior SNN training approaches: spike-representation gradients and BPTT with surrogate gradients.

3.1 Spiking Neural Networks

  • The paper uses a leaky integrate-and-fire neuron whose membrane potential integrates input current and generates a spike at threshold, followed by reset to resting potential.The discrete model uses a leak term and implements reset by subtraction.Source: §3.1 · p. 3

3.2 Previous SNN Training Methods

  • Spike-representation methods use weighted firing rates and weighted average inputs to establish ANN-like mappings between feedforward layers and calculate gradients.The mapping is described after time T and uses a clamp function in discrete settings or a ReLU function in continuous settings.Source: §3.2 · p. 4
  • For recurrent networks, convergent weighted average inputs yield an equilibrium representation whose gradients can be calculated by implicit differentiation.The passage describes a fixed-point equation for a single-layer network with contractive recurrent connections.Source: §3.2 · p. 4
  • BPTT unfolds the iterative SNN update over time and backpropagates through its computational graph, replacing non-differentiable spike derivatives with surrogate derivatives.The cited examples include rectangular and sigmoid derivatives.Source: §3.2 · p. 4

4 Online Training Through Time for SNNs

OTTT decouples temporal dependencies to calculate gradients online, then relates those gradients to spike-representation methods and three-factor Hebbian learning. The section also describes implementation choices for online or accumulated updates and for normalization without batch normalization.

4.1 Derivation of Online Training Through Time

  • Tracking presynaptic activity and using instantaneous loss lets OTTT calculate gradients at each time step without retaining BPTT’s unfolded graph, yielding constant training memory independent of time steps.Unlike BPTT, OTTT’s instantaneous gradients omit future influence on earlier states; the authors note equivalence with BPTT holds only for the last layer.Source: §4.1 · p. 5

4.2 Connection with Spike Representation-Based Methods for Descent Directions

  • Under the stated assumptions, OTTT gradients provide a descent direction aligned with spike-representation gradients in feedforward and recurrent settings with convergent inputs.The recurrent analysis considers contractive connections; the feedforward result assumes small errors between intermediate and final weighted firing rates.Source: §4.2 · pp. 6–7

4.3 Connection with Three-factor Hebbian Learning Rule

  • OTTT’s weight update has three factors: tracked presynaptic activity, a postsynaptic surrogate derivative, and a global error signal.The error signal can travel through an error-feedback path alongside feedforward propagation, and the analysis also covers delayed signals.Source: §4.3 · p. 7

4.4 Implementation Details

  • The implementation can update weights at every time step (OTTTO) or accumulate gradients over T steps before updating (OTTTA).OTTTO assumes each online update is small enough to have negligible effect on the following calculation.Source: §4.4 · p. 7
  • To avoid batch normalization, which requires statistics across time steps and prevents online gradients, the implementation uses scaled weight standardization for VGG and NF-ResNet.The authors also associate batch normalization with large memory costs.Source: §4.4 · p. 7

5 Experiments

Experiments evaluate OTTT on CIFAR-10, CIFAR-100, ImageNet, and neuromorphic datasets using VGG or NF-ResNet-34 architectures.

5.1 Comparison of Training Memory Costs

  • At 6 time steps, OTTT reduces training memory costs by 2–3×, while its memory remains constant as time steps increase.BPTT memory grows linearly with time steps; constant memory may also permit larger batch sizes with the same computational resources.Source: §5.1 · p. 8, Figure 2 · p. 8

5.2 Comparison of Performance

  • OTTT achieves higher performance than BPTT under the same settings and promising performance against representative conversion and direct-training methods.Reported SNN–ANN performance gaps are around 0.7% on CIFAR-10 and 2.08% on CIFAR-100; six-time-step results are competitive. With a sigmoid-like surrogate derivative, OTTTA and OTTTO achieve 93.58% and 93.73% test accuracy, respectively, under the same random seed.Source: §5.2 · p. 8, Table 1 · p. 9

5.3 Effectiveness for Recurrence

  • On CIFAR-100, VGG-F with feedback connections achieves higher performance than VGG, and OTTT training remains valid for the recurrent network.The reported improvement from feedback connections is more significant for OTTT than for BPTT.Source: §5.3 · p. 9, Table 3 · p. 10

5.4 Effectiveness for Training with Batch Size 1

  • Training with batch size 1 for 20 epochs remains valid, indicating potential for full online training with OTTT.The comparison uses the VGG network on CIFAR-10 and the same random seed as the default batch-size-128 setting.Source: §5.4 · p. 9, Table 4 · p. 10

5.5 Influence of Inference Time Steps

  • On ImageNet, a model trained with 6 time steps achieves higher performance when evaluated with more inference time steps.The study examines the influence of inference time steps on the trained model.Source: §5.5 · p. 10, Figure 3 · p. 10

5.6 Firing Rate Statistics

  • OTTT-trained models fire more in early layers and less in later layers than BPTT-trained models, with similar total synaptic operations.Overall firing rate is around 0.19; at 6 time steps, each neuron generates an average of 1.1 spikes, and synaptic operations are about 1.98 × 10^8 versus 1.93 × 10^8.Source: §5.6 · p. 10

6 Conclusion

OTTT decouples temporal dependencies through tracked presynaptic activities, with constant training memory independent of time steps. The paper connects its gradients to spike-representation methods and reports strong performance on large-scale static and neuromorphic datasets at low latency.

6 Conclusion

  • OTTT decouples temporal dependencies by tracking presynaptic activities, requiring constant training memory independent of time steps.This avoids BPTT's large training-memory costs.Source: §6 · p. 10
  • The authors connect OTTT gradients with spike-representation gradients and prove a descent guarantee under feedforward and recurrent conditions.They also describe OTTT as a three-factor Hebbian rule linking BPTT with surrogate gradients, spike-representation methods, and biological learning rules.Source: §6 · p. 10
  • Experiments demonstrate superior performance on large-scale static and neuromorphic datasets in a small number of time steps.Figure 4 presents average firing rates for OTTT- and BPTT-trained models on CIFAR-10.Source: §6 · p. 10, Figure 4 · p. 10

Checklist

The checklist reports that the paper addresses theoretical assumptions and proofs, experimental reproducibility, and error bars. It also clarifies how discrete time steps are notated across network layers.

Checklist

  • The checklist reports that the paper states assumptions for theoretical results and includes complete proofs.It directs readers to Section 4 for assumptions and Appendix A for proofs.Source: Checklist · p. 14
  • The checklist reports that code, data, and instructions for reproducing the main experimental results are available in supplementary materials.It also points to specified training details.Source: Checklist · p. 14
  • The checklist reports error bars and compute-resource details, with references to Section 5 and Appendix C.Source: Checklist · p. 14

A.2 A few notes for the notation of time for multi-layer networks

  • Layerwise discrete time indices simplify notation, while actual times may differ by synaptic delay across layers.The notation uses 0, 1, …, T for each layer; a receiving layer's signal time may be shifted by td.Source: §A.2 · p. 15

A.3 Proof of Theorem 1

Theorem 1's proof is presented under Assumption 1 and a small-error condition. The remarks qualify the gradient comparison for biases and for errors in spike-representation gradients.

A.3 Proof of Theorem 1

  • Theorem 1 assumes Assumption 1, Vth = 1, and small errors ϵl[t] = al[t] − al[T]; the remark says remaining membrane potential at the last time step causes bounded random error.Source: §A.3 · pp. 15–16
  • For biases, OTTT gradients omit presynaptic activities and, under Assumption 1, match spike-representation gradients up to a constant scaling factor.Source: §A.3 · p. 16
  • Spike-representation gradients may contain small errors because SNN computation differs from equivalent ANN-like mappings; larger time steps may improve their accuracy.The authors state that their overall descent-direction comparison tolerates small errors, which can be viewed as stochastic-optimization randomness.Source: §A.3 · p. 16

A.4 Proof of Theorem 2

The appendix extends the descent result to recurrent multi-layer settings, gives an OTTT training iteration, and describes normalization choices for the experiments.

A.4 Proof of Theorem 2

  • The descent conclusion is generalized from a single-layer condition to multi-layer networks with feedback, including arbitrary feedback connections represented as one larger structured weight matrix.The example assumes a contractive function and small errors for each layer.Source: §A.4 · p. 17

B Pseudocode of the OTTT algorithm

  • One OTTT iteration tracks presynaptic activity during the forward pass, computes instantaneous errors and gradients, then either updates parameters online or accumulates gradients.Source: B Pseudocode of the OTTT algorithm · p. 18

C.1 Scaled Weight Standardization and NF-ResNets

  • For SNNs, the authors set the sWS scale to approximately 2.74 to preserve signal variance after the Heaviside step function when Vth = 1.They apply sWS to all VGG weights and use the NF-ResNet structure described in [54].Source: §C.1 · p. 18
  • NF-ResNets omit batch normalization and use residual-branch scaling and input division by βl to maintain identity variance.The implementation follows [54], replaces activations with spiking neurons, and uses α = 0.2.Source: §C.1 · p. 19

C.2.1 Datasets

The experiments cover static-image and neuromorphic datasets, with dataset-specific preprocessing and training settings. Supplementary results report ImageNet firing rates and compare OTTT with BPTT when feedback connections are added.

C.2.1 Datasets

  • Experiments span CIFAR-10, CIFAR-100, ImageNet-1K, DVS-CIFAR10, and DVS128-Gesture, covering static images and neuromorphic data.The datasets include image classification benchmarks and event-based samples recorded or converted using dynamic vision sensors.Source: §C.2.1 · p. 19

C.2.2 Training Hyperparameters

  • Training settings vary by dataset: the CIFAR and DVS-CIFAR10 models use SGD, ImageNet uses SGD, and DVS128-Gesture uses Adam.The passage reports dataset-specific epochs, batch sizes, learning-rate schedules, and loss settings; implementation uses PyTorch on one RTX 3090 GPU.Source: §C.2.2 · pp. 19–20

D.1 Firing Rate Statistics on ImageNet

  • With six time steps, ImageNet neurons average 1.46 spikes; reducing to two steps yields 0.48 spikes and about a 2.5% accuracy drop.The reported overall firing rate is around 0.24.Source: §D.1 · p. 20, Figure 5 · p. 20

D.2 Comparison between OTTT and BPTT with Feedback Connections

  • Feedback connections improve both OTTT and BPTT on CIFAR-100, with a larger improvement for OTTT.The comparison covers VGG and VGG-F models.Source: §D.2 · p. 20, Table 5 · p. 20

D.3 Experiments on Fully Recurrent Structures

On Fashion-MNIST, the fully recurrent network results for OTTT and BPTT are similar, with BPTT slightly better.

D.3 Experiments on Fully Recurrent Structures

  • For the recurrent Fashion-MNIST model, OTTT and BPTT perform similarly, while BPTT is slightly better.The setup uses 784-dimensional flattened inputs, 400 recurrent spiking neurons, and a classification readout.Source: §D.3 · p. 21, Table 6 · p. 21

E Discussion of Limitations and Social Impacts

The paper identifies a limitation in applying some network techniques, including temporal batch normalization, while prioritizing online SNN training. It reports no direct negative social impact from its focus on training methods.

E Discussion of Limitations and Social Impacts

  • Focusing on online SNN training limits use of techniques such as batch normalization along the temporal dimension.Scaled weight standardization is used as an alternative, and may need additional regularization to match batch-normalization performance in ANNs.Source: E Discussion of Limitations and Social… · p. 21
  • The authors report no direct negative social impact because the work focuses only on SNN training methods.They suggest successful energy-efficient SNNs could broaden applications and alleviate ANNs’ energy consumption.Source: E Discussion of Limitations and Social… · p. 21

Related papers

Loading 2210.04195v2…