Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance

1 Introduction

• The paper develops a direct-training framework that avoids calculating neuron membrane potentials and addresses robustness through circuit evaluation of timing jitter and weight-quantized training.The introduction identifies both difficult training and robustness on noisy, resource-limited neuromorphic hardware as open challenges.Source: §1 · p. 1
• Closed-form spiking-neuron responses identify the nonleaky integrate-and-fire neuron with single-spike temporal coding as the preferred choice for direct training.This response analysis is presented as a way to make direct training easier.Source: §1 · p. 1
• Deep SpikingVGG16 and SpikingGoogleNet are trained on CIFAR-10 and ImageNet, with ImageNet direct training reported for the first time.The paper reports new benchmark results on these datasets.Source: §1 · p. 1
• The paper evaluates robustness through weight quantization and noise perturbation, and designs a phase-domain neuron circuit to assess energy efficiency and timing-jitter robustness.These evaluations target practical neuromorphic hardware conditions.Source: §1 · p. 1

2 Related Works

• Unsupervised STDP relies on local neuronal activity without a global supervisor and is reported to have low performance in deep networks (Caporale and Dan 2008; Diehl and Cook 2015; Kheradpisheh et al. 2018; Lee et al. 2018).The paper places STDP within unsupervised learning, one of three SNN training categories.Source: §2 · p. 1
• DNN-to-SNN translation has had the best SNN performance so far, but sacrifices spike sparsity and asynchronous processing and can make DNN operations inefficient to copy.The cited translation approaches map DNN values to SNN spike rates or spike times (Diehl et al. 2015; Rueckauer et al. 2017; Sengupta et al. 2019; Zhang et al. 2019).Source: §2 · p. 2
• Prior direct-training methods use spike-time losses, probability neurons, or gradients through responses and membrane dynamics, while existing approaches remain inefficient for large datasets such as ImageNet.The paper also notes low training performance linked to overly nonlinear neuron responses (Zhou and Li 2020).Source: §2 · p. 2
• Neuromorphic hardware studies report low energy use, including 45 pJ per spike and 185-fold efficiency over an FPGA-based DNN in one comparison (Cao, Chen, and Khosla 2015).Other reported examples include 26 pJ per spike for IBM TrueNorth and a 19 pJ-per-spike analog neuron circuit.Source: §2 · p. 2
• Noise, parameter drift, and memory limits can degrade hardware SNN performance; prior pruning and quantization work addressed shallow SNNs only (Esser et al. 2016; G¨oltz et al. 2019; Rathi, Panda, and Roy 2018).The paper frames robustness under these constraints as a hardware concern.Source: §2 · p. 2

3 Deep SNN with Easy and Direct Training

• The direct-train framework uses a closed-form layer response instead of calculating membrane potentials and trains weights that are reused in the hardware SNN.Backpropagation through the closed-form expression makes training similar to conventional DNN training.Source: §3 · p. 2, §3.3 · p. 4

3.1 Layer Response Models of Spiking Neurons

• For temporal coding, the authors prefer single-spike nonleaky IF neurons because their exponentially-decaying-waveform response is numerically stable, unlike the cited LIF response forms.Random weights can make LIF response expressions negative, complex, or undefined, preventing gradient updates.Source: §3.1 · p. 3

3.2 Circuit of Temporal-Coded Spiking Neuron

• The phase-domain circuit uses VCO feedback to realize an exponentially decaying synaptic kernel, then accumulates the frequency difference until a threshold triggers an output spike.The paper introduces phase-domain signal processing to SNN design and describes its advantages over voltage-domain processing.Source: §3.2 · pp. 3–4
• <10 pJ/spike in 65nm and a 45% energy-efficiency gain over Zhou et al. 2020 are reported for the proposed neuron circuit.The authors say energy consumption will reduce further with technology scaling.Source: §3.2 · p. 4

3.3 Direct-Train Framework for Deep SNNs

• The software algorithm sorts input spike times, computes candidate responses, and selects the first hardware-compatible output time; hardware spike ordering makes this algorithm unnecessary during inference.The authors report that the algorithm improves computation speed and convergence for deep-SNN training.Source: §3.3 · p. 4
• The framework maps max- and average-pooling, batch normalization, and local response normalization to temporal-coded SNN operations.The LRN implementation uses average pooling followed by neurons that implement the remaining operations.Source: §3.3 · p. 5
• Direct training uses real SNN weights without weight normalization or structural changes and can adjust threshold to promote sparsity; only neurons in C need to spike.The authors distinguish this neuron sparsity from the spike-count sparsity used in translated SNNs.Source: §3.3 · p. 5

4.1 Classification Accuracy

• 99.33% accuracy made the MNIST model the highest-accuracy SNN listed, with 21K trainable weights and sparsity 0.94.Its reported energy consumption was 205 nJ.Source: §4.1 · p. 5, Table 2 · p. 6
• 92.68% testing accuracy made SpikingVGG16 a new SNN record, higher than listed SNNs and CNN-based VGG16.It had sparsity 0.62 and consumed 0.336 mJ per image inference.Source: §4.1 · p. 5, Table 2 · p. 6
• 68.8% Top-1 testing accuracy for SpikingGoogleNet was only 1.0% lower than CNN-based GoogleNet on ImageNet.VGG16-based SNNs performed slightly better but had much larger network sizes.Source: §4.1 · p. 6, Table 2 · p. 6

4.2 Robustness

• Weight quantization caused worst accuracy losses of 0.22%, 1.75%, and 8.8% on MNIST, CIFAR-10, and ImageNet, respectively.The authors recommend at least 4-bit weights for larger networks.Source: §4.2 · p. 7, Table 4 · p. 7
• At quantization SNRs of 24 dB and above, accuracy reduction was negligible; the PLL low-pass-filtered input jitter, with output jitter ranging from 0.024% to 0.997%.For the 2-input case, 4-bit quantization increased output jitter from 0.004% at 32-bit quantization to 0.957%.Source: §4.2 · p. 7

5 Conclusion

• The proposed direct-train deep SNNs are described as easy to train and achieve accuracy within 1% of, or sometimes better than, similar DNNs.The conclusion also reports robustness to timing jitter, weight quantization, and noise, and a 45% energy-efficiency gain for the phase-domain neuron circuit over prior work.Source: §5 · p. 7

A.1 General Solution to Membrane Potential

• For rate coding, the accumulated membrane potential at T is used to define spike rate when the potential resets to zero after each spike.Assuming regularly distributed spikes, the first spike time can also be used to calculate the spiking rate.Source: §A.1 · p. 10
• The membrane-potential expression in (13) is valid for IF neurons but not LIF neurons because leakage affects reset and non-reset potentials differently.The passage states that applying (13) to derive LIF gradients is inaccurate.Source: §A.1 · p. 10
• For temporal coding, single-spike TTFS makes the first arriving spike the most significant, with each neuron emitting one spike before resetting.With k = 0, the second P term can be skipped.Source: §A.1 · p. 10
• Impulse, Heaviside, and exponential spike waveforms are presented as simple forms that can yield closed-form layer responses.The passage contrasts their use in TrueNorth and Loihi and says more complex waveforms usually lack such closed-form solutions.Source: §A.1 · p. 10

A.2 Layer Response of Rate-Coded Neuron

• Rate-coded response (18) is nearly identical to a DNN layer response, but its integer-spike/time and no-potential-loss assumptions introduce modeling errors that can accumulate in deep SNNs.Source: §A.2 · pp. 10–11
• The rate-coded Heaviside derivation assumes evenly distributed input spikes, and the rate-based expressions assume integer spike counts or rates.Source: §A.2 · pp. 11–12
• Rate-coded training formulas can fail when random weights make a logarithm undefined or make the response extremely sensitive, negative, or saturated.Negative values can stop training, while saturation can leave training stuck.Source: §A.2 · p. 11

A.3 Layer Response of Temporal-Coded Neuron

• Temporal coding uses a single-spike neuron, and the output spike time depends on input neurons that fire before it.The set C includes only inputs whose spike times precede the output spike time; the response can be written in standard DNN layer-response form.Source: §A.3 · p. 12
• Impulse spikes are excluded from temporal coding because repeated equal input and output spike times would reduce timing diversity as layers increase.Source: §A.3 · p. 12
• For temporal coding, a nonleaky IF neuron with an exponentially decaying spike has the same layer response as an LIF neuron with a Heaviside spike.The paper identifies the two formulations as equivalent.Source: §A.3 · p. 12
• For temporal-coded LIF neurons with exponentially decaying spikes, the derived response equations can yield negative or complex values under random weights, preventing gradient updates.Source: §A.3 · p. 13

• In the illustrated output-layer example, the sixth neuron fired first and the image was classified as digit 6.Source: Figure 5 · p. 13, §B.1 · p. 13
• Input pixels were encoded as spike times using ti = α(−pi + 1), with α = 3 and normalized pixels pi ∈[0, 1].Source: §B.1 · p. 13
• 99.33% accuracy with noisy input and 98.82% with non-noisy input were reported on MNIST.Both models' loss converged to near zero after several training epochs.Source: §B.1 · p. 13

• The three CIFAR-10 models had 73K, 11 million, and 33 million weights; data augmentation was used for the medium and large models, and their reported energy consumptions were 0.112 mJ and 0.336 mJ, respectively.The small model consumed 731 nJ.Source: §B.2 · pp. 13–14
• The small model, trained without data augmentation, reached 62.24% recognition accuracy at epoch 20; its accuracy did not improve much thereafter, though loss tended to level off after epoch 75.Source: §B.2 · p. 13
• 80.49% accuracy was reported for the medium model, compared with 82.95% for the translation-based SNN, which had 39 million weights versus the medium model's 11 million.Source: §B.2 · p. 14
• The medium model's visualization maps spike times to feature-map colors, with dark pixels indicating large spike times or no spike; the illustrated output classified the image as a frog.Source: §B.2 · p. 14

• 63.6% Top-1 accuracy was reported for the randomly selected ImageNet sample.Source: §B.3 · p. 14
• 84.1% Top-5 accuracy was reported for the same ImageNet sample.Source: §B.3 · p. 14
• The layer-activity visualization contains many dark-colored pixels, indicating a high degree of sparsity.Source: §B.3 · p. 14

B.4 Extra Results of Weight Quantization

• Figures 8 and 9 visualize spiking activity on a CIFAR-10 sample and classification results for an ImageNet sample, respectively.Source: Figure 8 · p. 15, Figure 9 · p. 15
• Retraining proceeds from 32-bit to 8-bit, 4-bit, then 2-bit weights, restoring low loss after each quantization step.Reducing precision initially increases loss; Figure 12 traces the four-stage training loss for the MNIST model.Source: §B.4 · pp. 14–15
• 2-bit quantization reduced accuracy by at most 0.18% on MNIST and no more than 1% on each of three CIFAR-10 models; one augmented large model reached 90.93%.Table 5 covers classification accuracy versus weight quantization; the 90.93% result is reported for the large CIFAR-10 model with data augmentation in Table 4.Source: Table 5 · p. 15, §B.4 · p. 15

B.5 Extra Results of Neuron Robustness to Random Hardware Mismatch

• 3% timing jitter corresponds to 30 dB SNR, while random mismatch contributes more output-spike jitter than weight quantization or device noise.Source: §B.5 · p. 15
• Reducing random mismatch by enlarging transistors or using high-threshold-voltage transistors increases area or reduces SNN speed, respectively.The paper identifies a trade-off among accuracy, area, and speed.Source: §B.5 · p. 15
• VCO center frequency varied −4% to +3%, tuning gain varied ±4%, and output-spike timing shifted by ±3% under simulated random mismatch.Figure 10 shows the distributions from 100 Monte Carlo runs; the smaller output-spike shift is attributed to the PLL making VCO2 track VCO1.Source: Figure 10 · p. 15, §B.5 · p. 15