PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN

Yangfan Hu, Qian Zheng, Xudong Jiang, Gang Pan

arXiv:2305.19868v1cs.NE

TL;DR

Deep SNN training and ANN-to-SNN conversion face challenges from discontinuous spike functions and conversion errors that can impair performance. Fast-SNN transfers quantization-error minimization to quantized-ANN training and addresses sequential error during conversion. It reports state-of-the-art performance at low latency across three computer-vision tasks.

  • Problem

    Discontinuous spike functions challenge deep SNN training, while conversion-related quantization and accumulating errors are addressed as sources of performance degradation.

    Source: §1 · p. 1, §7 · p. 15

  • Method

    Fast-SNN maps temporal SNN quantization to spatial ANN quantization, trains quantized ANNs to minimize quantization error, and uses signed IF neurons and layer-wise fine-tuning to address sequential error.

    Source: §1 · p. 2, §7 · p. 15

  • Results

    Fast-SNN reports state-of-the-art performance with low latency across image classification, object detection, and semantic segmentation, including 71.31% accuracy at 3 time steps on ImageNet.

    Source: §1 · p. 2

  • Takeaways & Limitations

    At latency 3, the SNNs are about 10× more efficient than their counterpart ANNs in number of operations.

    Source: §4.6 · p. 11

Abstract

from arXiv · show

Spiking neural networks (SNNs) have shown advantages in computation and energy efficiency over traditional artificial neural networks (ANNs) thanks to their event-driven representations. SNNs also replace weight multiplications in ANNs with additions, which are more energy-efficient and less computationally intensive. However, it remains a challenge to train deep SNNs due to the discrete spike function. A popular approach to circumvent this challenge is ANN-to-SNN conversion. However, due to the quantization error and accumulating error, it often requires lots of time steps (high inference latency) to achieve high performance, which negates SNN's advantages. To this end, this paper proposes Fast-SNN that achieves high performance with low latency. We demonstrate the equivalent mapping between temporal quantization in SNNs and spatial quantization in ANNs, based on which the minimization of the quantization error is transferred to quantized ANN training. With the minimization of the quantization error, we show that the sequential error is the primary cause of the accumulating error, which is addressed by introducing a signed IF neuron model and a layer-wise fine-tuning mechanism. Our method achieves state-of-the-art performance and low latency on various computer vision tasks, including image classification, object detection, and semantic segmentation. Codes are available at: https://github.com/yangfan-hu/Fast-SNN.

1 INTRODUCTION

Fast-SNN targets ANN-comparable performance at low inference latency by reducing quantization and accumulating errors in ANN-to-SNN conversion. It reports results across image classification, object detection, and semantic segmentation.

1 INTRODUCTION

  • SNNs use event-driven spikes and accumulate operations, which can reduce energy consumption and make their operations less costly than ANN multiply-and-accumulate operations.Neuromorphic platforms with co-located memory and computation can perform SNN inference with low power consumption.Source: §1 · p. 1
  • Direct SNN training is difficult because spike functions are discontinuous, while surrogate-gradient BPTT can be computationally inefficient and less effective for deep, complex tasks.ANN-to-SNN conversion reuses ANN training, but quantization and accumulating errors can degrade performance at short latencies; more time steps increase operations and running time.Source: §1 · pp. 1–2
  • Fast-SNN maps temporal SNN quantization to spatial ANN quantization, minimizes quantization error through supervised quantized-ANN training, and addresses sequential error with signed IF neurons and layer-wise fine-tuning.The quantized-ANN training also determines clipping ranges and weight and activation distributions for each layer.Source: §1 · p. 2
  • Accuracy 71.31% at 3 time steps on ImageNet, mAP 73.43% at 7 on PASCAL VOC 2007, and mIoU 69.7% at 15 on PASCAL VOC 2012.The paper reports state-of-the-art performance and low latency across these computer-vision tasks.Source: §1 · p. 2

2 RELATED WORK

Related work covers direct SNN training and ANN-to-SNN conversion. The paper distinguishes its approach by optimizing quantization during quantized-ANN training and addressing sequential error in conversion.

2.1 Direct Training with Surrogate Gradients

  • Surrogate-gradient methods approximate the spike function's derivative and use BPTT; spike-train sparsity makes training costly in computation and memory, and gradients can hinder deep networks.The literature includes spatio-temporal backpropagation, normalization, temporal loss, and methods targeting inter- and intra-neuron dependencies.Source: §2.1 · p. 2

2.2 ANN-to-SNN Conversion

  • The conversion framework maps ReLU activations to IF neurons, while quantization error from temporal discretization and accumulated layerwise errors can impair deep networks at low latency.Prior normalization methods adjust clipping ranges but leave weight and activation distributions unoptimized.Source: §2.2 · pp. 2–3
  • Prior approaches to accumulating error include residual architectures, statistically increasing deeper-layer firing rates, and capped-ReLU training with normalized firing thresholds.The cited text reports lower observed accumulating error for residual than plain architectures.Source: §2.2 · p. 3
  • Other methods use fixed-step quantization, temporal-quantization knowledge, layer-wise calibration, or tandem fine-tuning; the paper contrasts these with its supervised optimization of clipping range and weight and activation distributions.It characterizes methods,, and as locally optimizing distributions after FP32 training, while its quantized-ANN training provides a global optimum.Source: §2.2 · p. 3
  • The paper evaluates its conversion approach on image classification, object detection, and semantic segmentation, extending beyond prior SNN work that primarily focused on image classification.The cited passage describes this as a promising solution for converting deep ANN models to SNN counterparts.Source: §2.2 · p. 4

3 FAST-SNN

Fast-SNN converts quantized ANNs by mapping their activation values to SNN firing rates, then reduces conversion errors with a signed IF neuron and layer-wise fine-tuning. Its design targets low latency by minimizing quantization error during ANN training and addressing sequential error across SNN layers.

3 FAST-SNN

  • The framework links spatial quantization in ANNs to temporal quantization in SNNs, then addresses sequential error with a signed IF neuron and layer-wise fine-tuning.The paper describes sequential error as the primary cause of accumulating error after quantization error is minimized.Source: §3 · p. 4

3.1 Activation Equivalence between ANNs and SNNs

  • The mapping makes quantized ANN activations equivalent to SNN firing rates, with spike-train length set to T = 2^b − 1.The latency bound depends on both bit precision b and network depth L.Source: §3.1 · pp. 4–5

3.2 Quantization Error Minimization

  • Quantization-aware ANN training minimizes conversion quantization error by optimizing network parameters and learning the activation clipping threshold.The authors contrast supervised threshold learning with statistical post-processing based on activation percentiles.Source: §3.2 · p. 5

3.3 Accumulating Error Minimization

  • The signed IF neuron can emit negative spikes to cancel wrongly fired positive spikes and restore membrane potential.The model uses a small negative firing threshold, empirically set to -1e-3.Source: §3.3 · p. 6
  • Layer-wise fine-tuning minimizes Euclidean distance between SNN firing rates and quantized ANN activations using a proxy ANN that shares SNN parameters.It starts at the second layer and bypasses the final classification layer, which uses membrane potential directly.Source: §3.3 · p. 7, Figure 5 · p. 7
  • Compared with progressive tandem learning, the end-to-end fine-tuning mechanism requires less computation time and cost, and it incorporates bias terms.The bias term is used to learn a compensating membrane potential.Source: §3.3 · p. 7

3.4 Implementation Details

  • The experiments use uniform activation quantization and additive powers-of-two weight quantization when exploring low-precision SNNs.The authors report 93.66% accuracy for a 3-bit ResNet-56 on CIFAR-10 and 71.91% for a 2-bit-activation VGG-16 on ImageNet.Source: §3.4 · pp. 7–8

4 EXPERIMENTS ON IMAGE CLASSIFICATION

The experiments test Fast-SNN on CIFAR-10 and ImageNet, comparing classification accuracy and operation counts across converted models. They evaluate quantization-error and accumulating-error remedies, then assess computational efficiency and low-precision deployment relevance.

4.1 Experimental Setup

  • Setup spans CIFAR-10 and ImageNet, evaluating quantized AlexNet, VGG, and ResNet variants and counting SNN synaptic operations over time.Some models retain full-precision weights for comparison with full-precision SNNs.Source: §4.1 · p. 8

4.2 Overall Performance

  • On CIFAR-10 and ImageNet, Fast-SNN matches strong classification baselines at fewer time steps, with gains over latency baselines and small ANN accuracy gaps.The comparisons include AlexNet, VGG, and ResNet models.Source: §4.2 · p. 9, Table 5 · p. 9

4.3 Evaluation for Minimizing the Quantization Error

  • Learned firing thresholds outperform fixed alternatives, and latency (2b−1)×L yields the same accuracy as pretrained ANNs when sequential error is excluded.The comparisons include clipping threshold, maximum activation, and activation percentiles.Source: §4.3 · p. 10, Table 6 · p. 10

4.4 Evaluation for Minimizing the Accumulating Error

  • The signed IF neuron raises converted-network accuracy, while layer-wise fine-tuning makes performance nearly lossless versus quantized ANNs; its gain grows with depth and lower precision.For 2-bit ResNet-56, fine-tuning improves SNNβ accuracy by 8%.Source: §4.4 · pp. 10–11, Table 7 · p. 10, Table 8 · p. 10

4.5 Computational Efficiency

  • At latency 3, Fast-SNN's AlexNet/VGG-11 accuracies exceed PTL's latency-15 results by 1.75%/2.00%, with operation ratios 11.1%/24.8% versus 47.4%/74.9%.The operation ratio is the number of SNN operations divided by ANN operations.Source: §4.5 · p. 11, Figure 6 · p. 11

4.6 Discussion

  • Latency-3 SNNs are about 10× more operation-efficient than counterpart ANNs, and latency T gives L× faster running time than the bound in Eq. 11.Quantized weights also make the SNNs compatible with low-bit neuromorphic hardware.Source: §4.6 · p. 11

5 EXPERIMENTS ON OBJECT DETECTION

The object-detection experiments evaluate Fast-SNN with Tiny YOLO and YOLOv2(ResNet-34) on PASCAL VOC and MS COCO. They compare detection performance with prior SNN detectors and visualize results at different latencies.

5 EXPERIMENTS ON OBJECT DETECTION

  • Fast-SNN extends ANN-to-SNN conversion evaluation to object detection, where the cited existing Spiking-YOLO approach uses channel-wise normalization.The paper contrasts that context with its evaluation using the YOLO framework.Source: §5 · p. 12

5.1 Experimental Setup

  • Experiments evaluate Tiny YOLO and YOLOv2(ResNet-34) on PASCAL VOC and MS COCO, reporting mAP over 20 and 80 categories, respectively, at IoU = 0.5 for fair comparison with Spiking-YOLO.VOC testing uses the VOC 2007 test dataset.Source: §5.1 · p. 13

5.2 Overall Performance

  • On VOC 2007 and MS COCO 2017, Fast-SNN detectors outperform at substantially fewer time steps, including YOLOv2(ResNet-34) at latencies 3, 7, and 15.On VOC, YOLOv2 gains 24.61%/21.99%/17.13% mAP at latency 15/7/3; on COCO, it gains 20.62%/16.11%/7.67% mAP at those latencies. Visual results show latency-15 SNN detecting a vase missed by the full-precision ANN.Source: Table 9 · p. 12, Figure 7 · p. 12, §5.2 · p. 13

6 EXPERIMENTS ON SEMANTIC SEGMENTATION

The section evaluates Fast-SNN for semantic segmentation on PASCAL VOC 2012 and MS COCO 2017, reporting competitive performance at short latencies across VGG-9 and ResNet-34 + ASPP.

6 EXPERIMENTS ON SEMANTIC SEGMENTATION

  • Semantic segmentation is scarcely explored in SNNs; the section evaluates Fast-SNN in the Deeplab framework.The task predicts an object class for each pixel or assigns background status.Source: §6 · p. 13

6.1 Experimental Setup

  • Experiments use PASCAL VOC 2012 and MS COCO 2017, evaluating VGG-9 and ResNet-34 + ASPP with mean pixel IoU and 4/3/2-bit activations.The mean IoU averages over 21 classes on VOC and 81 on COCO.Source: §6.1 · pp. 13–14

6.2 Overall Performance

  • 31.14% mean IoU for VGG-9 on PASCAL VOC 2012 at latency 15 is 0.46% below its ANN; ResNet-34 + ASPP reaches 50.24% mean IoU on MS COCO 2017.These are the reported latency-15 comparisons with the corresponding ANN models.Source: Table 10 · p. 14, §6.2 · p. 15
  • Fig. 8 shows SNN segmentations close to full-precision ANN results, with some objects, such as TVs, segmented better by the SNN.The visual comparison uses ResNet-34 + ASPP on PASCAL VOC 2012 and MS COCO 2017.Source: Figure 8 · p. 14, §6.2 · p. 15
  • At latency 3, VGG-9 exceeds by 21.56% mean IoU using about 7× fewer time steps; latency-15 ResNet-34 + ASPP exceeds by 47.4% mean IoU.The comparisons with are reported on PASCAL VOC 2012.Source: §6.2 · pp. 14–15

7 CONCLUSION

Fast-SNN maps temporal SNN quantization to spatial ANN quantization, then addresses sequential error with signed IF neurons and layer-wise fine-tuning. The paper reports state-of-the-art performance at low latency across several vision tasks.

7 CONCLUSION

  • Fast-SNN maps temporal SNN quantization to spatial ANN quantization, minimizes quantization error through quantized ANN training, and addresses sequential error with signed IF neurons and layer-wise fine-tuning.The framework also derives an inference-latency upper bound for lossless conversion and is reported to achieve state-of-the-art performance at low latency on image classification, object detection, and semantic segmentation.Source: §7 · p. 15
Loading 2305.19868v1…