PaperlayerView exact revision on arXiv

Source-linked AI summary

DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks

Nitin Rathi, Kaushik Roy

arXiv:2008.03658v3cs.NEcs.LGstat.ML

TL;DR

Deep SNNs face high inference latency from inefficient input encoding and sub-optimal neuron parameters. DIET-SNN directly applies image pixels and jointly learns layerwise membrane leak, firing threshold, and weights, achieving competitive accuracy with low latency and improved energy efficiency. Across CIFAR and ImageNet experiments on VGG and ResNet, it reaches 5-timestep inference and substantially lower compute energy than comparable baselines.

  • Problem

    Deep SNNs are limited by inefficient input encoding and sub-optimal membrane-leak and firing-threshold settings, contributing to high inference latency.

  • Method

    DIET-SNN directly applies image pixels and jointly trains layerwise membrane leak, firing threshold, and weights, with the first convolutional layer generating spikes.

  • Results

    DIET-SNN achieves similar accuracy to ANN models with 6−18× less compute energy and 5 timesteps versus 100−2000 timesteps for state-of-the-art SNNs.

  • Takeaways & Limitations

    The combination of reduced latency and high activation sparsity improves DIET-SNN’s computational efficiency across CIFAR and ImageNet evaluations on VGG and ResNet architectures.

  • Takeaways & Limitations

    Inference latency in wall-clock time depends on the number of computations and the underlying hardware.

Abstract

from arXiv · show

Bio-inspired spiking neural networks (SNNs), operating with asynchronous binary signals (or spikes) distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. The state-of-the-art SNNs suffer from high inference latency, resulting from inefficient input encoding, and sub-optimal settings of the neuron parameters (firing threshold, and membrane leak). We propose DIET-SNN, a low-latency deep spiking network that is trained with gradient descent to optimize the membrane leak and the firing threshold along with other network parameters (weights). The membrane leak and threshold for each layer of the SNN are optimized with end-to-end backpropagation to achieve competitive accuracy at reduced latency. The analog pixel values of an image are directly applied to the input layer of DIET-SNN without the need to convert to spike-train. The first convolutional layer is trained to convert inputs into spikes where leaky-integrate-and-fire (LIF) neurons integrate the weighted inputs and generate an output spike when the membrane potential crosses the trained firing threshold. The trained membrane leak controls the flow of input information and attenuates irrelevant inputs to increase the activation sparsity in the convolutional and dense layers of the network. The reduced latency combined with high activation sparsity provides large improvements in computational efficiency. We evaluate DIET-SNN on image classification tasks from CIFAR and ImageNet datasets on VGG and ResNet architectures. We achieve top-1 accuracy of 69% with 5 timesteps (inference latency) on the ImageNet dataset with 12x less compute energy than an equivalent standard ANN. Additionally, DIET-SNN performs 20-500x faster inference compared to other state-of-the-art SNN models.

1 INTRODUCTION

DIET-SNN addresses inefficient input encoding and sub-optimal neuron parameters in deep SNNs by jointly learning membrane leak, firing threshold, and weights. It directly applies image pixels, trains the first convolutional layer as a spike generator, and reports low-latency, energy-efficient evaluation.

  • SNNs use binary signals distributed over time, motivating their potential for power-efficient computing on event-driven hardware.
  • Analog inputs require encoding into spikes, with rate, temporal, and rank-order coding among the available approaches.
  • DIET-SNN jointly optimizes each layer’s membrane leak, firing threshold, and network weights through gradient-based error backpropagation.The approach targets high accuracy at low inference latency while improving activation sparsity and energy efficiency.
  • The first convolutional layer becomes a learned spike generator whose spike rate depends on weights, membrane leak, and threshold, eliminating a separate generator function.
  • A timestep’s wall-clock duration depends on the number of computations and the underlying hardware.
  • 5 timesteps yields similar accuracy to ANN models with 6−18× less compute energy, versus 100−2000 timesteps for state-of-the-art SNN models.The evaluation uses VGG and ResNet architectures on CIFAR and ImageNet datasets.

2 BACKGROUND AND RELATED WORK

Prior deep-SNN training methods include ANN-to-SNN conversion, spike-based backpropagation, and hybrid training. DIET-SNN adopts hybrid initialization while further optimizing neuron parameters and using direct input encoding.

  • Efficient learning for deep SNNs remains an ongoing research challenge, with prior methods including conversion, spike-based backpropagation, and bio-plausible feedback-alignment variants.Feedback-alignment methods have been limited to simple tasks, motivating focus on scalable conversion and backpropagation approaches.
  • ANN-SNN Conversion: ANN-to-SNN conversion trains a restricted ReLU ANN, then initializes an iso-architecture SNN with its weights and integrate-and-fire neurons.
  • Error Backpropagation in SNN: SNN backpropagation is difficult because integrate-and-fire and leaky-integrate-and-fire neurons lack continuous derivatives.The spike-function derivative is undefined at a spike and zero otherwise.
  • Hybrid SNN Training: Hybrid training combines ANN-SNN conversion with surrogate-gradient backpropagation and BPTT, achieving reported inference latency of 100−250 timesteps.The converted model is fine-tuned after initialization, but the method still employs rate coding.
  • DIET-SNN initializes from ANN-SNN conversion and then optimizes membrane leak and threshold, with pixel intensities directly applied during threshold computation.

3 ALGORITHM FOR TRAINING DIET-SNN

DIET-SNN trains spiking neurons and network weights jointly, using direct input encoding and layerwise membrane dynamics to support low-latency inference.

  • Direct Input Encoding: Direct input encoding applies image pixel intensities at every timestep, while the first LIF convolutional layer generates spikes from weighted inputs.Its spike rate depends on learned weights, membrane leak, and firing threshold.
  • Neuron Model: The LIF model uses membrane potential, leak, weighted input, binary output spikes, and firing threshold to update neuron states over time.Leak controls retained potential, while threshold governs average input integration time; spikes trigger a soft reset.
  • Output Layer: The output layer accumulates inputs without leakage or output spikes, and cross-entropy loss is computed from its final-timestep membrane potentials.Softmax is applied to the final membrane potential vector before calculating the loss.
  • Parameter Optimization: Each hidden layer shares one learned leak and threshold across its neurons, reducing trainable parameters without observed improvement from individual neuron parameters.The shared parameters are optimized alongside weights using backpropagation with surrogate gradients.
  • Parameter Optimization: The training equations update weights, thresholds, and membrane leaks through gradient-based optimization across the unrolled temporal network.The supplied training description includes separate threshold and leak updates in addition to the weight update.

4 EXPERIMENTS

The experiments compare DIET-SNN with other spiking models across architectures and datasets, examining accuracy, latency, and energy efficiency.

  • Comparisons: DIET-SNN is compared with other SNN models in the reported experimental tables.The supplied table caption identifies the comparison as a model-level evaluation.
  • Experimental Setup: The evaluation uses VGG and ResNet architectures on CIFAR and ImageNet classification datasets.The converted SNN is subsequently trained with error-backpropagation while optimizing weights, membrane leak, and firing thresholds.

5 ENERGY EFFICIENCY

DIET-SNN reduces compute energy by exploiting event-driven sparse spike operations, whose cost is lower than ANN multiply-accumulates. Its layerwise spike rates and optimized neuron parameters support energy savings while maintaining similar accuracy.

  • Compute model: Each SNN operation uses one floating-point addition on binary spikes, whereas each ANN operation uses one floating-point multiply-accumulate.Event-driven execution skips computation when no spikes are present.
  • Compute model: 4.6pJ versus 0.9pJ: a 32-bit ANN MAC costs 5.1× more energy than an SNN addition in 45nm CMOS.The values may vary across technologies, but addition is generally cheaper than multiplication.
  • Spike sparsity: A spike rate of 1 makes ANN and SNN operation counts equal, while lower spike rates indicate greater event sparsity and energy efficiency.Spike rate is total spikes in a layer across timesteps divided by the number of neurons.
  • Spike sparsity: Around 1.6 average spike rate for VGG16 indicates DIET-SNN is more energy-efficient than ANN, with deeper layers becoming sparser as leak decreases and threshold increases.The reported average spike rate is approximately 1.6 during inference.
  • Energy results: DIET-SNN achieves similar accuracy to ANN while reducing compute energy by 6–18× across CIFAR and ImageNet evaluations.ResNet consumes more energy than VGG because over 50% of its operations occur in the first three high-spike-rate layers.

6 EFFECT OF DIRECT INPUT ENCODING AND THRESHOLD/LEAK

The experiments compare input encoding and neuron-parameter choices using VGG16 on CIFAR10 at similar accuracy, while Figure 2 reports layerwise spike rates, leaks, and thresholds for VGG16 on CIFAR100.

  • Layerwise analysis: Figure 2(a) reports layerwise spike rate during inference, calculated as total spikes divided by the number of neurons.An average spike rate of 1.65 means each neuron fired 1.65 times per image over all timesteps.
  • Layerwise analysis: Figure 2(b) compares trained layerwise leak and threshold with pre-training values for VGG16 on CIFAR100.Before training, the leak is unity in every layer and the threshold comes from ANN-SNN conversion.
  • Experimental comparison: Four VGG16 SNN variants compare Poisson rate encoding, direct input encoding, threshold optimization, and joint threshold/leak optimization.The variants are trained to achieve similar accuracy on CIFAR10.

7 CONCLUSIONS

DIET-SNN combines direct pixel input, learned LIF leak and threshold parameters, and backpropagation to target low latency and sparse activations. The resulting network reduces compute energy and timesteps while retaining similar accuracy to comparable models.

  • Conclusions: DIET-SNN trains membrane leak, firing threshold, and network weights jointly with error-backpropagation to optimize accuracy and latency.The network is initialized from a trained ANN to accelerate spike-based training.
  • Conclusions: Directly applying image pixels lets the first convolutional layer perform spike generation, producing high activation sparsity in convolutional and dense layers.The first layer is trained to perform the spike-generation operation.
  • Conclusions: 6–18× lower compute energy than an equivalent ANN is achieved with similar accuracy.The reduction combines high sparsity with low inference latency.
  • Conclusions: 20–500× fewer timesteps are required than in other state-of-the-art SNN models at similar accuracy.The conclusion attributes this comparison to inference latency measured in timesteps.
Loading 2008.03658v3…