PaperlayerView exact revision on arXiv

Source-linked AI summary

Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences

Weihua He, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, Yuan Xie

arXiv:2005.02183v1cs.CVcs.NEeess.IV

TL;DR

The paper addresses the unclear relative performance of SNNs and RNNs on neuromorphic data by systematically comparing them on vision datasets. It standardizes training and architecture choices, then evaluates converted and captured datasets across temporal resolutions. The reported conclusions favor SNNs over common RNNs in accuracy and compactness, while rate-coding-inspired training can make LSTM comparable or slightly better.

  • Problem

    The study asks how SNNs and RNNs compare on neuromorphic data, where RNN evaluation remains limited despite their ability to model temporal dynamics.

    Source: §I · p. 1

  • Method

    It systematically compares feedforward SNNs, vanilla RNNs, and LSTM using unified datasets, temporal resolutions, BPTT-based learning, rate-coding-inspired losses, architectures, and training hyper-parameters.

    Source: §I · p. 2, §III · p. 3, §V · p. 12

  • Results

    SNNs usually achieve better accuracy than common RNNs, while the rate-coding-inspired loss can make LSTM comparable to or slightly better than SNNs.

    Source: §V · p. 12

  • Takeaways & Limitations

    The findings support selecting models according to workload: SNNs for small temporal resolution or compact models, and rate-coding-trained LSTM for large temporal resolution.

    Source: §V · p. 12

Abstract

from arXiv · show

Neuromorphic data, recording frameless spike events, have attracted considerable attention for the spatiotemporal information components and the event-driven processing fashion. Spiking neural networks (SNNs) represent a family of event-driven models with spatiotemporal dynamics for neuromorphic computing, which are widely benchmarked on neuromorphic data. Interestingly, researchers in the machine learning community can argue that recurrent (artificial) neural networks (RNNs) also have the capability to extract spatiotemporal features although they are not event-driven. Thus, the question of "what will happen if we benchmark these two kinds of models together on neuromorphic data" comes out but remains unclear. In this work, we make a systematic study to compare SNNs and RNNs on neuromorphic data, taking the vision datasets as a case study. First, we identify the similarities and differences between SNNs and RNNs (including the vanilla RNNs and LSTM) from the modeling and learning perspectives. To improve comparability and fairness, we unify the supervised learning algorithm based on backpropagation through time (BPTT), the loss function exploiting the outputs at all timesteps, the network structure with stacked fully-connected or convolutional layers, and the hyper-parameters during training. Especially, given the mainstream loss function used in RNNs, we modify it inspired by the rate coding scheme to approach that of SNNs. Furthermore, we tune the temporal resolution of datasets to test model robustness and generalization. At last, a series of contrast experiments are conducted on two types of neuromorphic datasets: DVS-converted (N-MNIST) and DVS-captured (DVS Gesture).

I. INTRODUCTION

Neuromorphic vision datasets encode dynamic pixel changes as spike events, combining spatial and temporal information in an event-driven format. The paper compares SNNs and RNNs on converted and naturally captured datasets under unified evaluation settings.

Dataset Motivation

  • Neuromorphic vision datasets record pixel-intensity changes as binary spike events, preserving interacting spatial and temporal information.They support applications including visual recognition, motion segmentation, tracking control, and robotics.Source: §I · p. 1

SNNs and RNNs

  • SNNs use membrane-potential dynamics and threshold-triggered binary spikes, whereas RNNs can process spike events as ordinary binary values while retaining previous states.This creates a direct comparison between event-driven SNNs and non-event-driven recurrent ANNs.Source: §I · p. 1

Study Design

  • The study systematically compares SNNs and RNNs by unifying BPTT, rate-coding-inspired losses, network structures, and training hyper-parameters.It also varies temporal resolution to test robustness and generalization.Source: §I · p. 2
  • Experiments cover DVS-converted N-MNIST and DVS-captured DVS Gesture, spanning recognition accuracy, feature extraction, temporal resolution, generalization, complexity, and parameter volume.The experiments are intended to inform model selection and construction.Source: §I · p. 2

Dataset Types

  • DVS-converted data derive spikes from repeated camera movement over static images, whereas DVS-captured data arise from natural motion with essential spatial and temporal structure.N-MNIST illustrates the converted setting; DVS Gesture illustrates the captured setting.Source: A. Neuromorphic Vision Datasets · pp. 2–3

B. Spiking Neural Networks

The paper uses LIF-based feedforward SNNs and compares them with vanilla RNNs and LSTM models that represent temporal information through continuous recurrent states or gated memory.

Spiking Neural Networks

  • The SNNs use leaky integrate-and-fire neurons connected in feedforward networks with stacked fully connected or convolutional layers.LIF is selected because of its lower complexity and widespread practical use.Source: B. Spiking Neural Networks · p. 3
  • A LIF neuron maintains continuous membrane potential and binary output activity, firing and resetting when its potential crosses a threshold.Otherwise, the membrane potential leaks over time.Source: B. Spiking Neural Networks · p. 3

Recurrent Neural Networks

  • Vanilla RNNs introduce temporal dynamics through recurrent connections and maintain one continuous hidden-state variable per neuron.Their recurrent computation uses inter-layer and intra-layer weights with an activation function such as tanh.Source: C. Recurrent Neural Networks · p. 3
  • LSTM extends vanilla RNNs with forget, input, and output gates plus cellular and hidden states to improve long-term temporal dependence.The gates are designed to alleviate gradient vanishing.Source: C. Recurrent Neural Networks · p. 3

III. METHODOLOGY

The methodology frames feedforward SNNs, vanilla RNNs, and LSTM as models with comparable spatiotemporal propagation while preserving differences in neuron states, recurrence, and gating. It uses supervised spatiotemporal backpropagation to make learning comparisons fair.

Scope

  • The comparison is explicitly between feedforward SNNs and recurrent ANNs, represented by vanilla RNNs and LSTM.The study analyzes their modeling and learning similarities and differences.Source: §III · p. 3

Neuron and Gradient Models

  • The LIF model incorporates membrane-potential leakage, threshold firing, and reset, while surrogate derivatives are used to support backpropagation through the non-differentiable firing function.The gradient-width parameter controls the approximation used during backward propagation.Source: A. Rethinking the Similarities · pp. 3–4

Spatiotemporal Dynamics

  • SNNs, vanilla RNNs, and LSTM all propagate information across spatial layers and timesteps, producing broadly similar forward and backward spatiotemporal paths.LSTM includes additional internal gate and cell states, while SNN membrane potential directly affects the next timestep.Source: A. Rethinking the Similarities · p. 4, Figure 3 · p. 4

Learning

  • The study adopts a BPTT-inspired supervised spatiotemporal backpropagation algorithm for SNNs to align learning with RNN training.The approach is selected because RNNs are commonly trained with gradient-descent-based supervised learning.Source: A. Rethinking the Similarities · p. 4

B. Rethinking the Differences

SNNs, vanilla RNNs, and LSTM share spatiotemporal information propagation but differ in recurrent connections, neuron structure, activation functions, and loss formulations.

Connection Pattern

Neuron Model

  • SNN membrane-potential paths resemble LSTM forget-gate behavior, but reset bounds the membrane potential unlike LSTM’s unbounded cellular state.Vanilla RNNs have no gates, whereas LSTM uses multiple gates and cellular state.Source: B. Rethinking the Differences · p. 5, Figure 6 · p. 5
  • SNNs use binary step-like firing outputs, while vanilla RNNs and LSTM use continuous tanh and sigmoid activations.Figure 7 contrasts firing in SNNs with tanh and sigmoid functions in RNNs.Source: B. Rethinking the Differences · p. 5, Figure 7 · p. 5

Loss Function

  • SNN loss combines spike-rate coding with MSE and uses outputs from all timesteps, whereas mainstream RNN loss uses the final hidden state.The RNN formulation also includes a trainable output weight matrix.Source: B. Rethinking the Differences · pp. 5–6

C. Comparison Methodology

The evaluation makes SNN–RNN comparisons possible by recognizing their shared spatiotemporal dynamics and unifying key experimental factors for fairness.

Comparison Methodology

  • The study unifies datasets, temporal resolution, learning algorithm, loss function, network structure, and training hyperparameters to ensure comparability and fairness.This standardization follows the models’ shared spatiotemporal dynamics while addressing their differences.Source: C. Comparison Methodology · p. 6

1) Dataset Selection and Temporal Resolution Tuning:

The study evaluates models on N-MNIST and DVS Gesture while collapsing successive event slices to tune temporal resolution and control simulation length.

Dataset Selection

Temporal Resolution Tuning

  • Temporal collapse combines multiple successive event slices into one slice by placing a spike at a pixel when any slice in the window contains one.The collapsed sequence has fewer timesteps, with the new sequence indexed by the temporal-resolution factor.Source: 1) Dataset Selection and Temporal… · p. 6
  • Increasing temporal resolution raises the new slices’ spike rate and can capture more temporal dependence when the simulation timestep count remains fixed.The same fixed number of simulation steps then covers a longer actual simulation time.Source: 1) Dataset Selection and Temporal… · p. 6

2) Learning Algorithm and Loss Function Design:

The comparison uses supervised BPTT for both model families and provides RNN loss configurations that better align with the SNN rate-coding objective.

Learning Algorithm

  • SNNs use a BPTT-inspired algorithm with spatiotemporal gradient propagation to match the supervised BPTT training used for RNNs.This choice is intended to make the comparison fair.Source: 2) Learning Algorithm and Loss… · p. 6

Loss Function

  • The SNN rate-coding MSE loss receives gradient feedback at every timestep, which can alleviate vanishing gradients to some extent.It uses outputs across the simulation sequence rather than only a terminal output.Source: 2) Learning Algorithm and Loss… · p. 7
  • RNNs are evaluated with both the mainstream loss and a modified rate-coding-inspired loss to reduce differences from the SNN objective.The modified configuration is introduced because existing all-timestep RNN losses remain slightly different from the SNN formulation.Source: 2) Learning Algorithm and Loss… · p. 7

3) Network Structure and Hyper-parameter Setting:

The study matches SNNs and RNNs with common architectures, training settings, and evaluation procedures to make their neuromorphic-data comparison fair. Experiments then examine accuracy, feature extraction, membrane mechanisms, and convergence across N-MNIST and DVS Gesture.

Network structure

  • CNN structures are implemented for SNNs, vanilla RNNs, and LSTM, while N-MNIST uses MLPs and DVS Gesture uses both MLP and CNN structures.Source: 3) Network Structure and… · p. 7

Hyper-parameters

  • Training hyper-parameters are unified across models except for SNN-specific settings, including epochs, timesteps, batch size, and learning rate.Source: 3) Network Structure and… · p. 7

Comparison protocol

  • The comparison unifies datasets, temporal resolution, BPTT, loss function, network structure, and hyper-parameters to improve fairness.Source: 3) Network Structure and… · p. 7

Accuracy

  • On N-MNIST and DVS Gesture, rate-coding-inspired loss enables RNNs to achieve comparable or better accuracy than SNNs, although vanilla RNNs remain weaker on DVS Gesture.Source: B. Overall Accuracy Comparison · pp. 7–8

Training dynamics

Feature extraction

SNN mechanisms

C. Temporal Resolution Analysis

Temporal resolution affects both accuracy and model preference. Larger resolutions generally improve accuracy, while SNNs show an advantage on sparse, DVS-captured data at small resolutions and retain stronger cross-resolution generalization.

Accuracy versus resolution

  • Accuracy improves as temporal resolution increases across the evaluated neuromorphic datasets.Larger dt produces denser slices and includes more of the moving object within the same number of simulation timesteps.Source: C. Temporal Resolution Analysis · p. 9

Dataset dependence

Generalization test

Generalization mechanism

D. Loss Function & Temporal Contrast

The study analyzes temporal contrast through cross-entropy between event slices to explain when the rate-coding-inspired loss benefits RNNs. DVS-captured data have stronger temporal contrast, but excessive contrast can undermine the loss’s effectiveness.

Temporal contrast

Dataset comparison

Limitation

E. Number of Parameters and Operations

The comparison evaluates computational complexity and parameter volume as practical costs of SNNs and RNNs. SNNs use fewer parameters while maintaining competitive recognition accuracy through membrane-potential dynamics and restricted recurrence.

Operations

Parameter volume

Accuracy-cost trade-off

F. Discussion on Audio Data

An additional Spoken-Digits experiment extends the comparison beyond vision data. On this audio dataset, SNNs perform best and generalize better when testing temporal resolution varies.

Audio experiment

Broader comparison

  • Across the broader benchmark, rate-coding-inspired loss improves RNN accuracy, especially for LSTM, making it comparable to or slightly better than SNNs.Source: §V · p. 12
  • The study reports that SNNs are especially advantageous for sparse spike events at small temporal resolutions, while LSTM benefits from longer temporal contexts.Source: §V · p. 12

Related papers

Loading 2005.02183v1…