PaperlayerView exact revision on arXiv

Source-linked AI summary

Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations

Saima Sharmin, Nitin Rathi, Priyadarshini Panda, Kaushik Roy

arXiv:2003.10399v2cs.CVcs.NE

TL;DR

Adversarial attacks continue to challenge deep neural networks, and the paper investigates SNNs as a candidate for inherent robustness. It analyzes discrete Poisson encoding and IF/LIF activations, and proposes surrogate-gradient attacks for SNNs. SNNs show higher adversarial accuracy than ANNs in blackbox gradient-based attacks, with robustness varying by training method, neuron type, and timestep count.

  • Problem

    Existing defenses remain vulnerable to counterattacks, leaving adversarial robustness a continuing concern for deep neural networks.

    Source: §1 · pp. 1–2

  • Method

    The paper compares SNNs with ANNs, analyzes Poisson input discretization and IF/LIF activations, and uses surrogate gradients to craft SNN-based attacks.

    Source: §1 · p. 3, §3.3 · p. 7

  • Results

    SNNs have higher adversarial accuracy than ANNs in gradient-based blackbox attacks on deep CIFAR VGG and ResNet models, with comparable clean accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Backpropagated SNNs using LIF neurons and fewer timesteps are more robust than converted SNNs using IF neurons and more timesteps.

    Source: §1 · p. 3, §5 · p. 14

    The conclusion is validated only for gradient-based attacks on deep VGG and ResNet architectures.

    Source: §4.1 · p. 9

Abstract

from arXiv · show

In the recent quest for trustworthy neural networks, we present Spiking Neural Network (SNN) as a potential candidate for inherent robustness against adversarial attacks. In this work, we demonstrate that adversarial accuracy of SNNs under gradient-based attacks is higher than their non-spiking counterparts for CIFAR datasets on deep VGG and ResNet architectures, particularly in blackbox attack scenario. We attribute this robustness to two fundamental characteristics of SNNs and analyze their effects. First, we exhibit that input discretization introduced by the Poisson encoder improves adversarial robustness with reduced number of timesteps. Second, we quantify the amount of adversarial accuracy with increased leak rate in Leaky-Integrate-Fire (LIF) neurons. Our results suggest that SNNs trained with LIF neurons and smaller number of timesteps are more robust than the ones with IF (Integrate-Fire) neurons and larger number of timesteps. Also we overcome the bottleneck of creating gradient-based adversarial inputs in temporal domain by proposing a technique for crafting attacks from SNN

1 Introduction

Existing defenses remain vulnerable to counterattacks, motivating the search for inherent robustness; this paper examines SNNs because their discrete inputs and neuron activations differ from ANNs. It reports higher SNN adversarial accuracy in blackbox attacks and analyzes the conditions associated with that robustness.

1 Introduction

  • The paper motivates SNNs as potential inherently robust networks because their binary inputs and IF or LIF activations differ from ANNs’ continuous inputs and ReLU activations.The authors contrast Poisson input discretization with the piecewise-linear ReLU activation used in ANNs.Source: §1 · p. 2
  • Prior SNN adversarial-attack studies mostly used MNIST or shallow networks, whereas this work evaluates deep SNNs on CIFAR datasets.The comparison includes ANN-converted SNNs and converted networks further trained with surrogate-gradient backpropagation.Source: §1 · p. 2
  • SNNs achieve higher adversarial accuracy than ANNs in gradient-based blackbox attacks on deep VGG and ResNet models for CIFAR10 and CIFAR100, with comparable clean accuracy.For whitebox attacks, the comparison depends on the relative strengths of the adversary.Source: §1 · p. 2
  • Robustness is associated with Poisson input discretization and nonlinear IF or LIF activations, while backpropagated LIF models with fewer timesteps are more robust than converted IF models.The paper also reports that increasing LIF leak improves robustness and proposes surrogate-gradient-based attack generation for SNNs.Source: §1 · p. 3

2 Background

The background defines adversarial examples and describes FGSM and PGD, then introduces SNN input encoding, neuron dynamics, and training approaches. It explains how surrogate gradients support backpropagation despite discontinuous neuron derivatives.

2.1 Adversarial Attack

  • An adversarial image should remain visually similar to the clean image while causing the network to misclassify it.The distance metric and perturbation bound depend on the attack method.Source: §2.1 · p. 3
  • FGSM seeks a loss-increasing perturbation under an l∞ bound, while PGD iterates FGSM-like updates and projects them into an ϵ-ball around the input.The described PGD variant uses k steps and per-step perturbation α; another variant starts with a random perturbation.Source: §2.1 · p. 4, Fast Gradient Sign Method (FGSM) · p. 4, Projected Gradient Descent (PGD) · p. 4

2.2 Spiking Neural Network (SNN)

  • Poisson encoding converts analog inputs into time-varying binary spikes, and inference uses output-neuron membrane potential accumulated over T timesteps.IF neurons use λ=1, whereas LIF neurons use λ<1; threshold crossing produces a spike and resets or reduces the membrane potential.Source: §2.2 · pp. 4–5
  • ANN-SNN conversion tunes layer thresholds so IF activity mimics the corresponding ANN’s ReLU activity.Thresholds are balanced sequentially using maximum activations recorded from Poisson-encoded training inputs.Source: ANN-SNN Conversion · p. 5
  • Surrogate gradients approximate the discontinuous derivative of IF or LIF neurons, enabling backpropagation through time across the network’s timesteps.The hybrid procedure begins with ANN-SNN conversion and can train IF or LIF networks with fewer training timesteps than conversion alone.Source: ANN-SNN Conversion · pp. 5–6

3 Experiments

The experiments use CIFAR models trained as ANNs, converted SNNs, or converted-and-backpropagated SNNs. For SNN attacks, the method accumulates surrogate gradients at the first convolution and uses their sign to construct adversarial inputs.

3.1 Dataset and Models

  • Experiments use VGG5 and ResNet20 on CIFAR10 and VGG11 on CIFAR100, with architecture-specific configurations and pooling choices.The networks omit bias terms and use dropout in place of batch normalization.Source: §3.1 · p. 6

3.2 Training Procedure

  • The comparison covers ANN, ANN-SNN-converted, and backpropagated SNN models, with conversion using 2500 timesteps and backpropagated models using 100 or 200 timesteps.The input-quantization and leak analyses use VGG5 on CIFAR10; the leak factor is otherwise 0.99.Source: §3.2 · pp. 6–7

3.3 Adversarial Input Generation Methodology

  • For SNN-crafted FGSM and PGD attacks, surrogate gradients from the first convolution are accumulated across timesteps and their sign is used to construct the adversarial input.The method approximates the first-layer membrane potential using the timed-average input spike train, then applies standard FGSM or PGD construction.Source: §3.3 · pp. 7–8

4 Results

Across CIFAR VGG and ResNet settings, SNN-BP has higher adversarial accuracy than ANN and SNN-conv under blackbox attacks, while whitebox comparisons depend on adversary strength. The section examines input discretization, LIF dynamics, and attacks crafted from SNNs as factors in these results.

4.1 ANN vs SNN

  • SNN-BP has the highest adversarial accuracy in every blackbox FGSM and PGD case across the reported CIFAR architectures, while whitebox results vary with adversary strength and include a VGG11 CIFAR100 exception.The comparison covers CIFAR10 VGG5 and ResNet20 and CIFAR100 VGG11; SNN-BP's whitebox advantage on CIFAR10 is 2%–4.6%.Source: §4.1 · p. 8, Table 1 · p. 9
  • For SNN-BP, reducing timesteps from 150 to 50 raises FGSM accuracy by about 5% at 4-bit quantization, while SNN-conv is nearly timestep-insensitive because longer windows average Poisson outputs.Pre-quantizing inputs to 2 bits further improves SNN-BP adversarial accuracy; for SNN-conv, input pre-quantization raises accuracy from about 21% to 28%.Source: Effect of input quantization and… · pp. 10–11, Figure 2 · p. 10, Figure 3 · p. 11
  • In a proof-of-concept image comparison, LIF reduces perturbation l2-norm from 0.6 at the input to 0.3 at its output, whereas ReLU leaves it unchanged.The authors describe LIF's binary spikes, averaged over the time window, as a nonlinear transformation that can diminish perturbation propagation; IF also transforms nonlinearly, but its quantization effect is minimized over longer windows.Source: Effect of LIF (or IF) neurons and the… · pp. 11–12, Figure 4 · p. 12
  • For a VGG5 SNN-BP on CIFAR10, reducing the LIF leak factor to 0.95 raises FGSM and PGD adversarial accuracy by 3%–6%, with clean accuracy varying by about 2%.The authors associate increased leak with lower spike rate and stronger temporal dependence, and assume the resulting nondeterminism contributes to robustness.Source: Effect of LIF (or IF) neurons and the… · pp. 12–13, Figure 6 · p. 13

4.2 ANN-Crafted vs SNN-Crafted Attack

  • The proposed SNN-based attack-crafting technique uses surrogate gradients, and SNN-BP retains the highest adversarial accuracy against both ANN-crafted and SNN-crafted blackbox inputs.For an ANN-II target under FGSM, accuracy is 18.9% against ANN-I inputs versus 32.7% against SNN-conv-I and 31.3% against SNN-BP-I, suggesting ANN-crafted attacks are stronger than the corresponding SNN-crafted attacks.Source: §4.2 · pp. 13–14, Table 2 · p. 14

5 Conclusions

The paper concludes that SNNs show resilience to gradient-based attacks through discrete input encoding and nonlinear neuron activations, with robustness further improved by fewer timesteps and greater LIF leak. The conclusion is limited to deep VGG and ResNet networks on CIFAR datasets and calls for evaluation with more diverse attacks and architectures.

5 Conclusions

  • SNNs with LIF neurons and fewer timesteps are reported as more robust than converted SNNs with IF neurons and larger timestep counts.The authors attribute the resilience to discrete input encoding and nonlinear LIF or IF activations.Source: §5 · p. 14
  • The reported conclusion is validated only for gradient-based attacks on deep VGG and ResNet networks with CIFAR datasets.The authors call for future analysis using more diverse attack methods and architectures.Source: §5 · p. 14
  • The authors propose using surrogate gradients to generate gradient-based attacks from SNNs.Source: §5 · p. 14

Supplementary

The section develops an SNN-based FGSM attack by approximating the gradients that are discontinuous at neuron activations and input encoding. It treats the time-averaged input spike train as a continuous rate input to make the network differentiable for attack generation.

SNN-crafted adversarial example

Related papers

Loading 2003.10399v2…