Neuromorphic Data Augmentation for Training Spiking Neural Networks

1 Introduction

• DVS cameras asynchronously report pixel-brightness changes, and the paper integrates sparse event streams into multiple frames for feature extraction.The introduction describes this representation as sparse frame-based data.Source: §1 · pp. 1–2
• Fig. 2 compares CIFAR10-DVS training and test accuracy curves for ResNet-19 and VGG-11, with NDA-trained networks reported to converge better.The text describes baseline test accuracy as oscillatory and substantially below training accuracy.Source: Figure 2 · p. 2, §1 · p. 2
• Limited DVS datasets are associated with large training-test gaps: CIFAR10-DVS has 10k instances versus 60k in CIFAR10, and test accuracy oscillates at about half of training accuracy.Recording additional augmented RGB images with a DVS camera incurs prohibitively high latency, motivating augmentation directly on DVS data.Source: §1 · p. 2
• NDA transforms already-recorded DVS data and is reported to improve training stability and generalization while enabling unsupervised contrastive learning for SNNs.The paper presents NDA as compatible with existing training algorithms and reports negligible cost.Source: §1 · p. 3
• The paper evaluates Neuromorphic Data Augmentation (NDA) on CIFAR10-DVS, N-Caltech 101, N-Cars, and N-MNIST, reporting a significant accuracy boost over previous state-of-the-art results.Figure 1 shows event collection from a chair image in N-Caltech101 [47] and neuromorphic augmentation of event data.Source: Figure 1 · p. 2, §1 · p. 3

2.1 Data Augmentation

• EventDrop randomly removes events, while NDA addresses consistency when applying augmentation directly to event data; unlike video augmentation, NDA uses geometric rather than photometric and color augmentations.The text also identifies video augmentation as a related approach to spatial-temporal data.Source: §2.1 · p. 3

2.2 Spiking Neural Networks

• Prior SNN work emphasizes training algorithms and architectures, whereas this paper addresses overfitting in sparse DVS datasets during direct training.The paper excludes ANN-to-SNN conversion because its topic is neuromorphic datasets requiring direct training.Source: §2.2 · p. 4

3 Conventional Augmentations for RGB Data

• RGB augmentation is divided into photometric and color transformations, which alter image illumination or color space, and geometric transformations, which act on pixel coordinates.The figure groups examples 2–5 as photometric and color augmentations and examples 6–9 as geometric augmentations.Source: §3 · p. 4, Figure 3 · p. 5
• Contrast, saturation, grayscale, and Gaussian blur are described as value-based augmentations that transform pixel values or apply a Gaussian convolution.The text gives contrast enhancement as a pixelwise transformation.Source: §3 · p. 4
• Horizontal flipping, resizing, rotation, and cutout are geometric examples that change pixel coordinates or apply a mask.The text characterizes geometric augmentations as index-based.Source: §3 · p. 4, Figure 3 · p. 5

4.1 DVS Data and Augmentation

• NDA augments recorded DVS data directly, avoiding the costly runtime alternative of augmenting RGB images and then generating event streams.The method aims for the augmented DVS data to agree with event generation applied after augmentation.Source: §4.1 · p. 6
• Photometric augmentations are unsuitable because they can produce continuous-valued data and make post-augmentation event generation intractable, whereas geometric changes preserve event-stream structure.The paper identifies index-based geometric augmentation as suitable because it changes event positions while retaining the stream.Source: §4.1 · pp. 6–7
• The considered augmentations include horizontal flipping, rolling, rotation, cutout, horizontal shear, and CutMix.Flipping preserves polarity and time; CutMix combines two inputs and their labels using a binary mask and area ratio.Source: §4.1 · p. 7

4.2 Augmentation Policy

• The policy always enables flipping and CutMix, then samples 1–4 transformations from rolling, rotation, cutout, and ShearX at a chosen intensity.The number of sampled augmentations and their intensity are controlled by M and N, respectively.Source: §4.2 · p. 8

4.3 Application to Unsupervised Contrastive Learning

• Contrastive learning uses differently augmented views of the same input to learn similar features without labels, and the paper applies this approach to SNNs using NDA.The authors frame this as a way to learn low-level features for transfer to higher-level vision tasks.Source: §4.3 · p. 8
• For unsupervised contrastive learning, two NDA views of one DVS input are encoded to produce outputs trained for similarity, with a predictor on one branch and gradient detach on the other.After pretraining, the predictor is removed and the encoder is fine-tuned with a new classifier head for transfer learning.Source: §4.3 · p. 9

5 Experiments

• Experiments use ResNet-19 and VGG-11 SNNs with tdBN and M1N2 NDA, keeping other training hyperparameters aligned with the baseline.Training runs for 200 epochs on four GPUs, with a total batch size of 256.Source: §5 · p. 9

5.1 Ablation Study

• Geometric augmentation outperforms photometric/color augmentation across datasets, while augmentation intensity exhibits a bias-variance trade-off.The authors caution against augmentations that are too simple to diversify data or too complex to retain useful event information.Source: §5.1 · p. 10, Table 3 · p. 10

5.2 Comparison with Existing Literature

• NDA reaches 78.0% top-1 accuracy on CIFAR10-DVS with ResNet-19, and improves results on N-Caltech 101 and N-Cars over baselines.The reported gains are 15.8% on N-Caltech 101 and 4.8% on N-Cars; the comparison also includes CIFAR10-DVS, N-Caltech 101, and N-Cars top-1 accuracy.Source: §5.2 · pp. 10–11, Table 4 · p. 11
• NDA yields 0.12% higher accuracy on N-MNIST; on N-Cars, it improves ResNet-19 accuracy by 4.8% at 48×48 resolution and VGG-11 accuracy by 4.5% at full resolution.The N-MNIST gain is marginal because most baselines already exceed 99% accuracy.Source: §5.2 · p. 11

5.3 Analysis

• With Gaussian weight noise N(0, 0.02), NDA models lose 11.3% accuracy versus 43.4% without NDA, and NDA models have lower Hessian spectra and trace.The CIFAR10-DVS ResNet-19 noise-injection results are averaged over five runs per noise setting.Source: §5.3 · p. 12, Figure 5 · p. 13, Table 7 · p. 13
• NDA adds 10% to estimated operations, costs 1.7 ms per CIFAR10-DVS training image, and yields 16.3% higher accuracy with validation-based model selection.Operation count is estimated as FireRate×MAC, and the validation experiment uses 15% of training data for validation.Source: §5.3 · p. 13, Figure 6 · p. 13, Table 8 · p. 13

5.4 Unsupervised Contrastive Learning

• NDA-based unsupervised pretraining on N-Caltech 101 yields 80.8% CIFAR10-DVS accuracy after 100 fine-tuning epochs and 82.1% after 300.Supervised pretraining reaches 80.9%, 0.8% below the no-pretraining baseline; the 300-epoch unsupervised result is reported as a new state of the art.Source: §5.4 · p. 14, Table 9 · p. 14

• NDA improves SNN generalization on event-based data by generating new high-quality instances that encourage invariant features.Source: §6 · p. 14
• The authors describe NDA as regularization that improves the bias–variance trade-off and finds flatter minima with higher test accuracy.Source: §6 · p. 14
• NDA enables unsupervised pre-training for transfer learning.Source: §6 · p. 14
• Current NDA lacks value-based event augmentations, which the authors identify as a direction for future study using logical operations.Source: §6 · p. 14