PAFS: Perturbation-Aware Firing Scaffolds for Robust Event-Based SNNs

Introduction

• Existing robustness methods often require access to the training pipeline and backbone parameters, while prediction-loss-driven parameter-efficient adaptation does not identify informative temporal responses.This motivates labeled post-training calibration with the backbone frozen.Source: Introduction · p. 1
• Across four matched dataset–backbone settings, PAFS improves corrupted-average and worst-condition accuracy over the corresponding Adapter anchors.The benefit is attributed to residual-aligned internal firing structure rather than logits alone, generic low-rank compression, or a single firing source.Source: Introduction · p. 2
• Late-layer firing trajectories under temporal corruptions retain systematic information about residual prediction errors beyond the final logits.The proposed calibration uses that structure to correct remaining errors.Source: Introduction · pp. 1–2
• PAFS aligns standardized late-layer LIF trajectories with an Adapter anchor’s class-probability residuals to extract a compact residual-aligned firing subspace.At inference, projected scaffold features and anchor statistics predict a residual logit correction without updating the pretrained backbone or requiring another backbone pass.Source: Introduction · pp. 1–2

Event Based Neuromorphic Vision

• DVS-Gesture, DailyAction-DVS, and DVS-Gesture-Chain test recognition settings where temporal composition, missing windows, jitter, and speed changes matter.DailyAction-DVS introduces more varied action dynamics, while DVS-Gesture-Chain highlights dependence on temporal composition rather than event counts alone.Source: Event Based Neuromorphic Vision · p. 2

Deep SNN Training and Spiking Transformers

• SNN training and spiking Transformer methods strengthen optimization, temporal representation, and attention, but do not identify late firing trajectories that predict post-training residual errors.PAFS operates on late LIF responses and is evaluated with both Spikformer and Spikingformer.Source: Deep SNN Training and Spiking… · p. 2

Robustness of SNNs

• SNN robustness work spans adversarial objectives, gradient constraints, stochastic gating, input encoding, membrane regularization, and certified analysis.PAFS instead fixes the backbone and uses training-split internal activity to calibrate errors remaining after parameter-efficient adaptation.Source: Robustness of SNNs · p. 2

Parameter Efficient Adaptation and Firing Subspaces

• Adapters and LoRA motivate lightweight adaptation, but PAFS uses an Adapter only for its anchor and extracts its main error-aligned representation from LIF firing curves.The firing scaffold is a calibration representation, not a pruned or compressed replacement network.Source: Parameter Efficient Adaptation and… · pp. 2–3

Overview

• The overview’s three stages are Adapter-anchor training, residual-aligned scaffold construction, and same-pass firing projection with logit correction, all with the backbone frozen.After calibration, the learned components remain fixed and inference is label-free.Source: Overview · p. 3, Figure 1 · p. 4

Adapter Anchor

• The Adapter anchor is a frozen reference predictor selected by mean validation accuracy across clean and corrupted temporal domains.Only Adapter parameters are optimized when forming the anchor; the selected checkpoint and backbone are then frozen.Source: Adapter Anchor · p. 3

Residual-Aligned Firing Scaffold

• PAFS forms a scaffold by associating late-layer firing responses with class-wise anchor residuals, averaging weighted corruption-domain signatures, and retaining dominant residual-aligned directions with SVD.The residual preserves the signed class-wise probability adjustment; clean-domain signatures are excluded so the scaffold captures perturbation-induced structure.Source: Residual-Aligned Firing Scaffold · pp. 3–5

Scaffold-based Residual Calibration

• At inference, standardized firing trajectories are projected onto fixed scaffolds and combined with anchor logits, probabilities, and uncertainty statistics to predict a residual logit correction.Only correction-head parameters are optimized; the backbone, Adapter, and scaffold bases remain fixed.Source: Scaffold-based Residual Calibration · p. 5

Parameter and Inference Cost

• PAFS adds 8.796K trainable parameters, or 0.342% of the 2.569M-parameter predictor, for DailyAction-DVS with Spikingformer and uses one backbone pass at inference.Fixed scaffold buffers are excluded from the parameter ratio; inference adds scaffold projection and lightweight logit correction.Source: Parameter and Inference Cost · p. 5

Protocol

• The main evaluation uses CIFAR10-DVS, DailyAction-DVS, and DVS-Gesture-Chain with Spikingformer or Spikformer, and compares PAFS with LoRA, Adapter, and Plain MLP baselines.Inputs use 16 event-count frames at 128 × 128 resolution; corruptions include 30% window removal, timestamp offsets within ±2 temporal units, and twofold speed changes.Source: Protocol · pp. 5–6
• Scaffold construction and correction-head training use training data, hyperparameters are selected on validation data, and test data are reserved for final evaluation.The protocol notes that three inherited Full anchors were test-selected historically, but no test examples are used to fit or select PAFS or correction controls.Source: Protocol · p. 6

Robustness Across Backbones

• PAFS achieves the highest reported mean Corr. Avg. and Worst accuracies across all four dataset–backbone settings, with advantages over Adapter anchors across evaluated corruptions.The magnitude is task dependent: gains are clearer on DVS-GC and DailyAction-DVS, and marginal on CIFAR10-DVS.Source: Robustness Across Backbones · p. 7
• Across unseen severities, PAFS is highest or competitive over most Window Drop and Jitter levels, but does not dominate every Speed Change severity.Its clearest advantage is under severe event removal; accuracy declines toward both extreme speed factors.Source: Robustness Across Backbones · p. 7, Figure 2 · p. 7

Representation and Design Analysis

• Under matched Adapter-anchor correction protocols, PAFS outperforms logit-only, PCA-scaffold, and random-scaffold variants, with clearer gains on DailyAction-DVS than CIFAR10-DVS.The comparisons support residual alignment as useful beyond access to internal activations or low-rank compression alone.Source: Protocol · p. 6, Representation and Design Analysis · p. 7, Table 2 · p. 7
• PAFS remains broadly stable across scaffold ranks, with r = 4 selected by validation performance; increasing rank does not consistently improve robustness.The reported analysis interprets the dominant residual-related firing structure as compact.Source: Representation and Design Analysis · p. 7, Figure 3 · p. 8
• The combined attention–MLP scaffold achieves the strongest robustness across both backbones, while attention-only features better preserve clean accuracy.The reported interpretation is that attention and MLP firing responses capture complementary aspects of residual error structure.Source: Representation and Design Analysis · pp. 7–8, Figure 4 · p. 8

• PAFS improves robustness over Adapter and Logit-only correction without modifying the predictor or adding a backbone pass.It does so by extracting residual-aligned firing subspaces from a frozen SNN for post-training calibration.Source: Conclusion · p. 8

• The supplement covers architectural details, parameter and memory accounting, experimental protocols, and implementation settings for reproducing and interpreting the results.Source: Supplementary Material for PAFS:… · p. 11

• PAFS constructs residual-aligned scaffolds from a fixed training-only calibration pool, using Dfit = 384 by default and excluding Drest = 128 from scaffold construction and normalization.Both subsets remain part of training data and may appear during Adapter and correction-head optimization; neither is used for validation or test selection.Source: Appendix A · p. 11
• The procedure trains and freezes an Adapter anchor, standardizes firing trajectories, builds weighted firing-residual matrices, applies SVD, then trains a correction head using scaffold projections.The bases and normalization statistics are frozen after construction, and inference runs the frozen anchor once before projection and correction.Source: Appendix A · pp. 11–12
• For DailyAction-DVS with Spikingformer, the added trainable parameters are approximately 0.342% of the frozen 2.569M-parameter predictor.The fixed scaffold bases and normalization statistics require 72.0 KiB in FP32 and are excluded from that trainable-parameter percentage.Source: Appendix A · p. 12

Evaluation Scope

• The protocol tests whether PAFS works across event-recognition tasks with different dependence on temporal structure.It separately tests transfer across spiking-Transformer architectures and robustness to complementary temporal failure modes without sacrificing clean-input performance.Source: Appendix B · p. 12

CIFAR10-DVS

• CIFAR10-DVS serves as a weak temporal-order control because its labels reflect static object identity rather than naturally evolving motion.Its split is kept identical across compared methods because the release does not prescribe a unique train/validation/test partition.Source: Datasets · p. 12

DailyAction-DVS

• DailyAction-DVS is the principal setting for temporal-perturbation robustness on natural action recordings and is tested with both Spikformer and Spikingformer.The task, split, preprocessing, and corruption protocol are held fixed across the two backbones.Source: Datasets · p. 12

DVS-Gesture-Chain

• DVS-Gesture-Chain assigns different labels to different orders of gesture primitives, making it an explicitly order-sensitive benchmark.Its programmatically composed chains are treated as a controlled temporal-order benchmark, not naturally occurring long activities.Source: Datasets · p. 13

Why the datasets are complementary

• The dataset progression runs from spatial-evidence object recognition through natural action dynamics to classes distinguished by temporal composition.This progression is aligned with the hypothesis that late-layer firing trajectories can retain task-relevant temporal structure beyond anchor logits.Source: Datasets · p. 13

Shared architecture and input convention

• Both audited encoders retain membrane-state information across T = 16 steps, while spatial token interaction is computed within each step.Their shared configuration uses depth 2, hidden dimension 256, MLP hidden dimension 1,024, and patch size 16.Source: Backbone Architectures · p. 13

Why both backbones are evaluated

• Spikformer uses post-activation spike-element-wise residuals, whereas Spikingformer uses pre-activation blocks and propagates a membrane-valued shortcut.The Spikingformer variant evaluated is the base ConvBN→LIF→MaxPool tokenizer configuration.Source: Backbone Architectures · pp. 13–14

Architectural transfer

• DailyAction-DVS uses both backbones, while CIFAR10-DVS and DVS-Gesture-Chain use Spikingformer and Spikformer, respectively, for four settings total.The results support transfer across these spiking Transformers, not generalization to every SNN architecture.Source: Backbone Architectures · p. 14

How PAFS interfaces with the backbones

• PAFS collects attention-projection and first-MLP LIF responses from two late sources, spatially averages them while preserving time, and projects them onto a fixed rank-4 scaffold.The Adapter and backbone remain frozen, and the extraction interface is unchanged across the two backbones.Source: Backbone Architectures · p. 14

Temporal corruption operators

• The corruption suite evaluates Window Drop, timestamp jitter, and twofold speed change after conversion to 16 event-count frames.Window Drop removes a contiguous interval; jitter displaces binned frame contents, and speed change resamples the temporal axis and pads it to the original length.Source: Backbone Architectures · p. 14

Metrics aligned with the evaluation claims

• Corr. Avg. excludes clean accuracy, while Worst reports the lowest cross-run mean among the three corruptions; clean accuracy is a preservation constraint.Accuracies are reported as mean ± standard deviation over matched runs, with Worst’s variation taken from the same corruption as its mean.Source: Backbone Architectures · p. 15

Reproducibility protocol

• Training data alone supplies scaffold statistics and bases; validation selects checkpoints, hyperparameters, and deployment scales, while test data are reserved for reporting.Comparisons use matched splits, corruptions, and model seeds.Source: Appendix C · pp. 15, 17

Calibration sampling and data use

• The requested calibration pool is capped by available training examples; in the default DailyAction-DVS setting, all 512 requested examples are available.For each seed, 384 examples estimate normalization statistics and scaffold bases; the other 128 are excluded from scaffold estimation.Source: Appendix C · p. 15

Adapter-anchor optimization

• Stage 1 optimizes only the Adapter, selects it by mean validation accuracy across clean and corrupted domains, then freezes it as the common anchor.Adapter mini-batches draw from Clean, Window Drop, Jitter, or Speed one domain at a time.Source: Appendix C · pp. 15–16

Scaffold construction

• Stage 2 uses no gradient optimization: it computes normalized firing–residual associations and stores the extracted rank-r bases and normalization statistics as fixed buffers.Freezing the Adapter keeps trajectories and residuals defined relative to one anchor predictor.Source: Appendix C · p. 16

Correction-head optimization

• Stage 3 optimizes only the linear correction head, with the backbone, classifier, Adapter, normalization statistics, and scaffold bases fixed.Zero-initialized correction weights and bias make training begin exactly from the frozen Adapter anchor.Source: Appendix C · p. 16

Scale selection

• Deployment scales are selected after the correction head is fixed, and α = 0 remains an anchor-preserving validation fallback.The test predictor uses the validation-selected scale; both the trained head and selected scale remain fixed during test evaluation.Source: Appendix C · p. 16

Controls

• The Logit-only MLP matches PAFS correction-head capacity but receives only anchor logits, while PCA and random-scaffold controls retain the same anchor, firing sources, rank, and training protocol.Scalar channel controls vary the ranking criterion to distinguish individual-channel importance from multivariate residual-aligned structure.Source: Appendix C · pp. 17–18

Baseline optimization

• Adapter, LoRA, and Plain MLP use AdamW at learning rate 5 × 10^-4 for up to 160 epochs; PAFS and firing-space controls use the same learning rate for up to 80 correction epochs.Baseline methods use mixed precision in training, whereas correction training is FP32 with the residual scale fixed at αtrain = 1.Source: Appendix C · p. 17

Appendix D: Additional Experimental Results

• The appendix analyses whether modest training-only calibration pools suffice, whether projected firing coordinates help beyond matched correction capacity, and whether generic or independently ranked features can replace the scaffold.The scalar controls preserve the Adapter anchor, firing sources, retained-feature budget, correction head, and optimization protocol while changing only how firing representations are constructed.Source: Appendix D · pp. 18–19

Estimating residual-aligned scaffolds from limited calibration data

• Increasing the request cap from 128 to 256, or the fitting subset from 96 to 192 samples, improves both Corr. Avg. and Worst, but larger pools do not consistently improve results.The 512-cap configuration remains competitive; the 1,024 request realizes 929 samples, and this fixed-rank study supports estimability from a modest training-only subset, not monotonic sample efficiency.Source: Appendix D · p. 18, Figure S1 · p. 18

Efficiency–robustness trade-off

• PAFS improves Corr. Avg. and Worst over Adapter by 1.84 and 3.23 points, respectively, with the same learned-parameter count as a Logit-only MLP.The gains add 0.035M MACs, 10.6% latency, and 7.5% gross GPU board energy; runtime cost comes mainly from collecting and projecting two late-layer trajectories.Source: Appendix D · p. 18, Table S3 · p. 19

Residual alignment and multivariate firing structure

• Residual relevance reaches 88.62 Corr. Avg. and 87.08 Worst, while random ranking and generic activation statistics do not consistently outperform the Adapter anchor.The comparison supports residual alignment as more informative than generic firing salience or perturbation sensitivity alone; PAFS’s multivariate scaffold captures additional joint firing structure.Source: Appendix D · p. 19, Table S4 · p. 19
• 88.97 Corr. Avg. and 87.20 Worst are PAFS’s strongest overall results among scalar ranking rules under the same retained-feature budget and correction protocol.PAFS also exceeds residual relevance by 1.08 percentage points under global temporal rescaling; no scalar control matches it on both aggregate criteria.Source: Appendix D · p. 19, Table S4 · p. 19