SICAF: Time-Varying Focus Bottleneck for Self-Supervised Event-Based Optical Flow with Spiking Neural Network

1 Introduction

• Existing methods overlook temporal heterogeneity in event streams, limiting informative motion cues in complex scenes.The stated settings include fast motion, adverse illumination, and highly dynamic backgrounds; contrast maximization also neglects timestamps and temporal distributions, while uniform information bottleneck compression can miss salient temporal features.Source: §1 · p. 2
• The Figure 1 comparison reports more accurate flow predictions with SICAF, addressing incorrect flow direction and lost fine-grained motion details.The caption attributes the improvement to emphasizing temporal saliency and capturing structured, informative motion features.Source: Figure 1 · p. 2, §1 · p. 2
• SICAF formulates learning as constrained optimization with temporal modulation to compress event information while retaining temporally salient features.The paper names this strategy “Compactly Compress, Time-varying Focus.”Source: §1 · pp. 2–3
• A learnable temporal-attention estimator extracts cues from spiking neural activity and dynamically guides compression across time.It uses latent spike firing rates to model temporal dynamics and provide time-varying focus.Source: §1 · p. 3
• SICAF achieves superior performance to existing state-of-the-art SNN-based optical-flow models while significantly enhancing robustness.The introduction reports these overall experimental findings without specifying metric values.Source: §1 · p. 3

2.1 Event-based Optical Flow Estimation with SNNs

• Directly trained SNNs exploit spiking-neuron temporal dynamics for low-latency, efficient spatiotemporal feature learning.Prior work includes adaptive neurons, hybrid networks, self-supervised temporal cues, sensor fusion, and ANN-to-SNN conversion.Source: §2.1 · pp. 3–4
• Self-supervised SNN optical-flow work includes short-event-slice temporal cues, frame-based sensor fusion, and adaptive neural mechanisms.The cited examples are,, and, respectively.Source: §2.1 · p. 4

2.2 Direct Training of SNNs with the IB Principle

• SeLHIB and other information-bottleneck methods improve robustness or generalization, but uniform temporal compression can suppress critical temporal cues.The section positions SICAF as preserving compactness while selectively retaining temporally salient features.Source: §2.2 · p. 4

3 Preliminary

• Spiking neurons integrate incoming postsynaptic potentials and emit a spike when membrane potential crosses a firing threshold.This event-driven operation supports processing asynchronous data.Source: §3 · p. 4
• In the LIF model, u(t) is the post-update membrane potential, i(t) the synaptic input current, and o(t) the binary output spike.The passage also identifies pre as the pre-update membrane potential.Source: §3 · p. 4
• Contrast maximization motion-compensates events to a common reference frame and minimizes the normalized sum of squared average timestamps.The objective uses optical flow at each pixel and average timestamps at each pixel.Source: §3 · p. 4

4.1 Temporal Attention Scores

• Spike-rate matrices are projected into temporal queries and keys to produce attention weights normalized across key timesteps and averaged into a timestep distribution.The resulting distribution captures sparse, irregular temporal variability and guides selective accumulation.Source: §4.1 · p. 5

4.2 Time-Sensitive Rényi’s α-Entropy Estimator

• The time-sensitive Rényi’s α-entropy estimator weights latent information by temporal attention to preserve heterogeneity across time.Rényi’s α-entropy is used as a robust measure of information content.Source: §4.2 · pp. 5–6

4.3 SICAF Learning Strategy

• SICAF constrains input–latent mutual information while requiring attention-weighted temporal entropy to remain above a threshold.This formalizes compact compression while retaining temporally salient information.Source: §4.3 · p. 6

4.4 Mechanistic Analysis of Temporal Heterogeneity in SICAF

• Both SICAF and SeLHIB produce sparser maps than the baseline; SICAF’s lower adjacent-step SSIM than both indicates stronger temporal decorrelation and higher temporal resolution.The feature-map comparison supports compactness, while SSIM compares firing patterns at adjacent timesteps.Source: §4.4 · p. 7, Figure 4 · p. 7

5.1 Experimental Setup

• Experiments use 140 training sequences of 128×128 pixels, each with 500k events, and produce predictions at each frame timestep with dt=2.The sequences contain nearly 15 minutes of event data in total; dt=2 corresponds to 25% of window events at a time.Source: §5.1 · p. 8

5.2 Performance Evaluation on Event-based Optical Flow Estimation

• 8.16% improvement over the prior work applying IB to self-supervised SNN optical flow is reported, alongside MVSEC, ECD, and HQF evaluation.The MVSEC comparison is presented in Table 1.Source: §5.2 · p. 8, Table 1 · p. 8

5.3 Robustness of Event-based Optical Flow Estimation in Noisy Situations

• 13.49% improvement in accuracy under white-box adversarial conditions accompanies robustness gains under random and adversarial noise.The authors attribute the white-box robustness to time-sensitive adaptive compression that focuses on critical events and suppresses irrelevant or noisy data.Source: §5.3 · pp. 8–9, Table 2 · p. 9

5.4 Ablation on Loss Function Components

• Adding Lsmooth to Lcontrast improves estimation accuracy, while removing smoothing from SICAF decreases accuracy.The authors say smoothing helps denoise after contrast enhancement; without it, high-contrast noise may be mistaken for valid motion.Source: §5.4 · p. 9, Table 3 · p. 10

5.5 Ablation on Internal Components of SICAF

• Uniform compression decreases accuracy, and removing either compact compression or the temporally sensitive entropy constraint HTα(Z) degrades prediction accuracy.The authors describe compact compression as suppressing redundant information and HTα(Z) as preserving critical temporal information.Source: §5.5 · p. 10, Table 4 · p. 10

• SICAF combines learnable temporal attention with temporally modulated constrained optimization to extract compact representations while preserving temporal heterogeneity in event streams.The conclusion presents this as a pathway toward more efficient feature extraction and enhanced SNN robustness.Source: §6 · p. 10

• The paper reports no negative societal impacts.It describes its focus as improving compactness and diversity in SNN feature extraction for optical flow.Source: §A.1 · p. 12

• In practical applications, SICAF may require parameter tuning to achieve better accuracy, which could be time-consuming.The passage also says SICAF addresses information loss associated with traditional information bottleneck methods.Source: §A.2 · p. 12

A.3 Experiment Settings

• MVSEC is evaluated against ground truth, while ECD and HQF use contrast-maximization-based metrics because they lack ground truth.Predictions are produced at each frame timestep with dt=2 for MVSEC.Source: §A.3 · pp. 12–13
• Black-box attacks use random-direction zeroth-order gradient estimation, with five queries and perturbation magnitude ϵ=0.1.Inputs are formed by adding Gaussian random directions and clipped to [0, 1].Source: §A.3 · p. 13
• White-box robustness is tested with FGSM using perturbation magnitude ϵ=0.1.The perturbation follows the sign of the loss gradient with respect to the input.Source: §A.3 · p. 13

A.4 Additional Details on SNN Neurons

• The LIF membrane-potential threshold m and leakage parameter λ are learnable, and the neuron is adjusted to adapt more quickly to changing motion.The stated motivation is handling complex motion changes in optical flow estimation.Source: §A.4 · p. 13
• Table 5 lists SNN hyperparameters and their initialization ranges.The table is titled “SNN hyperparameter initialization table.”Source: Table 5 · p. 13, §A.4 · p. 13

A.5 Pseudocode

• The training network uses four strided-convolution encoders, two residual blocks, and four decoder layers, while adaptive-neuron microscopic parameters adjust during training.SICAF training maximizes conditional entropy of intermediate variables to balance latent compactness and diversity.Source: §A.5 · pp. 13–14

A.6 Implementation Details of the SICAF Objective

• For objective implementation, training uses the encoder mean as the bottleneck representation and a KDE-based mean surrogate rather than an exact KL evaluation.The RBF entropy kernel uses fixed bandwidth σker = 1.0; β = 4.8 × 10^-3 and γ = 4 × 10^-4.Source: §A.6 · pp. 14–15

A.7 Theoretical Guarantees of SICAF

• Under Theorem 1’s assumptions, SICAF’s generalization bound accounts for latent mutual information, temporal-attention variability, and finite-sample entropy-estimation error.A smaller information budget reduces the latent-representation complexity term, while more stable temporal attention tightens the bound.Source: §A.7 · p. 16
• Theorem 2’s sample-complexity guarantee retains temporal-modulation terms, requiring attention fluctuation and entropy-estimation error to fit within an error budget.The mutual-information-only expression is recovered when the temporal contribution is zero or negligible relative to the target tolerance.Source: §A.7 · p. 17

A.8 Additional Self-Supervised Optical Flow Estimation Results

• On ECD and HQF, SICAF is reported to improve self-supervised optical-flow generalization, with RSAT below 1 indicating better performance than null-vector flow.The authors attribute the generalization improvement to maximum-entropy information bottleneck regularization of intermediate variable T.Source: §A.8 · p. 18, Figure 6 · p. 18, Table 6

A.9 Additional Details on Robustness Experiments

• Robustness tests evaluate models on ECD and HQF with Gaussian and salt-and-pepper noise, plus white-box FGSM and black-box zero-order adversarial attacks.The supplied passages describe the attack procedures but do not provide the Table 7 or Table 8 outcome values.Source: §A.9 · pp. 18–19

A.10 Computational Efficiency Analysis

• SICAF reduces mean inference latency from 29.764 to 27.164 ms/batch and mean inference energy from 4.354 to 3.873 J/batch, while memory usage increases from 241.548 to 247.503 MiB.The mean number of spikes per inference batch also decreases by approximately 57.5%.Source: Table 9 · p. 20, §A.10 · p. 20

NeurIPS Paper Checklist

• The authors say the Appendix provides experimental details, parameter settings, and method pseudocode, and that code will be released for experiments on public benchmark datasets.They also report providing a detailed experimental setup and specify GeForce RTX 4090 and A6000 GPUs.Source: NeurIPS Paper Checklist · pp. 22–24

10. Broader impacts

• The authors mark societal impacts and release safeguards as not applicable, stating that the work has no negative societal impacts and that its model and benchmark data pose no such risk.Source: §10 · pp. 24–25
• The authors state that they credit and respect the licenses of existing code and datasets, release no new assets, and conduct no human-subject research or core-method development involving LLMs.Source: §10 · pp. 25–27