PaperlayerView exact revision on arXiv

Source-linked AI summary

SpikeSSL: A Universal Spike Inference Framework with Dynamics-Informed State-Space Layers

Chenghao Yue, Siming Xing, Shuran Liu, Angran Li, Yuanlong Zhang

arXiv:2610.11456v1cs.AI

TL;DR

Accurate spike inference across calcium indicators remains difficult because their fluorescence dynamics vary and common supervised architectures lack dynamics-matched inductive bias. SpikeSSL combines dynamics-informed state-space layers, conditioning, and simulation-augmented training. On five fixed splits from 33 public ground-truth datasets, it achieves state-of-the-art in-domain and zero-shot performance.

  • Problem

    Accurate spike inference across indicators remains difficult because indicator-specific kinetics and signal statistics vary, while common supervised backbones lack dynamics-matched inductive bias.

    Source: §1 · pp. 1–2

  • Method

    SpikeSSL combines bidirectional IIR state-space layers with multimodal conditioning and augments training using biophysically simulated traces.

    Source: §1 · p. 2

  • Results

    On five fixed evaluation splits built from 33 public ground-truth datasets, SpikeSSL achieves state-of-the-art performance in in-domain and zero-shot settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A single model generalizes across indicator types without retraining or target-domain ground truth.

    Source: §5 · p. 9

    Scarcity of paired real recordings still limits cross-indicator generalization.

    Source: §1 · p. 2

Abstract

from arXiv · show

Two-photon calcium imaging is a standard tool for recording large neural populations in vivo, yet inferring spikes accurately across the growing diversity of calcium indicators remains an open problem. Existing supervised methods achieve reasonable in-domain accuracy but generalize poorly to unseen indicators, because different indicators induce distinct fluorescence kinetics and signal statistics while existing architectures remain relatively simple generic temporal regressors without dynamics-matched inductive bias. We propose SpikeSSL, a universal spike inference framework whose temporal backbone is a bank of bidirectional IIR state-space layers broadly motivated by calcium dynamics. A multi-modal conditioning encoder maps indicator identity, sampling rate, and trace-level signal statistics into a global conditioning vector that modulates the backbone via Adaptive Layer Normalization, while a heteroscedastic variance head provides calibrated per-frame uncertainty. On a benchmark with five fixed evaluation splits built from 33 public ground-truth datasets, SpikeSSL achieves state-of-the-art performance in both in-domain and zero-shot leave-one-indicator-out settings. We also develop a biophysical simulation pipeline capable of generating paired fluorescence-spike traces with systematically varied kinetic parameters, spike statistics, response nonlinearities, baseline drift, and noise. Using this pipeline, we synthesize approximately 11,000 simulated traces. Augmenting training with these data effectively closes the cross-indicator domain gap and improves zero-shot generalization. Code is publicly available at https://github.com/detimage123/SpikeSSL.

1 Introduction

SpikeSSL addresses indicator-dependent fluorescence dynamics and generic temporal backbones with a dynamics-informed architecture and simulation-augmented training. It achieves state-of-the-art performance across in-domain and zero-shot settings, while simulated traces improve zero-shot transfer.

1 Introduction

  • Indicator-specific fluorescence dynamics and generic temporal regressors without calcium-matched inductive bias make inference on unseen indicators difficult.Source: §1 · pp. 1–2
  • SpikeSSL uses bidirectional IIR layers motivated by calcium decay, multimodal conditioning, and a heteroscedastic variance head for calibrated uncertainty.Source: §1 · p. 2
  • SpikeSSL achieves state-of-the-art performance in both in-domain and zero-shot LOIO settings on five fixed evaluation splits built from 33 public ground-truth datasets.Source: §1 · p. 2
  • Approximately 11,000 simulated traces spanning indicator kinetic regimes effectively close the cross-indicator gap in zero-shot transfer.Source: §1 · p. 3

2 Related Work

Prior spike-inference methods include explicit calcium-dynamics models and data-driven regressors, while cross-indicator transfer remains challenged by uneven recording coverage. Simulation-based augmentation offers a way to expand kinetic coverage where paired recordings are absent.

2.1 Spike Inference from Calcium Imaging

  • Model-based methods invert explicit calcium dynamics but require indicator-dependent kinetic parameters, while data-driven methods use learned representations and often generic temporal backbones.Source: §2.1 · p. 3

2.2 Cross-Domain Generalization

  • Existing methods can degrade on unseen indicators, and paired ground-truth coverage remains uneven across indicator families and experimental settings.Source: §2.1 · p. 3, §2.2 · p. 3
  • Simulation-based augmentation can extend training across systematic kinetic grids where real paired recordings are absent, a direction little explored in spike inference.Source: §2.2 · p. 3

3 Methodology

SpikeSSL combines a calcium-dynamics-informed temporal backbone, multimodal conditioning, per-frame uncertainty estimation, and biophysical simulation for cross-indicator spike inference.

3.1 Overall Architecture

  • The network processes fluorescence traces through stacked SpikeSSL blocks, then refinement layers and separate spike-rate, spiking-gate, and uncertainty heads.The blocks combine bidirectional IIR scans, Temporal Mixers, and SwiGLU feed-forward networks.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Dynamics-Informed IIR Backbone

  • Bidirectional IIR scans use a multi-scale bank of decay rates to represent fast indicator transients and slow trends.The rates are initialized to span timescales from a few frames to hundreds of frames, covering benchmark indicator kinetics.Source: §3.2 · p. 4

3.3 Multi-Modal Conditioning

  • A conditioning encoder fuses indicator identity, sampling rate, and trace statistics to modulate temporal processing through AdaLN and decay-rate shifts.The deterministic trace statistics act as proxies for signal-to-noise ratio and kinetics, allowing adaptation even when indicator identity is unspecified.Source: §3.3 · p. 5

3.4 Heteroscedastic Uncertainty Estimation

  • A variance head estimates per-frame uncertainty, with inflated variance on zero-shot unseen indicators serving as an out-of-distribution signal.The head projects shared backbone features and adds an indicator-specific bias.Source: §3.4 · p. 5

3.5 Biophysical Simulation Data Generation

  • The biophysical simulator augments empirical recordings with synthetic traces spanning indicator kinetics and firing regimes.It synthesizes spikes, generates calcium transients, and models observation noise; its mechanisms include response variation, baseline drift, and measurement noise.Source: §3.5 · pp. 5–6, Figure 3 · p. 6

3.6 Training Objective

  • Training combines heteroscedastic reconstruction, structural penalties, and a separate binary gate-detection loss.The reconstruction uses precision-weighted Huber loss; structural terms penalize overprediction, missed peaks, and mean-rate bias, while the gate uses cross-entropy.Source: §3.6 · p. 6

4 Experiments

Across in-domain and zero-shot evaluation, SpikeSSL performs strongly on five fixed splits spanning diverse datasets and held-out indicators. Ablations and additional analyses examine its conditioning, dynamics-informed backbone, uncertainty estimates, and response scale.

4.1 Datasets

  • The evaluation corpus contains 1,960 paired neuron recordings from 33 datasets, spanning 11 indicators, two species, multiple brain regions, spinal cord, and diverse cell types.Biophysically simulated traces are used exclusively during training.Source: §4.1 · p. 7

4.2 Benchmark and Metrics

  • Five fixed splits evaluate in-domain performance and five leave-one-indicator-out conditions, with indicator decay constants ranging from about 50 ms to 600 ms.LOIO evaluation withholds each target condition from training; metrics include per-neuron Pearson correlation, normalized error, and mean bias.Source: §4.2 · p. 7

4.6 Ablation Studies

  • Under LOIO, ablations assess AdaLN versus concatenation, the signal-statistics branch, the bidirectional IIR module, the variance head, and the binary gate head.The reported mean correlation is across all five held-out splits; the authors report that each of these components contributes to cross-indicator adaptation or predictive quality.Source: Table 4 · p. 8, §4.6 · p. 9

4.7 Discussion

  • Figure 5 examines uncertainty calibration, while Figure 6 reports lower deviation from linearity and lower error across flat, sparse, and dense activity windows for SpikeSSL.The variance head’s predicted uncertainty increases under LOIO shift and higher predicted standard deviation is associated with larger absolute error; this calibration analysis is limited to the five-split LOIO benchmark.Source: Figure 5 · p. 8, §4.7 · p. 9, Figure 6 · p. 9

4.3 Implementation Details

  • The model uses eight blocks with dimension 192 and conditioning dimension 64, converges within 20 epochs, and assigns a large gate-loss weight to suppress false positives in silent periods.Training uses Adam with an initial learning rate of 10^-3.Source: §4.3 · p. 8

4.4 In-Domain Results; 4.5 Zero-Shot LOIO Results

  • SpikeSSL achieves state-of-the-art in-domain performance and the highest mean correlation across all five held-out LOIO splits.In LOIO, simulation-augmented training further improves zero-shot performance across held-out splits, especially where real training data sparsely cover the kinetics.Source: §4.4 · p. 8, §4.5 · p. 8

4.7 Discussion

  • Scaling CASCADE and ENS2 improves their performance but does not close the gap to SpikeSSL, while a larger SpikeSSL variant adds no gains.The authors attribute SpikeSSL’s advantage to its IIR backbone, indicator-aware conditioning, and uncertainty-aware objective rather than parameter count alone.Source: Table 5 · p. 9, §4.7 · p. 9

5 Conclusion

SpikeSSL combines a multi-scale bidirectional IIR backbone with multi-modal conditioning to narrow cross-indicator generalization gaps. On five fixed splits it achieves state-of-the-art in-domain and zero-shot performance, and a single model can generalize across indicator types without retraining.

5 Conclusion

  • A multi-scale bidirectional IIR backbone and multi-modal conditioning provide structured inductive bias, while simulation-augmented training bridges kinetic gaps for unseen indicators.On five fixed splits built from public ground-truth datasets, SpikeSSL achieves state-of-the-art performance in both in-domain and zero-shot settings without target-domain ground truth.Source: §5 · p. 9
  • Generalization across indicator types without retraining can reduce ground-truth data-collection burden and facilitate adoption of new calcium indicators.The conclusion presents these as potential practical benefits of a single generalizing model.Source: §5 · pp. 9–10

A Supplementary Material

The supplement consolidates technical details and extended validation across architecture, datasets, simulation, benchmark construction, and additional analyses.

A. Supplementary Material

  • Appendix A.1 expands architecture modules, A.2 documents recording sets and fixed splits, A.4 specifies the simulation pipeline, and A.5 details benchmark construction.Other subsections cover conditioning, domain breadth, data scaling, ablations, adaptation regimes, learned dynamics, simulation, and robustness.Source: A Supplementary Material · p. 12

A.1 Supplementary architecture modules

The supplementary architecture description expands the conditioning encoder and other modules within each SpikeSSL block. The encoder maps indicator identity, sampling rate, and signal statistics into a global conditioning vector.

A.1 Supplementary architecture modules

  • This subsection expands three modules named in the main text but not visualized in Figure 2: the conditioning encoder, dilated refinement module, and Temporal Mixer and SwiGLU FFN sub-blocks within each SpikeSSL block.Source: §A.1 · p. 12
  • Three branches encode indicator identity, log sampling rate, and seven trace statistics before their outputs are fused into a 64-dimensional global conditioning vector.The statistics are standard deviation, skewness, excess kurtosis, and autocorrelation at lags 1, 3, 5, and 10; unknown indicators use ID 0.Source: Figure 7 · p. 12

A.1.1 Conditioning encoder internals

The conditioning encoder combines indicator identity, sampling rate, and trace statistics into a global vector through separate modality-specific branches. These branches are fused only after each produces a 64-dimensional representation.

A.1.1 Conditioning encoder internals

  • The indicator branch embeds 14 IDs, reserving ID 0 for unknown or unseen indicators, then projects each embedding to 64 dimensions.Named indicators include OGB-1, GCaMP6 variants, R-CaMP, jRCaMP1a, jGECO1a, and XCaMPgf.Source: §A.1.1 · p. 12
  • The sampling-rate branch maps log fs through a two-layer, width-64 MLP and uses log 30 when frame rate is unavailable.Source: §A.1.1 · p. 12
  • A separate width-64 MLP processes seven trace statistics; the three branch outputs are concatenated and fused by a 192 →64 →64 MLP into c ∈R64.The descriptor includes standard deviation, skewness, excess kurtosis, and autocorrelation at lags 1, 3, 5, and 10.Source: §A.1.1 · p. 12

A.1.2 Dilated refinement module

A residual dilated convolution stack follows the conditioned blocks and final LayerNorm, complementing the bidirectional IIR scan with local-to-medium-range refinement at full temporal resolution.

A.1.2 Dilated refinement module

  • Three residual refinement layers use dilation factors 1, 2, and 4, expanding the receptive field without downsampling.Each uses a kernel-3 1D convolution, batch normalization, GELU, dropout, and residual addition.Source: §A.1.2 · p. 12, Figure 8 · p. 13
  • The refinement stack provides local correction as a lightweight post-backbone smoother and sharpener, complementing the IIR scan’s long-range indicator-specific decay.Source: §A.1.2 · pp. 12–13

A.1.3 Temporal Mixer and SwiGLU FFN inside each block

Each SpikeSSL block pairs an AdaLN-conditioned bidirectional IIR layer with unconditioned temporal and channel-mixing modules. This confines indicator-conditioned modulation to the branch associated with indicator-specific decay kinetics.

A.1.3 Temporal Mixer and SwiGLU FFN inside each block

  • The unconditioned Temporal Mixer applies LayerNorm, depthwise kernel-width-7 and pointwise convolutions, GELU, dropout, and a residual addition for local temporal mixing.Source: §A.1.3 · p. 13
  • The unconditioned SwiGLU FFN splits a doubled-width projection into gate and value tensors, applies SiLU(gate) ⊙value, then projects back to D residually.The default D=192 and expansion factor 2 give intermediate width 384.Source: §A.1.3 · p. 13
  • AdaLN modulation is confined to the IIR branch, where indicator-specific decay kinetics have the clearest physical interpretation.Source: §A.1.3 · p. 13

A.2 Benchmark data composition and split definitions

The benchmark draws on 33 ground-truth recording sets spanning diverse indicators and acquisition contexts, with recording-level separation and five fixed evaluation splits. Its composition tests variation beyond nominal sensor identity.

A.2 Benchmark data composition and split definitions

  • The benchmark combines chemical dyes and genetically encoded indicators across species, anatomical regions, induction methods, and sampling rates of approximately 2.5–500 Hz, testing acquisition and biological-context variation beyond sensor identity.Recordings span cortical, hippocampal, olfactory, and spinal preparations collected by multiple laboratories.Source: §A.2 · p. 13
  • Recording-level separation prevents overlapping windows from the same acquisition from being divided between training and evaluation.Source: §A.2 · p. 13, Table 8 · p. 15
  • The benchmark groups ground-truth recordings by indicator family and specifies five fixed evaluation splits; under LOIO, source-side recordings are excluded from training.Evaluation recordings are held out in all settings.Source: §A.2 · pp. 13–14, Table 7 · p. 14, Table 8 · p. 15

A.3 Loss function ablation under +sim LOIO

The appendix evaluates SpikeSSL’s loss-function ablation and reports additional simulation, benchmark, transfer, and robustness analyses. It also examines artifact sensitivity and sliding-window inference across acquisition settings.

A.3 Loss function ablation under +sim LOIO

  • Each added loss term improves mean correlation on all five +sim LOIO splits, and the full three-component loss scores highest on every split.The compared runs use identical data and hyperparameters, differing only in their active loss terms.Source: §A.3 · p. 15, Table 9 · p. 15

A.4.1 Spike Time Simulation

  • Spike times are simulated either from stochastic inter-spike-interval models spanning standard, refractory, quasi-periodic, and bursty firing, or from Izhikevich membrane dynamics.The Izhikevich approach uses multi-scale synaptic drive and reset dynamics to represent distinct neuronal subclasses.Source: §A.4.1 · pp. 15–16

A.4.2 Fluorescence Signal Simulation

  • The fluorescence simulator models spike-dependent calcium influx, indicator binding, nonlinear response, baseline drift, and measurement noise.Calcium influx weights vary with stochastic amplitude and depletion with recovery; binding kinetics accommodate GECIs and dyes, and the observed signal combines dynamic bound indicator, drift, and additive noise.Source: §A.4.2 · pp. 16–17

A.5 Detailed benchmark construction; A.6.1 Representative transfer functions; A.6.2 Long-trial LOIO visualizations

  • Fixed recording-set partitions define in-domain and LOIO evaluation; transfer curves show SpikeSSL better preserves gain across firing rates, and long-trial visualizations show it tracks bursts and sustained firing across indicators.The benchmark uses raw fluorescence without offline denoising or resampling, and simulation augmentation is restricted to training. The transfer-function analysis reports that OASIS and MLSpike underpredict moderate and high rates, while CASCADE and ENS2 retain broader inter-neuron variability.Source: §A.5 · p. 17, §A.6.1 · pp. 17–18, Figure 9 · p. 18

A.7 Test-Time Conditioning Uses No Target-Domain Pool

  • LOIO conditioning uses an unknown indicator ID and per-window statistics, without target labels, pooled target windows, or parameter updates; removing the ID branch changes mean LOIO correlation from 0.654 to 0.647.Removing the ID branch reduces in-domain correlation from 0.741 to 0.703, while shuffling the observed ID yields 0.578; zero-shot transfer is supported by sampling rate, window statistics, and the trace.Source: §A.7 · p. 20, Table 11 · p. 21

A.8 Breadth of the Universal Benchmark; A.9 Data Scaling and Statistical Robustness; A.10 Ablations

  • The benchmark spans substantial kinetic and SNR variation, while data quantity and coverage improve LOIO performance and the reported ablations show contributions from each major model component.Mean decay ranges from 0.050 s to 0.261 s and mean SNR from 3.886 to 11.627; across five seeds, mean LOIO correlation is 0.660 ± 0.016 and in-domain correlation is 0.739 ± 0.004.Source: §A.8 · p. 21, Table 12 · p. 21, §A.9 · p. 22

A.11 Universal, Specific, and Few-Shot Regimes; A.12 Disentangling Indicator and Acquisition Shifts

  • SpikeSSL benefits from pooling indicator families, exceeds its zero-shot value with five labeled support neurons, and gives the best correlation in the tested species, cell-type, and tissue shifts.Few-shot fine-tuning reaches 0.719 at 20 support neurons; mini-domain tests vary one factor while holding the indicator fixed.Source: §A.11 · pp. 22–23, Table 17 · p. 23, §A.12 · p. 23

A.13 Dynamics Readouts of the Conditioned IIR Bank; A.14 Simulation: Benefits and Failure Modes; A.15 Threshold, Artifact, and Runtime Sensitivity

  • The learned dynamics readouts vary by indicator, simulation alignment gains differ across targets, and the inference-gate threshold maximizing mean correlation is τ = 0.5.ACF τ is positively associated with measured τ (Spearman ρ = 0.886, p = 5.6 × 10−5). SC-GCaMP6s has the largest simulation gain; Other-RCaMP’s −0.012 change is within five-seed variability and does not establish harmful negative transfer. Normalized error is smallest at τ = 0.6 and absolute bias at τ = 0.7.Source: §A.13 · pp. 23–24, Table 19 · p. 24, §A.14 · p. 24

A.15 Threshold, Artifact, and Runtime Sensitivity

  • Baseline drift is SpikeSSL’s dominant artifact-related degradation, while sparse motion is milder; standard motion correction and baseline detrending are recommended.Per-window statistics do not make the model invariant to these perturbations.Source: Table 22 · p. 25, §A.15 · p. 25
  • On an A100, one forward pass is approximately 29 times faster than the arrival of a 30-Hz frame window, but inference remains noncausal within each window.Sliding-window inference avoids holding an entire recording in memory and can keep pace with many acquisition settings.Source: §A.15 · p. 25
Loading 2610.11456v1…