Source-linked AI summary
SCINTILLA-SNN: A Spiking Multi-Scale Selective Aggregation Network for Perineural Invasion Prediction
Youngung Han, Yului Jeong, Kyeonghun Kim, Dohyun Kweon, Suah Park, Hyunsu Go, Sungha Park, Anna Jung, Jinyong Jun, Yunho Choe, Yunjin Seo, Ken Ying-Kai Liao, Hyuk-Jae Lee, Nam-Joon Kim
TL;DR
Preoperative MRI prediction of PNI in CCA is difficult because relevant cues are subtle, sparse, and localized, while dense volumetric models are computationally costly. SCINTILLA-SNN uses hierarchical spiking extraction and selective multi-scale aggregation, achieving AUROC 0.748 and a 23.18× reduction in estimated energy versus dense MAC-only computation.
Problem
Preoperative MRI-based PNI prediction in CCA is clinically meaningful but difficult because PNI-related cues are subtle, sparse, and localized, while dense volumetric processing is energy-intensive.
Method
SCINTILLA-SNN combines a four-stage hierarchical spiking backbone with MSSA, which uses content values and firing-rate and membrane-potential cues to selectively aggregate diagnostic tokens.
Results
AUROC 0.748 and 23.18× lower estimated inference energy than dense MAC-only computation were achieved on a retrospective cohort of 182 CCA patients.
Takeaways & Limitations
SCINTILLA-SNN achieved the best accuracy–energy trade-off among evaluated models for MRI-based PNI prediction in the studied CCA cohort.
Takeaways & Limitations
External, multiinstitutional validation is needed to assess generalizability across imaging protocols and clinical settings.
Abstract
from arXiv · showhide
Preoperative prediction of perineural invasion (PNI) in cholangiocarcinoma (CCA) is clinically valuable but remains challenging because PNI-related cues on magnetic resonance imaging (MRI) are subtle, sparse, and spatially localized around the tumor boundary. Standard 3D CNN and transformer architectures process volumetric data in a dense or spatially uniform manner, which can dilute subtle PNI-related evidence while requiring a large number of multiply-accumulate operations over 3D feature grids. To address these limitations, we propose SCINTILLA-SNN, a 3D spiking network composed of a four-stage hierarchical backbone and a Multi-Scale Spike Aggregation (MSSA) module for PNI prediction. The backbone extracts hierarchical volumetric representations through spiking convolutional stages and local spike window modulation stages. Given the resulting stage-wise representations, MSSA maps each spatial token to a learnable content value and modulates it with a spike-dynamics gate derived from firing rate and timestep-wise membrane-potential variability. The resulting score, referred to as the diagnostic token score, is used to selectively aggregate sparse PNI-related evidence. Experiments on a 10-year retrospective cohort of 182 CCA patients show that SCINTILLA-SNN achieves an AUROC of 0.748 under 5-fold cross-validation, while reducing the estimated inference energy by 23.18$\times$ compared with dense MAC-only computation of the same network.
I. INTRODUCTION
SCINTILLA-SNN addresses the difficulty of detecting subtle, sparse, peripheral PNI cues in MRI while reducing the inefficiency of dense volumetric processing. It combines hierarchical spiking representations with selective multi-scale aggregation guided by spike dynamics.
- MRI-based PNI prediction is clinically meaningful but difficult because PNI-related findings are subtle, sparse, and localized near the tumor periphery.
- Dense or spatially uniform volumetric processing may weaken localized PNI signals while increasing MAC-based inference energy.
- SCINTILLA-SNN combines a four-stage spiking backbone with MSSA to selectively aggregate diagnostic evidence across multiple feature scales.
- MSSA uses learnable content values and spike-dynamics cues to select informative tokens instead of uniformly pooling volumetric spike features.
- The architecture processes a two-channel T2W volume through four spiking stages, with MSSA aggregating outputs from stages s2, s3, and s4.
- On a retrospective cohort of 182 patients, SCINTILLA-SNN achieved the highest AUROC among evaluated baselines, with ablations supporting its gating and selective aggregation design.
II. METHOD
SCINTILLA-SNN maps a two-channel MRI volume to binary PNI logits through direct timestep-wise processing, a spiking patch stem, a hierarchical backbone, MSSA, and an MLP classifier. Its LIF neurons integrate inputs, emit thresholded spikes, and reset membrane potentials after firing.
- SCINTILLA-SNN maps a two-channel input volume consisting of normalized T2W intensity and tumor-mask channels to binary PNI logits.
- The network processes the volume over T timesteps through a 3D spiking patch stem, four-stage backbone, MSSA, and MLP classifier.
- The LIF update integrates the previous membrane potential and input, emits a Heaviside-thresholded spike, and applies a hard reset.
- The membrane decay factor β, firing threshold vth, and Heaviside function H define the neuron’s integration, firing, and reset behavior.
- Pre-reset membrane potentials are retained for MSSA to compute membrane-potential variability as a spike-dynamics cue.
C. 3D Hierarchical Spiking Backbone
The hierarchical backbone combines residual spiking convolutions in early stages with local spike-window modulation and spiking MLP blocks in later stages. These stages produce multi-scale outputs for downstream aggregation.
- C. 3D Hierarchical Spiking Backbone: Stages 1–2 use residual spiking convolution blocks, while stages 3–4 use local spike-window modulation blocks followed by spiking MLP processing.
- C. 3D Hierarchical Spiking Backbone: Stage 2 downsamples with stride 2 and produces s2, while stages 3 and 4 further downsample and produce s3 and s4 with 4C and 8C channels.
- C. 3D Hierarchical Spiking Backbone: Within local 3D windows, convolutional projections and LIF neurons generate spike-form Q, K, and V for lightweight element-wise modulation.
- C. 3D Hierarchical Spiking Backbone: Spike-window modulation avoids dense softmax attention, then combines the modulated features with residual addition and LIF activation.
E. Multi-Scale Spike Aggregation
MSSA converts multi-scale stage outputs into diagnostic token scores by combining learnable content values with firing-rate and membrane-potential variability cues, then selectively aggregates the highest-scoring tokens.
- E. Multi-Scale Spike Aggregation: MSSA assigns each spatial token a learnable content value and combines it with spike-dynamics cues to form a diagnostic token score.The module operates on stage outputs s2, s3, and s4, using separate content and detached cue paths.
- E. Multi-Scale Spike Aggregation: Firing rate and timestep-wise membrane-potential variability provide complementary token-selection cues after normalization across spatial tokens.TV uses pre-reset membrane potentials to capture real-valued fluctuations complementary to binary-spike-derived FR.
- E. Multi-Scale Spike Aggregation: The spike-dynamics gate uses stage-specific learnable parameters and separately weighted FR and TV contributions to modulate normalized content values.The sigmoid gate shares parameters across tokens within each stage, while λFR and λTV control cue contributions.
- E. Multi-Scale Spike Aggregation: Top-k pooling retains the tokens with the largest diagnostic scores and aggregates them using score-weighted softmax pooling.The resulting per-stage evidence vectors are projected by per-stage heads and concatenated after normalization.
F. Objective Function
The training objective combines class-weighted focal loss with label smoothing and regularizers for sparse firing and weight control.
- F. Objective Function: The objective combines class-weighted focal loss with label smoothing and two lightweight regularizers.The classifier produces two-class logits before applying the combined training objective.
- F. Objective Function: The firing regularizer applies an L1 penalty to mean stage firing activity, while the weight regularizer applies standard L1 weight penalization.The firing penalty is computed from the spike-output stacks of Stages 2–4 and encourages sparse firing.
A. Dataset and Implementation Details
The study uses a single-center retrospective cohort of 182 histologically confirmed CCA patients and trains models with specified preprocessing, augmentation, and optimization settings.
- A. Dataset and Implementation Details: The dataset contains 182 CCA patients, including 112 PNI-negative and 70 PNI-positive cases, with expert-delineated tumor and liver masks.The cohort was collected over more than a decade and has a 38.5% positive-class proportion.
- A. Dataset and Implementation Details: MRI volumes were resampled to 1.5625×1.5625×4.0 mm spacing and cropped to tumor-centered regions of 96×96×48 voxels.The input used T2-weighted MRI with the study’s constructed channel representation.
- A. Dataset and Implementation Details: Models were trained for 100 epochs per fold with batch size 2 using AdamW, cosine annealing, gradient clipping, mixed precision, and limited augmentation.Augmentation consisted of axial flips applied to image and mask channels and image-only intensity jitter.
- A. Dataset and Implementation Details: SCINTILLA-SNN used C=32, T=4 LIF timesteps, dropout 0.3, and stage-specific spike-window settings, with MSSA top-k values of 512, 256, and 128.The corresponding top-k selections were specified for stages s2, s3, and s4.
- A. Dataset and Implementation Details: Training used class-weighted focal loss with γ=2.0, label smoothing of 0.05, and firing-rate and weight regularization coefficients of 10^-3 and 10^-7.The LIF settings included β=0.5, vth=1.0, and surrogate steepness αs=4.0.
B. Main Results
Under 5-fold cross-validation, SCINTILLA-SNN achieves the strongest reported accuracy–energy trade-off, while MSSA-selected evidence concentrates more near the tumor contour than stage-wise Grad-CAM responses.
- B. Main Results: 0.748 AUROC: SCINTILLA-SNN achieves the highest AUROC with 2.83M parameters and lower operation-level energy than transformer baselines.The comparison reports higher AUROC than CNN and transformer baselines, while pure spiking baselines use less energy but achieve lower AUROC.
- B. Main Results: MSSA-selected evidence is more concentrated near the tumor contour than the diffuse responses in stage-wise Grad-CAM maps for a representative PNI-positive case.The visualization supports selective extraction of sparse, tumor-associated tokens from multi-scale features.
- B. Main Results: Best accuracy–energy trade-off: SCINTILLA-SNN combines the strongest evaluated discrimination with operation-level efficiency.The paper’s main comparison is conducted under 5-fold cross-validation.
C. Qualitative Visualization
MSSA concentrates selected evidence near the tumor contour, while ablations support spike-dynamics gating and selective aggregation; the energy analysis reports substantial savings under conservative accounting.
- C. Qualitative Visualization: MSSA-selected evidence is more concentrated near the tumor contour than stage-wise Grad-CAM responses in a representative PNI-positive case.This supports selecting sparse, tumor-associated tokens from multi-scale features.
- D. Ablation Study: Removing firing-rate input causes the largest AUROC drop, while membrane-potential variability provides a smaller complementary improvement over content-only gating.The full model outperforms both single-cue variants, indicating that firing rate and membrane-potential variability jointly refine token selection.
- D. Ablation Study: Replacing top-k selection with global average pooling further degrades performance, supporting selective aggregation for sparse PNI-related features.
- E. Energy Analysis: 23.18× lower estimated energy is achieved than dense MAC-only computation, requiring 35.99 mJ instead of 834.10 mJ under conservative auxiliary-as-MAC accounting.The estimate includes 1.82G MACs, 17.53G AC/SOPs, and 2.58G auxiliary operations.
IV. CONCLUSION
SCINTILLA-SNN combines hierarchical volumetric spiking extraction with MSSA-based selective multi-scale aggregation for MRI-based PNI prediction. It achieves the best accuracy–energy trade-off among evaluated models on a single-center CCA cohort, while requiring external multi-institutional validation for generalizability.
- IV. CONCLUSION: SCINTILLA-SNN combines hierarchical volumetric spiking feature extraction with MSSA-based selective multi-scale aggregation for MRI-based PNI prediction.
- IV. CONCLUSION: The model achieves the best accuracy–energy trade-off among evaluated models on a single-center CCA cohort.
- IV. CONCLUSION: External, multi-institutional validation is needed to assess generalizability across imaging protocols and clinical settings.