PaperlayerView exact revision on arXiv

Source-linked AI summary

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding

Yuchen Zhou, Jiamin Wu, Zichen Ren, Zhouheng Yao, Weiheng Lu, Kunyu Peng, Qihao Zheng, Chunfeng Song, Wanli Ouyang, Chao Gou

arXiv:2506.23075v1cs.HCcs.LGeess.SPq-bio.NC

TL;DR

EEG foundation models commonly use scale-agnostic tokens and dense attention despite the cross-scale structure of neural activity. CSBrain combines cross-scale tokenization with structured sparse attention; across 11 tasks and 16 datasets, it outperforms task-specific and foundation baselines.

  • Problem

    EEG foundation models commonly use scale-agnostic tokenization and dense attention, although decoding tasks involve diverse cross-scale spatiotemporal patterns.

    Source: §1 · p. 2

  • Method

    CSBrain alternates Cross-scale Spatiotemporal Tokenization (CST), which forms compact scale-aware tokens, with Structured Sparse Attention (SSA) across temporal windows and brain regions.

    Source: §1 · pp. 2–3

  • Results

    Across 11 EEG tasks and 16 datasets, CSBrain outperforms task-specific and foundation baselines; its macro-average overall score exceeds CbraMod by 3.35%.

    Source: §1 · p. 3, §3.3 · p. 8

  • Takeaways & Limitations

    The findings establish cross-scale spatiotemporal modeling as a key inductive bias for generalized EEG decoding.

    Source: §1 · p. 3, §4 · p. 10

    Computational resources constrain model- and data-scaling exploration, preventing empirical assessment at the scale of vision-language or language-only models.

    Source: §G.2 · p. 36

Abstract

from arXiv · show

Understanding and decoding brain activity from electroencephalography (EEG) signals is a fundamental challenge in neuroscience and AI, with applications in cognition, emotion recognition, diagnosis, and brain-computer interfaces. While recent EEG foundation models advance generalized decoding via unified architectures and large-scale pretraining, they adopt a scale-agnostic dense modeling paradigm inherited from NLP and vision. This design neglects a core property of neural activity: cross-scale spatiotemporal structure. EEG task patterns span a wide range of temporal and spatial scales, from short bursts to slow rhythms, and from localized cortical responses to distributed interactions. Ignoring this diversity leads to suboptimal representations and weak generalization. We propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain introduces: (i) Cross-scale Spatiotemporal Tokenization (CST), which aggregates multi-scale features from localized temporal windows and anatomical brain regions into compact scale-aware tokens; and (ii) Structured Sparse Attention (SSA), which captures cross-window and cross-region dependencies, enhancing scale diversity while removing spurious correlations. CST and SSA are alternately stacked to progressively integrate multi-scale dependencies. Experiments on 11 EEG tasks across 16 datasets show that CSBrain consistently outperforms task-specific and foundation model baselines. These results establish cross-scale modeling as a key inductive bias and position CSBrain as a robust backbone for future brain-AI research.

1 Introduction

Task-specific EEG models generalize poorly, while foundation models use scale-agnostic tokens and dense attention that mismatch EEG’s cross-scale structure. CSBrain combines scale-aware tokenization and structured sparse attention, outperforming task-specific and foundation baselines across diverse decoding tasks.

1 Introduction

  • Task-specific EEG models are tightly coupled to datasets and task formats, limiting their generalizability and scalability.Recent foundation models improve unification but commonly retain fixed-scale tokenization and dense attention, mismatching EEG’s cross-scale spatiotemporal structure.Source: §1 · p. 2
  • EEG tasks vary from short transient bursts to slow oscillations and from localized activations to distributed regional interactions.Scale-agnostic tokens can dilute task patterns, while dense attention over noisy signals can introduce spurious dependencies and computational overhead.Source: §1 · p. 2
  • CSBrain combines Cross-scale Spatiotemporal Tokenization (CST) with Structured Sparse Attention (SSA) to build scale-aware tokens and capture dependencies across windows and regions.Alternately stacking CST and SSA progressively integrates cross-scale dependencies.Source: §1 · pp. 2–3
  • CSBrain outperforms task-specific and foundation baselines across 11 representative EEG tasks and 16 datasets.The findings support cross-scale modeling as an inductive bias for generalized EEG decoding.Source: Figure 1 · p. 3, §1 · p. 3

2 Model Architecture

CSBrain preprocesses EEG, forms scale-aware tokens across temporal windows and anatomical regions, and uses structured sparse attention to integrate dependencies before task-specific prediction or reconstruction. Its architecture also supports masked-autoencoding pretraining to learn transferable EEG representations.

2 Model Architecture

  • CSBrain alternates Cross-scale Spatiotemporal Tokenization (CST) and Structured Sparse Attention (SSA), then attaches a task head for reconstruction, classification, or regression.The modules are stacked for L layers to progressively integrate cross-scale spatiotemporal dependencies.Source: §2 · p. 3, Figure 2 · p. 4

2.1 EEG Signal Preprocessing

  • Preprocessing filters and standardizes EEG, segments it into non-overlapping windows, then combines convolutional temporal features, FFT-derived spectral features, and positional encoding.Signals are resampled to 200 Hz and scaled to 100 µV; the resulting embedding combines temporal and spectral information.Source: §2.1 · pp. 3–4

2.2 Cross-scale Spatiotemporal Tokenization (CST)

  • CST applies multi-scale temporal convolutions to localized windows, concatenating their outputs into tokens that capture patterns at varying temporal resolutions.The paper motivates this design by the range of temporal patterns across EEG tasks and the limits of fixed token granularity.Source: §2.2 · pp. 4–5
  • CST groups spatially adjacent electrodes within anatomically defined regions and applies multi-scale spatial kernels to produce scale-aware tokens.The resulting tokens provide features for downstream attention computation.Source: §2.2 · p. 5
  • CST allocates more embedding dimensions to smaller kernels and fewer to larger kernels, balancing fine-grained features with coarser context.The dimensions across scales follow an exponentially decaying scheme.Source: §2.2 · p. 5

2.3 Structured Sparse Attention (SSA)

  • SSA groups tokens across windows for long-range temporal attention, and its structured design models temporal and spatial dependencies while mitigating redundant interactions and spurious correlations.The model alternates SSA with CST to build global context while preserving scale-awareness.Source: §2.3 · pp. 5–6
  • SSA forms spatial groups using representative tokens and mean-pooled regional features, then applies attention across anatomical regions.Layer normalization and a feedforward network with a residual connection refine the output.Source: §2.3 · pp. 5–6

2.4 Pretraining with Masked Autoencoding

  • For pretraining, CSBrain randomly masks temporal segments, encodes visible and learned masked embeddings through CST and SSA, then reconstructs masked EEG using mean squared error.The masked-autoencoding objective is intended to learn spatiotemporal dependencies from unlabeled EEG for downstream transfer.Source: §2.4 · p. 6

3 Experiments

CSBrain is evaluated across a broad set of EEG tasks and datasets, with comparisons against task-specific and foundation-model baselines. Its full design leads overall, while component analyses examine cross-scale tokenization, sparse attention, and task-specific activation patterns.

3.1 Pre-training Setup

  • Pretraining uses TUEG EEG segmented into 30-second samples, with band-pass and notch filtering, resampling to 200 Hz, and amplitude normalization.The corpus contains 1,109,545 segments, or over 9,000 hours; masked autoencoding uses a 50% masking ratio for 40 epochs.Source: §3.1 · pp. 6–7

3.2 Experimental Setup of Downstream BCI Tasks

  • Downstream evaluation spans 11 BCI tasks across 16 public EEG datasets, using task-appropriate metrics and comparisons with task-specific and foundation models.The protocols follow training, validation, and test splits; results are averaged over five random-seed runs.Source: §3.2 · p. 7, §3.3 · p. 7

3.3 Experimental Results

  • CSBrain’s macro-average exceeds CbraMod, LaBraM, and BIOT by 3.35%, 3.98%, and 7.73%, respectively.The comparison covers 11 BCI tasks across 16 datasets.Source: Table 2 · p. 8, §3.3 · p. 8
  • Across four representative tasks, the K = 3 stacked tokenization performs best, while K = 1 performs worst; stacking also beats applying cross-scale tokenization once.On motor imagery, the reported gains over the single-scale variant are 10.8%, 20.7%, and 12.1% on three evaluation metrics.Source: §3.3 · p. 8, Figure 4 · p. 9
  • SSA achieves the best Balanced Accuracy, Cohen’s Kappa, and Weighted F1, with TUEV gains of +8.1% and +5.3% over Criss-cross Attention.SSA groups tokens across temporal windows and brain regions; the comparison also considers dense and Criss-cross Attention.Source: §3.3 · p. 9, Figure 5 · p. 9
  • Grad-CAM visualizations show task-specific patterns, from localized contralateral motor-cortex activation to broader emotion and imagined-speech activations.Vigilance primarily activates temporal and occipital lobes; emotion and imagined speech show broader distributed activation.Source: §3.3 · p. 9

4 Conclusion

The conclusion presents cross-scale spatiotemporal modeling as central to CSBrain’s approach to generalized EEG decoding. Results across 11 tasks and 16 datasets support its effectiveness and the use of cross-scale modeling as an inductive bias.

4 Conclusion

  • CSBrain combines Cross-scale Spatiotemporal Tokenization and Structured Sparse Attention to capture EEG dependencies across multiple spatiotemporal scales.The conclusion characterizes cross-scale modeling as essential for generalizable brain foundation models.Source: §4 · p. 9
  • Results across 11 EEG decoding tasks and 16 datasets validate CSBrain’s effectiveness and support cross-scale modeling as an inductive bias for EEG representation learning.The conclusion describes the resulting representations as robust, scalable, and physiologically aligned.Source: §4 · p. 10

A Related Work

  • EEG is presented as a non-invasive, high-temporal-fidelity, affordable modality with applications in cognitive science, medical diagnostics, and BCI.The cited overview also notes EEG’s portability and use in clinical and everyday settings.Source: A Related Work · p. 17

A.1 Task-Specific EEG Decoding Models

The related-work discussion moves from task-specific decoding approaches to EEG foundation models. It identifies fixed-scale tokens and dense attention as a mismatch for EEG’s cross-scale spatiotemporal structure, motivating CSBrain’s structure-aware approach.

A.1 Task-Specific EEG Decoding Models

  • Handcrafted pipelines and deep task-specific decoders face generalization limits tied to task-specific priors, dataset coupling, and the need for retraining or redesign.Conventional methods combine engineered features with shallow classifiers; deep approaches learn from raw EEG but remain coupled to datasets and task formats.Source: §A.1 · p. 17

A.2 Brain Foundation Model

  • Many EEG foundation models use fixed-scale tokens and dense attention, which the paper says fail to align with EEG’s intrinsic cross-scale spatiotemporal structure.The related-work discussion cites masked modeling, contrastive learning, structured tokenization, and spatiotemporal attention among prior approaches.Source: §A.2 · p. 17

B.1 Hyperparameter

  • Table 3 summarizes CSBrain’s input configuration, model architecture, optimization settings, and training strategy.The supplied passages do not specify the table’s individual hyperparameter values.Source: §B.1 · p. 18, Table 3 · p. 18

B.2 Training Loss Curve

The supplementary material reports pretraining convergence, downstream fine-tuning settings, and the baselines used for comparison. The loss declines rapidly at first and then stabilizes during pretraining.

B.2 Training Loss Curve

  • During 40 pretraining epochs, the training loss decreases rapidly at first and then gradually stabilizes under the masked reconstruction objective.The loss curve is shown in Figure 7.Source: §B.2 · p. 18, Figure 7 · p. 19

C.1 Hyperparameter Settings

  • Downstream fine-tuning replaces the reconstruction head with an MLP task head, then fine-tunes the entire model using classification- or regression-appropriate loss.The losses are binary cross-entropy, cross-entropy, and mean squared error for binary classification, multiclass classification, and regression, respectively.Source: §C.1 · p. 18, Table 4 · p. 19

C.2 Baselines

  • The baselines include task-specific CNN and hybrid models alongside foundation models BIOT, LaBraM, and CBraMod.EEGNet uses depthwise and separable convolutions; EEGConformer combines CNN feature extraction with transformer-based long-range modeling.Source: §C.2 · pp. 19–20

C.3 Metrics

The section defines the metric sets used for classification and regression, then outlines evaluation across 11 EEG tasks and 16 public datasets under established data splits.

C.3 Metrics

  • Multiclass tasks use Balanced Accuracy, Cohen’s Kappa, and Weighted F1, capturing class-wise recall, chance-corrected agreement, and support-weighted precision–recall.Balanced Accuracy averages recall across classes; Weighted F1 weights each class by its number of true instances.Source: §C.3 · pp. 20–21
  • Binary tasks use Balanced Accuracy, AUC-PR, and AUROC, measuring class-wise recall, precision–recall trade-offs, and discrimination across thresholds.AUROC integrates the true-positive rate against the false-positive rate; AUC-PR uses precision and recall.Source: §C.3 · pp. 20–21
  • Regression tasks use Pearson correlation, R2 score, and RMSE to assess linear dependence, explained variance, and prediction-error magnitude.Lower RMSE indicates better model fit; an R2 below zero is worse than predicting the target mean.Source: §C.3 · p. 21

D Additional Results on Downstream Tasks

  • Evaluation covers 11 representative BCI tasks across 16 public EEG datasets, following established train, validation, and test splits.Incompatible or unavailable baselines are reimplemented and retrained under a unified protocol.Source: D Additional Results on Downstream Tasks · p. 21

D.1 Motor Imagery Classification

Across three motor-imagery and two emotion datasets, CSBrain is reported as best or top-performing, with gains over CBraMod across these task settings.

D.1 Motor Imagery Classification

  • 0.5657 balanced accuracy, 0.4209 Cohen’s kappa, and 0.5637 weighted F1; CSBrain exceeds CBraMod by +5.2% in balanced accuracy and +6.9% in kappa.On BCIC-IV-2a, the evaluation uses a subject-independent split.Source: Table 5 · p. 22, §D.1 · p. 22
  • 0.6417 balanced accuracy, 0.7230 AUC-PR, and 0.7200 AUROC; CSBrain exceeds CBraMod by +0.47% in balanced accuracy and +2.12% in AUROC.SHU-MI evaluates binary left-hand versus right-hand motor imagery.Source: Table 6 · p. 22, §D.1 · p. 22
  • 0.6304 balanced accuracy, 0.5071 Cohen’s kappa, and 0.6308 weighted F1; CSBrain improves over CBraMod by +1.3% in balanced accuracy and +1.7% in kappa.The text also reports that CSBrain surpasses EEGConformer and CNN-Transformer by large margins.Source: §D.1 · pp. 22–23, Table 7 · p. 23

D.2 Emotion Recognition

  • 0.4091 balanced accuracy, 0.2569 Cohen’s kappa, and 0.4101 weighted F1; CSBrain improves on CBraMod’s kappa by +2.1%.SEED-V uses short-time emotional-state samples.Source: Table 8 · p. 23, §D.2 · p. 23
  • 0.5752 balanced accuracy, 0.5204 Cohen’s kappa, and 0.5796 weighted F1; CSBrain improves over CBraMod by +2.4% in balanced accuracy and +1.6% in kappa.The reported gains span FACED’s long EEG segments and SEED-V’s short-windowed data.Source: §D.2 · pp. 23–24, Table 9 · p. 24

D.3 Seizure Detection

Across seizure detection, sleep staging, imagined speech, and other EEG tasks, CSBrain generally outperforms strong baselines. Pretraining benefits all evaluated tasks, with especially large gains when downstream data is limited.

D.3 Seizure Detection

  • On CHB-MIT, CSBrain reaches AUC-PR 0.5164 versus CBraMod’s 0.3689; on Siena, it improves AUC-PR by 7.6 points (0.4871 vs. 0.4107).CSBrain also leads on AUROC on both datasets, though its CHB-MIT balanced accuracy is marginally lower than CBraMod’s.Source: §D.3 · pp. 24–25, Table 10 · p. 25, Table 11 · p. 25

D.4 Sleep Staging

D.5 Imagined Speech Classification

  • On BCIC2020-3, CSBrain improves balanced accuracy over CBraMod by 6.3 points (0.6004 vs. 0.5373).It also achieves the highest balanced accuracy and strong Cohen’s kappa and weighted F1 scores.Source: §D.5 · p. 27, Table 14 · p. 27

D.6 Vigilance Estimation

  • On SEED-VIG, CSBrain achieves R2 0.2363 and RMSE 0.2774, improving R2 by 5.6 points over LaBraM.Its Pearson correlation is 0.6314, close to LaBraM’s 0.6347.Source: §D.6 · p. 27, Table 15 · p. 27

D.7 Mental Stress Detection; D.8 Mental Disorder Diagnosis

  • On MentalArithmetic and Mumtaz2016, CSBrain improves balanced accuracy over CBraMod by 3.0 points (0.7558 vs. 0.7256) and 0.8 points (0.9643 vs. 0.9560), respectively.It achieves the highest balanced accuracy on both datasets.Source: §D.7 · p. 28, Table 16 · p. 28, §D.8 · p. 28

D.9 Event Type Classification

  • On TUEV, CSBrain achieves balanced accuracy 0.6903 and Cohen’s kappa 0.6833, with weighted F1 0.8333 close to CBraMod’s 0.8342.The reported gains in balanced accuracy reflect improved performance on minority classes.Source: §D.9 · p. 29, Table 18 · p. 29

D.10 Abnormal Detection; D.11 Slowing Event Classification

  • On TUAB, CSBrain reaches balanced accuracy 0.8172 and AUC-PR 0.9005; on TUSL, it achieves balanced accuracy 0.8571 and improves on LaBraM by 9.46 points.TUAB’s AUROC is slightly lower than LaBraM’s, while TUSL Cohen’s kappa improves by 14.21 points over LaBraM.Source: §D.10 · p. 30, Table 19 · p. 30, §D.11 · p. 30

E.1 The Effectiveness of Pretraining

  • Pretraining improves performance across four tasks, with BCIC-IV-2a balanced accuracy rising from 0.39 to 0.57; gains are more modest on TUAB’s over 400,000 samples.The authors report that gains are especially pronounced in data-limited settings, while remaining meaningful on larger datasets.Source: §E.1 · p. 31, Figure 8 · p. 31

E.2 Scaling Analysis

CSBrain performance improves as pretraining data and model capacity increase. Data-volume gains begin to plateau beyond 5000 hours, and the scaling exploration is computationally constrained.

Effect of Pretraining Data Volume

  • From 10 to 9000 pretraining hours, accuracy, Cohen’s kappa, and F1 improve steadily, particularly from 500 to 5000 hours, then begin to plateau.Figure 9 presents the effect of pretraining data volume; the reported plateau occurs under fixed model capacity.Source: §E.2 · p. 31, Figure 9 · p. 32

Effect of Model Capacity

  • As model size rises from 1.4M to 4.9M parameters, downstream performance improves steadily.The tested sizes correspond to 2 to 12 Transformer layers; Figure 10 depicts the parameter-size effect.Source: §E.2 · p. 31, Figure 10 · p. 32

Scaling Analysis

  • The scaling exploration is constrained by computational resources and may not reach the scaling limits.The amount of EEG data required for effective large-scale foundation-model pretraining remains an open question.Source: §E.2 · p. 32

E.3 Ablation on Regional Descriptor

The default regional descriptor combines a sequentially sampled token with a region-level mean. It performs best across the four tested tasks and metrics.

E.3 Ablation on Regional Descriptor

  • Across four representative tasks, the default regional descriptor consistently achieves the best performance across all metrics.Figure 11 reports the regional-descriptor ablation results.Source: §E.3 · p. 32, Figure 11 · p. 32
  • Using only the token or only the mean causes noticeable drops in accuracy and robustness, while random sampling performs slightly better than either single-source variant but remains below the default.The default combines a sequentially sampled token and the average feature across tokens in each region.Source: §E.3 · pp. 32–33
  • The results indicate that local token features and region-level context are both important, while sequential sampling provides consistent spatial coverage.Source: §E.3 · p. 33

F.1 Topography Visualization

Grad-CAM visualizations show task-specific EEG channel activation topographies, while t-SNE and channel-similarity analyses depict learned representations with class separation and local and long-range spatial relationships.

F.1 Topography Visualization

  • SEED-VIG vigilance states primarily activate temporal and occipital lobes, whereas fatigue shows reduced activation there; BCIC-IV-2a motor imagery shows contralateral or bilateral motor-cortex patterns.The topography visualizations are Figures 12 and 13.Source: §F.1 · p. 33, Figure 12 · p. 33, Figure 13 · p. 33
  • SEED-V emotional tasks activate frontal and occipital lobes, while BCIC2020-3 imagined speech activates frontal and temporal lobes, including left-hemisphere electrodes corresponding primarily to Broca’s area.Figures 14 and 15 show these task-specific topographies.Source: §F.1 · p. 33, Figure 14 · p. 34, Figure 15 · p. 34

F.2 Representation Visualization

F.3 Channel Relationship Visualization

  • Channel similarities show high similarity within anatomical regions and long-range connections such as frontal–parietal and temporal–occipital pairs.The analysis visualizes channel pairs with cosine similarity greater than 0.5.Source: §F.3 · p. 35

G.1 Implication

CSBrain’s cross-scale, structure-aware modeling is presented as a reusable template for EEG foundation models, with further work needed to assess scaling and extend the approach across datasets and modalities.

G.1 Implication

  • CSBrain’s cross-scale modeling is presented as a reusable architectural template, with performance across 11 tasks and 16 public datasets suggesting unified architectures can scale across heterogeneous brain-decoding scenarios.The authors also describe potential generalization to other neurophysiological modalities.Source: §G.1 · pp. 35–36

G.2 Limitation

  • Computational resources constrained exploration of model and data scaling, leaving scaling at vision-language or language-only model scales unassessed and EEG pretraining data sufficiency open.The authors also identify dataset unification, normalization, and cross-institutional collaboration as further needs.Source: §G.2 · p. 36

G.3 Future Work

  • Future work includes expanding pretraining datasets and model capacity, studying downstream generalization and scaling limits, and integrating MEG or fMRI with EEG for cross-modal brain foundation models.The authors suggest these signals’ complementary spatiotemporal properties could support richer neural representations.Source: §G.3 · pp. 36–37
Loading 2506.23075v1…