PaperlayerView exact revision on arXiv

Source-linked AI summary

EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo, Dongrui Wu

arXiv:2601.17883v3cs.LGcs.CV

TL;DR

EEG foundation models lack a consistent comparison because their training and evaluation choices vary widely. EEG-FM-Compass reviews 55 models and benchmarks 12 open-source models across 13 datasets, finding that fine-tuning helps, specialists remain competitive, and scaling alone does not ensure generalization.

  • Problem

    Heterogeneous pre-training objectives, preprocessing choices, and evaluation protocols complicate consistent conclusions about what drives EEG foundation-model generalization.

  • Method

    The paper reviews 55 models and benchmarks 12 open-source EEG foundation models against specialist baselines across 13 datasets and nine BCI paradigms.

  • Results

    Full-parameter fine-tuning outperforms linear probing, while specialist models remain competitive and larger EEG foundation models do not consistently generalize better.

  • Takeaways & Limitations

    EEG foundation models have not yet achieved universally transferable representations, motivating improved pre-training and robustness to noise and cross-task heterogeneity.

  • Takeaways & Limitations

    Under current data regimes and training practices, limited noisy EEG corpora and pre-training objectives may not translate increased model capacity into richer representations.

Abstract

from arXiv · show

Electroencephalography (EEG) foundation models (FMs) have recently emerged as a promising paradigm for brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings. Despite rapid progress, a fair and comprehensive comparison of existing EEG FMs is still lacking, owing to inconsistent pre-training objectives, preprocessing choices, and downstream evaluation protocols. To fill this gap, we present EEG-FM-Compass. We first review 55 representative models and organize their design choices into a unified taxonomic framework including data standardization, model architectures, and self-supervised pre-training strategies. We then evaluate 12 open source FMs and competitive specialist baselines across 13 EEG datasets spanning nine brain-computer interface paradigms. Emphasizing real-world deployments, we consider both cross-subject generalization under a leave-one-subject-out protocol and rapid calibration under a within-subject few-shot setting. We further compare full-parameter fine-tuning with linear probing to assess the transferability of pre-trained representations, and examine the relationship between model scale and downstream performance. Our results indicate that: 1) linear probing is frequently insufficient; 2) specialist models trained from scratch remain competitive across many tasks; and 3) larger FMs do not necessarily yield better generalization performance under current data regimes and training practices.

1. INTRODUCTION · Q3. Do EEG FMs exhibit behavior consistent with scaling laws? · 2. OVERVIEW OF EEG FMs

EEG-FM-Compass frames EEG foundation models as a response to fragmented, data-intensive BCI development and provides a unified overview of their designs, definitions, and evaluation questions. It surveys 55 models and motivates fair benchmarking of transferability, specialist competitiveness, and scaling behavior across heterogeneous EEG tasks.

  • 1. INTRODUCTION: EEG supports diverse BCI paradigms spanning motor, perceptual, cognitive, and clinical brain states.Examples include motor imagery, SSVEP, ERP, epilepsy recognition, mental workload, and imagined speech.
  • 1. INTRODUCTION: Inconsistent datasets, preprocessing, and evaluation protocols make direct comparison difficult and motivate questions about generalization, scratch-trained competitiveness, and scaling laws.The paper explicitly asks whether EEG FMs generalize across downstream tasks, outperform traditional and scratch-trained methods, and benefit from larger models or more pre-training data.
  • 1. INTRODUCTION: Deployment remains difficult because labeled EEG data are costly, devices have heterogeneous channel layouts, and many models have limited transferability.These constraints motivate foundation models intended to reduce task-specific data requirements and accommodate varied electrode configurations.
  • Q3. Do EEG FMs exhibit behavior consistent with scaling laws?: 55 EEG FMs are surveyed through a unified taxonomy covering pre-training data, preprocessing, objectives, and architectural choices.The survey is described as the most comprehensive collection to date.
  • Q3. Do EEG FMs exhibit behavior consistent with scaling laws?: EEG FMs use large-scale heterogeneous data and self-supervised learning to produce representations intended to transfer across diverse downstream BCI tasks.The premise builds on foundation-model progress in language and vision while targeting EEG data scarcity and heterogeneous devices.
  • 2. OVERVIEW OF EEG FMs: Benchmarking compares full-parameter fine-tuning with classification-head fine-tuning across LOSO and within-subject few-shot adaptation scenarios.Within-subject fine-tuning uses approximately 1/20 ∼1/100 of the data typically used in LOSO protocols.
  • 2.1. Advances and Trends of EEG FMs: The surveyed field expanded rapidly, with 16.4% of studies published in 2024 and 67.3% in 2025 or 2026.This growth coincides with increasing diversity in model scope, modalities, backbones, and training methods.
  • 2.1. Advances and Trends of EEG FMs: Current EEG FMs are dominated by Transformer backbones, span fewer than one million to several billion parameters, and remain difficult to compare because training and downstream protocols vary.The review also distinguishes generalized from paradigm-specific models and motivates standardized analysis of factors affecting generalization.

Data collection and curation

EEG foundation model development begins by aggregating large-scale, heterogeneous EEG recordings from public datasets and self-collected sources. Because downloaded data often contain poor-quality subjects, corrupted channels, and artifacts, careful curation is critical for effective training.

  • Data collection: Public datasets and self-collected recordings are the primary sources for EEG foundation-model data.General-purpose models aggregate large volumes of unlabeled data spanning diverse tasks for pre-training.
  • Data collection: Large-scale pre-training corpora combine unlabeled recordings from diverse EEG tasks and paradigms.This aggregation supports models intended to accommodate various paradigms.
  • Data curation: Directly downloaded EEG data often contain poor-performing subjects, corrupted channels, and various artifacts, making high-quality curation critical.Common strategies include subject-level screening for low task performance or excessive noise and channel-level screening for persistent artifacts or disconnections.

Data preprocessing

EEG foundation-model pipelines preprocess and normalize signals to reduce nuisance variability from heterogeneous subjects, devices, and recording protocols. Common components include channel unification, resampling and filtering, normalization or alignment, while augmentation remains relatively under-explored.

  • Preprocessing and normalization reduce nuisance variability and stabilize optimization amid shifts from electrode placement, impedance, noise, and physiological state.
  • Channel Unification: Channel unification addresses heterogeneous electrode layouts, channel counts, reference schemes, and protocols, but fixed-montage strategies can discard mismatched recordings and limit transfer to unseen layouts.
  • Resampling and Bandpass Filtering: 50.0% of surveyed studies resample EEG signals to 200 Hz, while 34.1% resample to 250 Hz or 256 Hz; bandpass and notch filtering suppress drifts, high-frequency noise, and power-line interference.
  • Normalization and Marginal Alignment: z-score normalization, CAR, and EMA normalization are widely used, whereas Euclidean Alignment explicitly reduces session-level covariance shifts.EMA is suitable for online or streaming settings because it adapts to non-stationary signals without precomputed global statistics.
  • Data Augmentation: Data augmentation remains relatively under-explored in EEG FM pre-training, with only a few surveyed models using perturbed views, overlapping windows, or signal-level perturbations.

Model pre-training

EEG foundation models predominantly use masked reconstruction, especially over patched inputs, while contrastive, frequency-domain, codebook-based, and auxiliary objectives provide alternative or complementary pre-training signals. The section highlights a trade-off between waveform fidelity and robustness to EEG noise, motivating tokenization, denoising, and regularization under heterogeneous BCI settings.

  • Pre-training trends: Approximately 83.6% of EEG foundation models adopt Transformer-based backbones, while masked reconstruction is the predominant self-supervised pre-training strategy.The reviewed taxonomy covers 55 foundation models and identifies eight empirical trends.
  • Raw-signal reconstruction: Raw-signal reconstruction preserves waveform structure and cross-channel dependencies but can devote capacity to artifacts and recording-specific noise in low-SNR non-invasive EEG.This risk arises when models reconstruct waveform targets containing nuisance variability that is not predictive for downstream tasks.
  • Token reconstruction: Token-level reconstruction reduces sensitivity to amplitude scaling and local waveform noise, often improving optimization stability for large encoders and supporting patch-based processing.Tokenizer-compressed targets can emphasize spatio-temporal structure rather than low-level waveform detail.
  • Complementary objectives: Token reconstruction often benefits from complementary objectives, while frequency-domain supervision emphasizes rhythmic structure and auxiliary time-domain losses preserve waveform fidelity and task-relevant information.Examples include classification, global contrastive, and frequency-domain reconstruction terms.
  • Codebook objectives: Codebook-based objectives compress EEG into symbolic representations that reduce low-level noise influence and enable sequence modeling, causal generation, and prompt-style adaptation.They also introduce design choices involving codebook size, commitment regularization, residual quantization depth, and update schedules.

Downstream generalization

Downstream evaluation tests whether EEG foundation-model representations transfer effectively to practical BCI tasks. The benchmark is designed to assess both cross-subject transfer with abundant data and few-shot calibration with limited data under consistent protocols.

  • Evaluation regimes: The benchmark evaluates EEG foundation models across data-rich cross-subject transfer and data-limited calibration regimes using consistent protocols.This design reflects the expectation that foundation models should reduce reliance on task-matched labeled data and support adaptation with minimal calibration.
  • Task coverage: Downstream datasets span clinical EEG, motor imagery, emotion recognition, and sleep-related paradigms.Examples include TUAB, TUEV, and CHB-MIT; BCIC-IV-2A and PhysioNetMI; FACED, SEED, and SEED-V; and Sleep-EDF.

3. BENCHMARK OF EEG FMs

EEG-FM-Compass benchmarks 12 open-source EEG foundation models against specialist baselines across 13 datasets and nine BCI paradigms. It evaluates both cross-subject generalization and within-subject few-shot personalization, while comparing full-parameter fine-tuning with linear probing.

  • Benchmark scope: The benchmark evaluates 12 open-source EEG FMs and 8 specialist baselines across 13 datasets spanning 9 representative BCI paradigms.The baselines include conventional machine-learning and deep-learning methods.
  • Evaluation scenarios: LOSO measures cross-subject transfer within a fixed task and headset configuration without labeled data from the held-out test subject.Models are fine-tuned on labeled data from other subjects and evaluated on held-out subjects, enabling zero-calibration testing.
  • Fine-tuning strategies: Full-parameter fine-tuning updates the entire network, whereas linear probing freezes the pre-trained encoder and trains only the classification head.Their comparison assesses the quality and transferability of pre-trained representations.
  • Evaluation scenarios: Within-subject few-shot evaluation measures rapid personalization by fine-tuning on a small labeled subset from the target subject and testing on its remaining data.This setting reflects deployment scenarios where only limited calibration data are available for a new user.
  • Evaluated models: Among the evaluated models, MIRepNet is specialized exclusively for motor-imagery tasks, while the other 11 EEG FMs are general-purpose models intended for multiple BCI paradigms.The benchmark includes BENDR, BIOT, LaBraM, Neuro-GPT, EEGPT, CBraMod, TFM, BrainOmni, EEGMamba, SingLEM, and LUNA alongside MIRepNet.
  • Pre-training considerations: Masked reconstruction objectives may enhance temporal-dynamics modeling because periodic, temporally structured EEG signals provide patterns these objectives can capture.This rationale is linked to signals generated by stimuli flickering at distinct frequencies.

Can FMs learn generalized representations?

EEG foundation models learn transferable representations to some extent, but their generalization remains insufficiently robust across tasks. Full-parameter adaptation is usually necessary, while model performance varies substantially by task and paradigm.

  • Adaptation and transferability: Full-parameter fine-tuning generally outperformed linear probing, showing that downstream adaptation must update encoder parameters.The evaluation found head-only adaptation inferior for most models, often substantially so.
  • Adaptation and transferability: 9 out of 12 FMs showed significant positive median Δ values favoring full fine-tuning over linear probing.A scenario-level paired Wilcoxon signed-rank analysis quantified this advantage.
  • Task-specific variability: Model performance varied considerably by task: CBraMod ranked first and second on SEED under LOSO and few-shot settings but had the highest RMSE on SEED-VIG under LOSO.These results illustrate that competitive performance on one dataset does not ensure broad generalization.
  • Few-shot generalization: On Nakanishi2015, several FMs, including BENDR, EEGPT, and Neuro-GPT, achieved strong performance despite only 12 trials per class for fine-tuning.Nakanishi2015 represents an SSVEP paradigm with extremely limited adaptation data.
  • Limits of generalization: Even the best-performing FMs experienced notable degradation on specific tasks, indicating that truly universal EEG representations remain an open challenge.The benchmark therefore supports only limited, rather than robust, transferability of pre-trained representations.

Can FMs consistently outperform specialist models?

EEG foundation models do not consistently outperform specialist EEG models: specialists achieve stronger overall rankings and accuracy, especially in few-shot personalization, while performance is comparable under LOSO. These results are accompanied by substantially higher computational costs for FMs and no clear advantage from greater model scale.

  • Model rankings: EEGNet achieved the most top-1 placements with only 2K parameters, while ShallowConv achieved the most top-3 placements; four of the top five models were specialists.These rankings were computed across all tasks and scenarios using identical test sets and comparable fine-tuning data.
  • Computational cost: 361 GFLOPs per forward pass were required by EEGPT versus 0.35 GFLOPs for EEGNet, while EEGPT’s fine-tuning throughput was about 46 times lower.This example uses the 4s, 10-channel setting and illustrates the orders-of-magnitude computational gap between FMs and lightweight specialists.
  • Model rankings: 16 first-place finishes and 47 top-3 placements were achieved by specialist models, outperforming the seven selected FMs.The comparison selected the seven FMs with the highest top-3 counts to account for the larger number of FMs.
  • Evaluation scenarios: Specialist models achieved higher average decoding accuracy overall, with most top-1 placements in few-shot evaluation but comparable performance to FMs under LOSO.The few-shot setting requires rapid personalization with minimal calibration, whereas LOSO assumes labeled multi-subject data.
  • Statistical analysis: CBraMod attained the best average rank, but its differences from EEGNet, ShallowConv, Conformer, Deformer, and LMDA were not statistically significant.A Friedman test found significant overall ranking differences, followed by a Nemenyi post-hoc comparison.
  • Scaling behavior: CBraMod ranked first with average ranks of 6.56 and 4.0M parameters, while the 2K-parameter EEGNet ranked second, providing no consistent scale advantage.The rankings covered 13 datasets under both LOSO and few-shot scenarios.

4. DISCUSSION

The discussion finds no universally superior pre-training objective or consistent objective–paradigm pairing, while paradigm-specific FMs and entropy alignment offer practical directions with important constraints. It also cautions that larger EEG FMs do not yet guarantee better downstream generalization.

  • Model scaling: Increasing EEG FM parameter count does not guarantee better downstream generalization under current data regimes and training practices.The scaling behavior commonly observed in vision and language foundation models has not yet been clearly established for EEG, partly because EEG data are costly and limited.
  • Pre-training objectives: No single pre-training objective category dominates overall rankings, and reconstruction targets are not decisive for downstream transferability.Raw-signal, embedded-token, and codebook-based objectives span broad rank ranges, indicating that target choice alone is insufficient.
  • Pre-training objectives: Specific pre-training objectives show no consistent advantage for particular BCI paradigms, including frequency-domain supervision for SSVEP on Nakanishi2015.BENDR, EEGPT, and Neuro- are identified among the top performers despite not using the expected explicit frequency-domain advantage.
  • Paradigm-specific FMs: Paradigm-specific pretrained FMs are a viable direction because target paradigms are generally known before downstream data acquisition.Examples include MEET for emotion recognition, MIRepNet for MI, PSGFM for sleep staging, and EpilepsyFM for epilepsy detection; auxiliary cross-paradigm data remains an open question.
  • Entropy alignment: EA improved generalization performance for the majority of models on BNCI2014001 by reducing inter-subject discrepancies in second-order statistics.For fair comparison, EA was applied only to FMs whose original implementations used it, because mismatched pre-training and fine-tuning spaces could confound evaluation.

Efficient pre-training strategies … 8. AUTHOR CONTRIBUTIONS

EEG-FM-Compass unifies prior EEG-FM design choices and benchmarks open-source models against specialist baselines across diverse BCI settings. Its findings motivate more effective pre-training, improved robustness to noise and heterogeneity, and caution against relying on model scale or frozen representations alone.

  • Efficient pre-training strategies: Masked reconstruction remains the dominant pre-training objective, targeting raw signals, frequency-domain representations, or embedded tokens.However, no single model has demonstrated consistently strong performance across all tasks.
  • Efficient pre-training strategies: Future research should address cross-device heterogeneity and develop pre-training strategies that learn truly universal, transferable representations.Current EEG FMs also do not exhibit scaling-law behavior.
  • Large-scale high-quality data construction: Low signal-to-noise ratios may cause raw-signal reconstruction to fit noise patterns instead of learning generalizable downstream representations.Noise arises from hardware limitations, environmental interference, and variation in subject attention during acquisition.
  • 5. CONCLUSION: 55 studies are organized into a unified framework covering common EEG-FM pipeline components and pre-training objectives.The framework enables structured comparison across heterogeneous devices and BCI paradigms.
  • 5. CONCLUSION: 12 open-source EEG FMs and competitive specialist baselines are evaluated across 13 datasets spanning 9 BCI paradigms.Evaluation uses cross-subject LOSO and within-subject few-shot protocols.
  • 5. CONCLUSION: Full-parameter fine-tuning consistently outperforms linear probing, indicating that pretrained encoders are not directly usable as fixed feature extractors across diverse tasks.The result suggests current EEG FMs have not achieved universally transferable representations.
  • 5. CONCLUSION: Specialist models trained from scratch remain highly competitive across downstream EEG tasks.This finding challenges the assumption that foundation-model pre-training consistently dominates task-specific training.
  • 8. AUTHOR CONTRIBUTIONS: D.L. conceived the study, conducted the analysis, and wrote the manuscript, while other authors contributed experiments, literature review, data collection, revisions, and supervision.The conflict-of-interest statement declares none.

A2. PRE-TRAINING AND DOWNSTREAM DATASETS

This section catalogs the data resources underlying EEG foundation models, covering both pre-training and downstream datasets. It also characterizes 13 benchmark datasets, within-subject few-shot protocols, and heterogeneous preprocessing choices used in evaluation.

  • Dataset resources: Tables 6–13 systematically document the pre-training and downstream datasets used by existing EEG foundation models.The overview covers the data resources supporting current EEG FMs.
  • Benchmark datasets: The benchmark comprises 13 EEG datasets, with their characteristics summarized alongside corresponding within-subject few-shot fine-tuning protocols.These protocols are summarized in Table 15, while dataset characteristics are summarized in Table 14.
  • Benchmark datasets: The benchmark spans diverse paradigms, including motor imagery, visual object decoding, and vigilance estimation during simulated driving.BNCI datasets represent motor-imagery tasks, Things-EEG2 supports visual object decoding, and SEED-VIG measures vigilance using PERCLOS.
  • Evaluation protocols: Some benchmark protocols modify the original data or restrict training and evaluation segments to satisfy model-input and experimental requirements.One protocol extends 0.8s epochs to 1s, while another fine-tunes on the first seizure and its 10-min pre-ictal segment before evaluating remaining seizures.

Parameter-Efficient Fine-tuning

The section preliminarily evaluates parameter-efficient fine-tuning for EEG foundation models using LoRA on CBraMod and Neuro-GPT across two datasets and two evaluation scenarios. It also situates evaluated models through rankings, average ranks, parameter sizes, pre-training objectives, and leakage-aware analysis.

  • LoRA evaluation: LoRA is preliminarily evaluated on CBraMod and Neuro-GPT using BNCI2014001 and BNCI2014009 under LOSO and within-subject few-shot scenarios.The results are reported in Table 57.
  • Model comparison: Overall rankings, average ranks, and parameter sizes are reported across 13 datasets for LOSO and within-subject few-shot evaluation.Additional analyses group EEG FMs by pre-training objective and provide a leakage-aware ranking.

Computational Efficiency · Statistical Significance Analysis

The section compares fine-tuning costs under different feature aggregation strategies and evaluates whether full fine-tuning, linear probing, and overall model rankings differ statistically. It also documents representation visualizations across datasets, models, and adaptation strategies.

  • Statistical Significance Analysis: Scenario-level paired Wilcoxon signed-rank tests compare full fine-tuning with linear probing across dataset–scenario pairs for each EEG FM.Each valid pair uses the same model and dataset–scenario under both strategies.
  • Computational Efficiency: Tables 55 and 56 compare fine-tuning computational costs under flatten and pooling feature aggregation, reporting Full / Linear results.Full denotes full-parameter fine-tuning, whereas Linear denotes head-only fine-tuning.
  • Statistical Significance Analysis: For SEED-VIG, the test sign is reversed because lower RMSE is better.“Full wins”, “Linear wins”, and “Ties” count dataset–scenario pairs favoring each outcome, while rank-biserial correlation r measures effect size.
  • A5. Representation Visualization: Figs. 15–17 visualize learned representations on the SEED, TUAB, and BNCI2014008 datasets using t-SNE.Each dataset includes seven specialist models and nine EEG FMs, with both full-parameter fine-tuning and linear probing shown for every FM.
  • A5. Representation Visualization: The representation visualizations directly compare specialist models and EEG FMs across full-parameter fine-tuning and linear probing conditions.The comparisons span three datasets and show both adaptation strategies for each EEG FM.
  • A5. Representation Visualization: Overall model rankings were assessed with the Friedman test followed by the Nemenyi post-hoc test using 25 dataset–scenario combinations as evaluation blocks.Models with incomplete results, including Traditional ML, BrainOmni-Tiny, and BrainOmni-Base, were excluded; ranks were recomputed within each scenario for the remaining 17 models.

A6. EXISTING SURVEYS AND BENCHMARKS

This section positions EEG-FM-Compass against recent EEG and brain foundation-model surveys and benchmarks by comparing six evaluation dimensions. It highlights broad coverage of 55 EEG foundation models and examines how fine-tuning data ratios affect model performance and rankings.

  • Existing surveys and benchmarks: The comparison spans six aspects, including contribution type, EEG-FM coverage, benchmark scope, evaluation protocols, specialist baselines, and each work’s main distinction.EEG-FM-Compass is compared with representative surveys, reviews, and benchmarks published over the past two years.
  • Existing surveys and benchmarks: 55 EEG foundation models are surveyed, distinguishing EEG-FM-Compass from prior surveys and benchmarks.The supplied passage states that this coverage is substantially larger than that of a prior effort, but the comparison value is truncated.
  • Existing surveys and benchmarks: 98.19 and 7.56×10−14 were obtained for the reported Friedman test, while the Iman-Davenport statistic was 7.81 with p=4.34×10−16.The Nemenyi post-hoc test had a critical difference of 4.94 at α=0.05; models joined by a horizontal bar were not significantly different.
  • Existing surveys and benchmarks: Across fine-tuning ratios from 10% to 90%, most models improved consistently, while relative rankings remained largely stable.TRCA consistently achieved the highest accuracy on Nakanishi2015, and EEGNet remained competitive across settings on BNCI2014008.
Loading 2601.17883v3…