Source-linked AI summary
Test-Time Adaptation for EEG Foundation Models: A Systematic Study under Real-World Distribution Shifts
Gabriel Jason Lee, Jathurshan Pradeepkumar, Jimeng Sun
TL;DR
EEG foundation models remain vulnerable to distribution shifts, while the effectiveness of TTA for EEG is insufficiently established. The paper introduces NeuroAdapt-Bench to evaluate representative TTA methods across models, tasks, and realistic shifts, finding inconsistent gains overall but greater stability for optimization-free approaches.
Problem
EEG foundation models often degrade across clinical settings, devices, populations, and protocols, while TTA effectiveness for EEG remains underexplored.
Method
NeuroAdapt-Bench systematically evaluates representative TTA methods across multiple EEG foundation models, downstream tasks, and in-distribution, out-of-distribution, and extreme shifts.
Results
Standard TTA methods yield inconsistent gains and often degrade performance, whereas optimization-free approaches such as T3A show greater stability and often positive gains.
Takeaways & Limitations
Existing TTA methods from other domains do not transfer reliably to EEG, motivating EEG-specific adaptation strategies and realistic robustness evaluation.
Takeaways & Limitations
The benchmark covers only three TTA methods, four EEG foundation model variants, and five downstream datasets, and larger models can make adaptation memory-intensive.
Abstract
from arXiv · showhide
Electroencephalography (EEG) foundation models have shown strong potential for learning generalizable representations from large-scale neural data, yet their clinical deployment is hindered by distribution shifts across clinical settings, devices, and populations. Test-time adaptation (TTA) offers a promising solution by enabling models to adapt to unlabeled target data during inference without access to source data, a valuable property in healthcare settings constrained by privacy regulations and limited labeled data. However, its effectiveness for EEG remains largely underexplored. In this work, we introduce NeuroAdapt-Bench, a systematic benchmark for evaluating test-time adaptation methods on EEG foundation models under realistic distribution shifts. We evaluate representative TTA approaches from other domains across multiple pretrained foundation models, diverse downstream tasks, and heterogeneous datasets spanning in-distribution, out-of-distribution, and extreme modality shifts (e.g., Ear-EEG). Our results show that standard TTA methods yield inconsistent gains and often degrade performance, with gradient-based approaches particularly prone to heavy degradation. In contrast, optimization-free methods demonstrate greater stability and more reliable improvements. These findings highlight the limitations of existing TTA techniques in EEG, provide guidance for future development, and underscore the need for domain-specific adaptation strategies.
1. Introduction
EEG foundation models face severe distribution shifts across clinical sites, devices, subjects, and protocols, while TTA offers source-free adaptation to unlabeled target data. NeuroAdapt-Bench evaluates this problem systematically and finds that optimization-free methods are generally more stable than gradient-based methods.
- Motivation: EEG foundation models can fail to generalize across hospitals, acquisition devices, patients, sessions, tasks, and clinical sites.These shifts are especially complex because EEG recordings vary in patient-specific dynamics and acquisition protocols.
- Motivation: TTA adapts models during inference using unlabeled target data without requiring source data or target labels.This source-free property is valuable when privacy restrictions, limited labels, and fine-tuning costs constrain healthcare deployment.
- Research gap: EEG TTA remains underexplored, with prior studies typically limited to single tasks or architectures.Such narrow evaluations make it difficult to determine whether reported gains generalize across settings.
- Benchmark: NeuroAdapt-Bench evaluates representative TTA methods across multiple EEG foundation models, downstream tasks, and realistic deployment shifts.The benchmark includes in-distribution subject variability, cross-dataset shifts, and extreme modality shifts such as Ear-EEG, using No-TTA as the reference.
- Findings: +18.9 percentage points in balanced accuracy is T3A's largest mean gain for REVE-Base on CHB-MIT, while T3A is the only method with positive mean improvement across all evaluated shift settings.Overall, standard TTA methods produce inconsistent gains and can degrade performance; optimization-free methods are generally more stable.
- Implications: Reliable clinical deployment requires evaluating stability and robustness under realistic shifts, not only in-distribution accuracy.The study also releases open-source code and evaluation pipelines to support reproducible benchmarking.
2. Related Work
EEG foundation models are designed for transfer across heterogeneous settings, but their robustness under real-world shifts remains insufficiently characterized. This work connects EEG foundation models with TTA through systematic evaluation across models and downstream conditions.
- EEG foundation models: EEG foundation models use large-scale self-supervised pretraining to support generalization across subjects, channel configurations, acquisition protocols, and tasks.Current models remain heterogeneous in their pretraining data, architectures, and evaluation.
- EEG foundation models: Encoder-only EEG foundation models target discriminative tasks, whereas generative models focus on language alignment and generative objectives.The benchmark evaluates TTA on encoder-based models.
- Test-time adaptation: TTA adapts source-trained models to unlabeled target samples during inference under shifted distributions, often without source data or target labels.This setting addresses deployment-time performance degradation caused by distribution shift.
- Study scope: The study systematically benchmarks representative TTA approaches across multiple EEG foundation models and diverse downstream settings.This enables analysis of both adaptation gains and failures, method stability, and clinical-deployment implications.
3. NeuroAdapt-Bench
NeuroAdapt-Bench standardizes evaluation of representative test-time adaptation methods for EEG foundation models across adaptation regimes, models, datasets, tasks, and distribution shifts. Its pipeline fine-tunes a shared classifier, adapts on unlabeled target data, and evaluates target-domain performance using standardized metrics.
- Benchmark scope: NeuroAdapt-Bench evaluates Tent, SHOT, and T3A under different adaptation regimes and update mechanisms.The benchmark compares these methods within a unified formulation alongside a no-adaptation baseline.
- TTA methods: Tent minimizes target-batch predictive entropy while adapting normalization-layer affine parameters and keeping the remaining network parameters fixed.SHOT instead keeps the classifier fixed and adapts the target feature extractor using confidence, diversity, and pseudo-label losses.
- TTA methods: T3A adapts classifier geometry without gradient-based optimization by refining class prototypes from target test features while keeping the feature extractor fixed.Its class support sets contain target feature vectors assigned to each class, whose means define the class templates.
- Pipeline: The benchmark uses a three-stage pipeline: classifier fine-tuning, test-time adaptation, and target-domain evaluation.Adaptation uses unlabeled held-out test signals, while ground-truth labels are reserved for metric computation after adaptation.
- Pipeline: A shared lightweight classifier and frozen encoder standardize comparisons across pretrained EEG foundation models.This design reduces confounding from model-specific heads and preserves a consistent initialization for subsequent adaptation.
- Evaluation: The evaluation spans four foundation-model variants, five EEG datasets, binary and multiclass tasks, patient-disjoint splits, and metrics including accuracy, balanced accuracy, ROC-AUC, PR-AUC, Cohen’s κ, and weighted F1.Each method–model–dataset combination is summarized by mean and standard deviation across five random seeds, with relative improvement reported to isolate adaptation effects.
4. Results and Discussion
On in-distribution TUEV and TUAB data, gradient-based TTA methods degrade performance, while T3A is more stable but provides only modest gains.
- Gradient-based methods, including Tent and SHOT, consistently and often substantially degrade performance across TUEV and TUAB.
(a) TUEV
On in-distribution datasets, TTA behaves differently across tasks: T3A modestly improves TUEV but every method degrades TUAB, while gradient-based updates can disrupt aligned representations.
- TUEV: T3A provides modest balanced-accuracy improvements on TUEV with lower variability across seeds and batch sizes.
- TUAB: All TTA methods degrade TUAB performance, with Tent showing the largest drop.
- Interpretation: When target data closely matches pretraining data, adaptation may have limited room to help and can cause negative transfer.The passage attributes this risk particularly to gradient-based approaches disrupting already aligned representations.
(a) SLEEPEDF-78
Under cross-dataset and extreme modality shifts, TTA is generally unstable: SleepEDF-78 produces broad degradation, while optimization-free T3A is comparatively more stable.
- SLEEPEDF-78: SleepEDF-78 causes greater degradation across nearly all TTA methods and metrics.
- SLEEPEDF-78: T3A offers only marginal gains on SleepEDF-78, while Tent and SHOT consistently degrade performance.TFM-Tokenizer shows relatively smaller performance drops across TTA approaches.
- Ear-EEG: Under the unseen Ear-EEG modality shift, SHOT and Tent consistently degrade performance, whereas T3A is more stable and improves some models.T3A improves CBraMod across all metrics and provides moderate balanced-accuracy gains for REVE.
- Batch size: Increasing adaptation batch size does not consistently improve performance; T3A remains insensitive to batch size because it updates class prototypes without gradient optimization.
- Stability: Across methods, T3A causes the least degradation relative to No-TTA and occasionally provides benefits, while gradient-based methods often cause substantial degradation.
(c) CHB-MIT (e) EAREEG (d) SLEEPEDF-78 (b) TUAB (a) TUEV
Across TUEV, TUAB, CHB-MIT, SleepEDF-78, and EarEEG, EEG foundation models are not plug-and-play under realistic shifts; optimization-free TTA is more stable, but the benchmark remains limited in coverage and deployment analysis.
- Discussion and Implications: Performance degrades substantially under out-of-distribution conditions, particularly under the extreme EarEEG shift.
- Discussion and Implications: Optimization-free TTA methods exhibit greater stability than gradient-based approaches, which are prone to performance degradation.
- Discussion and Implications: Representation type influences adaptation behavior, motivating TTA methods tailored to continuous or discrete EEG foundation-model representations.
- Limitations: The benchmark covers three TTA methods, four EEG foundation-model variants, and five downstream datasets, but not the full space of methods and model families.
- Limitations: Computational costs, especially for larger models such as REVE-Large, can make adaptation memory-intensive and constrain batch sizes and hardware accessibility.
5. Conclusion
NeuroAdapt-Bench evaluates test-time adaptation for EEG foundation models under realistic distribution shifts. Standard TTA methods are inconsistent and can degrade performance, while optimization-free approaches are more stable and often improve results.
- NeuroAdapt-Bench systematically evaluates TTA methods for EEG foundation models under realistic distribution shifts.
- Standard TTA methods yield inconsistent gains and often degrade performance relative to the No-TTA baseline.
- Optimization-free approaches such as T3A show greater stability and often positive gains.
- Gradient-based methods are more prone to degradation than optimization-free approaches.
- The findings motivate EEG-specific test-time adaptation approaches and provide empirical guidance for reliable adaptation strategies.
A.1. Dataset Preprocessing and Split Policy
The benchmark standardizes temporal support and amplitude scaling while preserving dataset-specific channel geometry. Table 3 documents the resulting preprocessing and split policy.
- Preprocessing standardizes temporal support and amplitude scaling across datasets.
- Dataset-specific channel geometry is preserved rather than forcing all recordings into a single global montage.
- Table 3 summarizes the benchmark’s dataset preprocessing and split policy.
A.2. Shared Downstream Classifier and Fine-Tuning
The benchmark uses a shared lightweight classifier head to compare pretrained EEG backbones under a common downstream protocol. Evaluation aggregates results across adaptation batch sizes and study seeds, reporting deltas relative to No-TTA.
- Each foundation model uses the same lightweight classifier head for common downstream comparison.The head applies LayerNorm, projects to 128 dimensions, uses GELU and dropout, and maps features to task logits.
- Encoder backbones remain frozen during the reported downstream fine-tuning experiments.REVE additionally consumes channel-position information, while other backbones use waveform input only.
- The benchmark uses method configurations and evaluation settings summarized in Tables 5 and 6.
- Results report performance deltas relative to the No-TTA baseline for individual datasets, separated by foundation model and aggregated across seeds and adaptation batch sizes.Values are reported as mean ± standard deviation; the evaluated batch sizes are 64, 128, and 256.
- The benchmark includes reported delta tables for TUEV, TUAB, CHB-MIT, Sleep-EDF, EAR-EEG, EarEEG, SleepEDF-78, and TUEV.