Source-linked AI summary
Parameter-Efficient Self-Supervised Adaptation for EEG-FM under Fixed Computational Budgets
Meghal Dani, Stefanie Liebe
TL;DR
The paper asks whether EEG foundation models can be aligned to diverse clinical target domains without the computational and annotation burden of full retraining. It adapts only a small parameter subset using unlabeled target EEG and evaluates frozen representations across models, tasks, and distribution conditions. SSL adaptation consistently improves linear probing, reaches peak performance with 20−50% of available unlabeled data, and shows no significant patient-count effect when total windows are fixed.
Problem
EEG foundation models have limited generalization across clinical datasets, while full fine-tuning is impractical under resource constraints and annotated EEG is rare.
Method
The method adapts BIOT and CBraMod with parameter-efficient SSL on unlabeled target-domain EEG, updating the final encoder layer before frozen-encoder linear probing.
Results
SSL adaptation consistently outperforms linear probing across three clinical datasets and two models, with gains up to 20× AUCPR and peak performance using 20−50% of available unlabeled data.
Takeaways & Limitations
Parameter-efficient SSL adaptation provides a lightweight way to improve EEG-FM representations for clinical target domains, with temporal window coverage more influential than patient count under fixed compute.
Takeaways & Limitations
The patient-count finding is based on a limited CHB-MIT patient range of 3–22, and robustness across compute regimes remains future work.
Abstract
from arXiv · showhide
EEG foundation models pretrained via self-supervised learning promise transferable representations, but their generalization remains limited, especially across diverse clinical datasets. Full fine-tuning is impractical for resource-constrained clinical settings due to high computational requirements. In this work, we investigate whether parameter-efficient self-supervised adaptation, updating only 9% of parameters suffices to align representations to target tasks. We evaluate our method on two state-of-the-art models with different pretraining objectives: BIOT (contrastive) and CBraMod (masked reconstruction), and evaluate on three clinical EEG datasets for abnormality detection (TUAB), event classification (TUEV), and seizure detection (CHB-MIT) under both in-distribution and out-of-distribution conditions. SSL adaptation yields consistent gains over linear probing, up to 20x AUCPR. Under a fixed compute budget, peak performance requires only 20--50% of available unlabeled data. Critically, when total window count is fixed, performance remains invariant to patient count, suggesting that performance is dependent on overall temporal window diversity only. Our findings demonstrate that parameter-efficient adaptation enables effective deployment of EEG Foundation models (EEG-FM) with minimal computational overhead and data collection burden. Code available at: https://github.com/c3n-group/efficient-eeg-adapt
1 Introduction
EEG foundation models use self-supervised pretraining to learn transferable representations from abundant unlabeled recordings, but clinical deployment still requires adaptation to target-domain shifts. This work studies parameter-efficient SSL adaptation across two models, three clinical tasks, and both in-distribution and out-of-distribution settings.
- Unlabeled EEG recordings are abundant at clinical sites, but annotated datasets are rare, limiting supervised automation approaches.
- EEG foundation models are pretrained on massive unlabeled corpora via self-supervised learning to produce transferable representations without expert annotations.
- Contrastive learning and masked reconstruction are the two SSL objective families represented by BIOT and CBraMod, respectively.
- Clinical deployment faces distribution shifts from recording hardware, channel montages, and sampling rates, while retraining from scratch is impractical because of annotation costs.
- The study adapts BIOT and CBraMod using only unlabeled target-domain EEG, updates the final encoder layer, and evaluates representations with frozen-encoder linear probing.
- Up to 20× AUCPR gains, near-peak performance with 20−50% of unlabeled data, and window-count dependence under fixed compute are reported across three clinical datasets.
2 Method
The method performs parameter-efficient SSL adaptation on unlabeled target-domain EEG, then evaluates the resulting frozen representations with linear probing under normalized compute. The protocol separates computational budget from unique-window and patient diversity.
- Method: The pipeline has two stages: parameter-efficient SSL representation adaptation followed by linear probing for downstream evaluation.
- Parameter-Efficient SSL Adaptation: The encoder maps an EEG segment x ∈ R^C×T to a representation h = fθ(x) ∈ R^d.
- Parameter-Efficient SSL Adaptation: Only the final encoder layer is trainable while lower layers remain frozen, preserving the original SSL objective for BIOT or CBraMod.
- Downstream Evaluation: Linear probing freezes the adapted encoder and optimizes only a linear classifier on labeled EEG segment-class pairs using task-specific cross entropy.
- Normalized Evaluation: The protocol normalizes adaptation using optimizer update steps and decomposes data diversity into unique EEG windows and unique patients.
- Normalized Evaluation: C = N × E defines the fixed computational budget as total EEG windows seen, while r = C/N quantifies average processing repetitions per unique window.
- Normalized Evaluation: Varying N while adjusting E at fixed C tests how much unique unlabeled data is needed and separates repetition from diversity effects.
- Patient vs. Window Diversity: Patient-versus-window diversity is tested by fixing N = P × W and C while varying patient count, with smaller P yielding more windows per patient.
3 Experiments and Results
Across three clinical EEG tasks and two foundation models, parameter-efficient SSL adaptation improves linear-probe performance while updating only a small parameter subset. Under fixed compute, performance saturates with partial data, remains stable across patient counts, and final-layer adaptation offers the best cost-performance balance.
- Clinical-task evaluation: SSL adaptation improves linear probing across TUAB, TUEV, and CHB-MIT for BIOT and CBraMod, updating only 28.8% and 9.0% of parameters, respectively.The reported gains range from +1.79 weighted F1 to +36.2 AUCROC.
- Clinical-task evaluation: +36.2 AUCROC is achieved by BIOT on CHB-MIT, increasing AUCROC from 46.93 to 83.08%.
- Clinical-task evaluation: 31.52% AUCPR is achieved by BIOT after adaptation on CHB-MIT, compared with 1.53% for frozen-feature linear probing.CHB-MIT has 1.4% seizure prevalence, and the frozen BIOT result is near the 1.48% random-classifier precision baseline.
- Parameter efficiency: Final-layer adaptation provides the optimal cost-performance balance, with only marginal BIOT gains and degraded CBraMod performance when more layers are unfrozen.On CHB-MIT, updating additional layers yielded +1.17% AUCPR for BIOT and reduced CBraMod AUCPR from 31.25% to 29.41%.
- Data efficiency: 20–50% of available unlabeled data reaches peak adaptation performance under fixed compute, after which additional unique data provides no measurable benefit.The fixed budget is C=N×E, so using more unique windows reduces repetition per window.
- Patient versus window diversity: Increasing patient count produces no significant performance change when total windows and compute are fixed across CHB-MIT, TUEV, and TUAB.The tested patient-count ranges were 3–22, 40–230, and 900–1800, respectively.
4 Conclusion
Parameter-efficient SSL adaptation updates only 9% of EEG-FM parameters while improving performance across clinical tasks. Under fixed compute, strong performance uses 20–50% of available data, but the patient-composition finding is limited in CHB-MIT.
- 4 Conclusion: 20–50% of available data is sufficient for peak performance under the fixed-compute protocol.
- 4 Conclusion: Performance does not significantly differ across patient-window compositions when compute and total samples are fixed, suggesting temporal window coverage may matter more than patient diversity.For CHB-MIT, this observation is based on a limited patient range of 3–22.
- 4 Conclusion: Future work should test alternative parameter-efficient methods, other compute regimes, and whether the data-efficiency patterns generalize beyond EEG foundation models and medical time series.