PaperlayerView exact revision on arXiv

Source-linked AI summary

DEFT: Disentanglement-Enhanced Fine-Tuning for EEG Foundation Models

Yuqiu Deng, Muyun Jiang, Zongpeng Zhang, Mingqing Xiao, Yansen Wang, Peiliang Gong, Xinliang Zhou, Yi Ding, Ziyu Jia, Cuntai Guan, Chenyu Liu

arXiv:2610.14497v1cs.LG

TL;DR

Pretrained EFM embeddings can make subject-related structure highly accessible, although that structure may not be the intended target in cross-subject decoding. DEFT separates tokens into content and style branches and predicts from content; it improves matched baselines across task families and backbones.

  • Problem

    Pretrained EFM embeddings can make subject-related structure highly accessible to downstream classifiers even when it is not the intended target of cross-subject decoding.

    Source: §1 · p. 1, §6 · p. 9

  • Method

    DEFT adds a post-backbone adapter that separates mixed tokens into prediction-oriented content and subject-consistent style, then predicts from content.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    DEFT improves matched fine-tuning baselines across multiple EEG task families and four EFM backbones; across 24 matched cells, mean B-ACC improves by 0.0460.

    Source: §1 · p. 3, §4.2 · p. 7

  • Takeaways & Limitations

    Ablations, representation analysis, and content/style probing support the intended branch organization alongside the observed benchmark improvements.

    Source: §6 · p. 9

    DEFT adds loss weights and a temporal-variance prior selected under downstream validation; the prior is intended for settings where subject-linked structure acts as a nuisance.

    Source: §3.3 · p. 6, B Limitations, Ethics, and… · p. 15

Abstract

from arXiv · show

EEG foundation models (EFMs) have improved EEG representation learning by transferring knowledge from large-scale multi-subject pretraining to downstream tasks. However, their pretrained embeddings often retain substantial subject-related variation, placing an additional burden on the downstream classifier, especially in cross-subject settings. To address this issue, we propose Disentangled EFM Fine-Tuning (DEFT), a plug-and-play adapter for EFM fine-tuning. DEFT encourages branch disentanglement of pretrained tokens into task-oriented content and subject-consistent style, then performs downstream prediction from the content branch. It instantiates this factorize-before-classify design with recursive Gaussian refinement, temporal-variance regularization, subject-aware contrastive learning, and reconstruction. Experiments across multiple EEG datasets and four EFM backbones show that DEFT improves matched fine-tuning baselines. Further ablations and analyses support the intended content/style branch organization.

1 Introduction

EEG foundation models transfer representations learned from large multi-subject corpora, but their embeddings can make subject identity more accessible than task labels. DEFT separates pretrained tokens into task-content and subject-style branches before prediction, and improves matched fine-tuning baselines across multiple task families and four backbones.

1 Introduction

  • EFMs learn reusable representations by pretraining on large multi-subject EEG corpora for transfer to downstream settings.Source: §1 · p. 1
  • Pretrained EFM embeddings make subject identity more linearly accessible than raw EEG, while class-label probing improves only modestly, leaving subject and task structure mixed.The resulting subject-related variation can burden cross-subject classification, where the subject cues may be poorly aligned with the desired decision rule.Source: §1 · pp. 1–2
  • DEFT is a post-backbone adapter that separates mixed pretrained tokens into subject-style and task-content branches, then predicts from the content branch.It uses recursive Gaussian disentanglement, temporal-variance constraints, subject-aware contrast, and reconstruction to shape the branches.Source: §1 · pp. 2–3
  • DEFT improves matched EFM fine-tuning baselines across multiple EEG task families and four foundation backbones.Ablations, representation analysis, and probing also support the intended content/style branch organization.Source: §1 · p. 3

2 Preliminary

This section defines the pretrained EFM token representation and contrasts standard fine-tuning with a functional content/style view of those tokens.

2.1 EEG Foundation Model

  • A pretrained EFM backbone maps an EEG trial x to mixed tokens Z, with M channel groups, W temporal windows, and token width D.Source: §2.1 · p. 3

2.2 EFM Fine-Tuning

  • Standard fine-tuning attaches a lightweight classification head to pretrained tokens for task prediction, typically using token aggregation and a linear or shallow MLP classifier.Source: §2.2 · p. 3
  • Content tokens C retain label-relevant brain-state structure and temporal variation, while style tokens S collect trial-stable residual structure organized primarily by subject identity.The prediction head operates on content; temporal-stability and subject-aware constraints shape style.Source: §2.2 · p. 3

3 Methodology

DEFT inserts a factorization step between pretrained tokens and downstream prediction, shaping task-content and subject-style branches through recursive inference, temporal constraints, and external training objectives.

3.1 Framework Overview

  • DEFT factorizes mixed pretrained tokens into task-content and subject-style branches, then uses the content branch alone for downstream prediction.A compact window-level sequence is split into branches and reinjected into the original token grid.Source: §3.1 · pp. 3–4

3.2 Recursive Gaussian Disentangler

  • Four alternating inference steps refine branch estimates using counterpart-subtracted observations, uncertainty-aware Gaussian fusion, and distinct style and content transitions.The style transition is static, while the content transition is a gated mixture of learnable bases.Source: §3.2 · p. 5

3.3 Intrinsic Temporal-Variance Constraint

  • For cross-subject tasks, the label-free, task-dependent temporal prior encourages greater window variation in content and within-trial stability in style.The variance-ordering loss enforces a margin between branch variances, while a separate penalty reduces style variance.Source: §3.3 · pp. 5–6

3.4 External Constraints

  • Subject-aware contrast organizes style by subject, while reconstruction preserves information across the content-style split.Training uses same-subject positives where available; anchors without them are excluded from the contrastive loss, and subject identities are not required at inference.Source: §3.4 · p. 6

4 Experiments

Experiments compare DEFT with matched fine-tuning across six subject-disjoint EEG datasets and four foundation backbones. DEFT improves overall B-ACC, while ablations and representation analyses examine component sensitivity and branch organization.

4.1 Experimental Setup

  • The evaluation uses four EFM backbones and six representative subject-disjoint datasets, reporting B-ACC for all tasks and complementary metrics by task type.The datasets cover motor imagery, emotion recognition, sleep staging, workload assessment, speech decoding, and MDD classification.Source: §4.1 · pp. 6–7

4.2 Main Results

  • Across 24 matched backbone–dataset cells, DEFT raises mean B-ACC by 0.0460 (cluster-bootstrap 95% CI [0.0240, 0.0745]).The main comparison uses multi-seed results over available seeds.Source: Table 1 · p. 7, §4.2 · p. 7
  • Gains span distinct EEG task families, with the largest mean B-ACC gains on MentalArith., BCIC-IV-2a, and FACED; most complementary metric values also improve.Backbone-averaged B-ACC gains are +0.0628 for CBraMod, +0.0574 for CSBrain, +0.0425 for CodeBrain, and +0.0211 for LaBraM.Source: §4.2 · p. 8

4.3 Ablation Study

  • Component effects depend on task: BCIC-IV-2a is especially sensitive to invariance, variance-ratio, and Gaussian components, while MentalArith. and BCIC-Speech show substantial reductions across ablations.For BCIC-IV-2a, removing invariance changes B-ACC by −0.1606 (Welch 95% CI [−0.2999, −0.0213]) across three seeds.Source: §4.3 · p. 8

4.4 Representation Analysis

  • The content branch improves the subject-controlled class-separation score on five of six datasets, indicating clearer class-centroid separation after subject offsets are removed.Figure 4 shows the score and class t-SNE visualizations before the module and for the content branch.Source: §4.4 · p. 8, Figure 4 · p. 9
  • BCIC-Speech is the exception: its geometry score falls from 0.405 to 0.307 despite metric gains for all four backbones; probes show content favors task prediction and style favors subject prediction in all six settings.The authors suggest speech cues may be localized across channels and windows, making them less visible as global PCA clusters.Source: §4.4 · pp. 8–9

5 Related Work

Prior EEG foundation models provide reusable pretrained representations, while downstream classification commonly fine-tunes a task head and backbone in the mixed representation space. DEFT targets this adaptation stage by factorizing pretrained tokens before prediction.

EEG Foundation Models

  • Encoder-based EEG foundation models learn reusable representations with varied pretraining objectives, while decoder- or language-connected models broaden the available interfaces.For encoder-based downstream classification, the common recipe remains attaching a task head and fine-tuning the pretrained backbone.Source: §5 · p. 9

Disentanglement Learning

  • DEFT differs from the usual encoder-based fine-tuning recipe by inserting a tailored adapter that factorizes mixed pretrained tokens before prediction.Source: §5 · p. 9
  • Disentanglement methods separate factors such as content and style or identity and task information; EEG approaches also seek subject-invariant representations through adversarial, alignment, or sequential-factorization objectives.DEFT applies factorization after a pretrained EFM backbone and assigns task prediction and subject consistency to separate token branches.Source: §5 · p. 9

6 Conclusion

DEFT inserts a post-backbone adapter that separates pretrained EFM tokens into prediction-oriented content and subject-consistent style, then predicts from content. The authors report improved matched fine-tuning results across task families and four backbones, with analyses supporting the intended branch organization.

6 Conclusion

  • DEFT addresses subject-related structure that can be highly accessible in pretrained embeddings by factorizing tokens into content and style before prediction.The prediction head operates on the content branch; the conclusion reports improved matched fine-tuning results across multiple EEG task families and four backbones.Source: §6 · p. 9

A.1 Scalp EEG Foundation Models

  • Encoder-based EFMs learn transferable token representations with diverse modeling objectives, while downstream classification commonly still feeds pretrained embeddings directly to a task head.DEFT inserts a representation-level adapter between the EFM and prediction head.Source: §A.1 · p. 15

A.2 Disentanglement and Subject-Related EEG Variation

  • Prior work separates latent factors and addresses EEG subject variation with invariant or aligned representations; DEFT factorizes pretrained EFM embeddings into task-oriented content and subject-consistent style.The content branch preserves task-relevant temporal variation, while the style branch collects subject-consistent structure stable within a trial.Source: §A.2 · p. 15

B Limitations, Ethics, and Reproducibility

DEFT’s validation-selected loss weights and temporal-variance prior constrain the training setup; the paper also specifies how it handles subject identities, sensitive data, and dataset usage terms.

B Limitations, Ethics, and Reproducibility

  • DEFT’s loss weights and temporal-variance prior are selected under the downstream validation protocol.For trait-level or demographic targets, validation selects prior strength so stable label-relevant evidence remains in the content branch.Source: B Limitations, Ethics, and… · p. 15
  • With one sample per training subject, same-subject contrastive pairs are unavailable, while classification, variance, and reconstruction objectives remain applicable.Source: B Limitations, Ethics, and… · p. 15
  • Fully unsupervised disentanglement fine-tuning remains future work; training-time regularization is omitted at inference.Source: B Limitations, Ethics, and… · p. 16
  • Subject identities are used only during training; EEG and MDD-related data raise privacy and clinical-interpretation risks, and DEFT is not a diagnostic system without dataset-specific clinical validation.Source: B Limitations, Ethics, and… · p. 16
  • The paper reports approximate compute for the main evaluation, complete seed-0 suite, and diagnostics, and uses datasets under their original terms without redistributing raw data or pretrained checkpoints.Non-commercial, no-derivatives, and academic-only terms are not open commercial licenses; datasets without an identified standard license are reported as such.Source: B Limitations, Ethics, and… · p. 16

C.1 Dataset Statistics

The experiments use standardized EEG tensors and released EFM checkpoints, with backbone-specific changes limited mainly to model input interfaces. Recursive Gaussian inference refines content and style branches through counterpart removal and Gaussian fusion.

C.1 Dataset Statistics

  • Table 5 summarizes dataset tensor shapes, class counts, task types, and subject-disjoint split statistics; ISRUC uses sleep-epoch sequences rather than the usual sample shape.Most other local sample shapes follow (channels, temporal segments, samples per segment).Source: Table 5 · p. 17, §C.1 · p. 17

C.2 Backbone-Specific Preprocessing

  • Inputs reuse the corresponding foundation-model pipelines, with fixed-duration EEG tensors, labels, subject identifiers, and split keys; most samples use 200 Hz resampling and one-second patches.Source: §C.2 · p. 18
  • Preprocessing differences are chiefly interface-specific reshaping, channel metadata, resampling, or tokenizer requirements, while signal-level preprocessing is shared whenever possible.CSBrain reshapes samples for its tokenization; LaBraM maps channel names to standard indices, and BCIC-Speech uses a corrected 60-channel, three-window format.Source: §C.2 · p. 18

D Implementation Details

  • Experiments use released CodeBrain, CBraMod, LaBraM, and CSBrain checkpoints without retraining pretraining; matched comparisons use the same checkpoint, split, and trainable-backbone policy.Source: D Implementation Details · p. 18

D.1 Details of Recursive Gaussian Inference

  • Recursive Gaussian refinement updates content and style using counterpart-removed observations, with a learnable gate controlling how much opposite-branch information is subtracted.The posterior mean combines temporal prediction and the counterpart-removed observation through Gaussian fusion.Source: §D.1 · p. 18
  • The recursive procedure reinjects refined window-level content and style estimates into the token grid; a lightweight style-conditioned FiLM head reconstructs mixed windows while content retains window-varying information.Source: §D.1 · p. 19

D.2 Representation Geometry Metric

The appendix defines a post-hoc score for within-subject class separation, documents validation-only hyperparameter selection, and reports adapter parameter and compute overhead.

D.2 Representation Geometry Metric

  • The post-hoc representation score averages subject-level class-centroid separation relative to within-class dispersion after subject offsets are removed.Representations are standardized and projected to at most 50 PCA dimensions; larger scores indicate greater relative class separation.Source: §D.2 · p. 19

D.3 Training Hyperparameters

  • Hyperparameters and checkpoints are selected only on the validation split for each subject-disjoint partition, and the reported settings specify token-grid inputs and content-only prediction.Source: §D.3 · p. 20

D.4 Parameter Overhead

  • The active adapter adds 2.07M parameters, or 1.55%–10.81% of the matched original model size across six CBraMod settings.The percentage varies with the downstream prediction head, while the CBraMod backbone and adapter size remain fixed.Source: §D.4 · p. 20, Table 8 · p. 21

D.5 Compute and Memory Overhead

  • The compute diagnostic measures parameter, memory, and one-step training FLOP overhead for a representative CBraMod setting; the table reports approximate values.Source: §D.5 · p. 20, Table 9 · p. 21

E Complete Main Results

Complete seed-0 results show gains across backbones and task families, with variation by dataset and metric; separate probes show functional specialization of the content and style branches.

E Complete Main Results

  • CodeBrain shows clear gains on BCIC-IV-2a, FACED, MentalArith., BCIC-Speech, and ISRUC-S1 across five EEG task families.Source: E Complete Main Results · p. 21, Table 10 · p. 22
  • CBraMod shows the strongest breadth pattern in the seed-0 records, with gains across several task families and a small number of metric-level regressions.Source: E Complete Main Results · p. 23, Table 11 · p. 23
  • LaBraM’s more conservative gains are most visible on BCIC-IV-2a, FACED, MI-KoreaU, PhysioNet, BCIC-Speech, and ISRUC-S1, while some probabilistic metrics remain challenging.Source: E Complete Main Results · p. 24, Table 12 · p. 24
  • CSBrain shows clear gains on BCIC-IV-2a, MI-KoreaU, MentalArith., BCIC-Speech, and ISRUC-S1; lower values remain in isolated secondary metrics or high-baseline binary settings.Source: E Complete Main Results · p. 25, Table 13 · p. 25
  • Across the complete seed-0 evaluation suite, DEFT’s gains vary by dataset, backbone, and metric, with remaining decreases concentrated in isolated high-baseline or calibration-sensitive metrics.Tables 10–13 report all recorded metrics for vanilla fine-tuning and DEFT within each fixed backbone.Source: E Complete Main Results · pp. 21, 25, Table 10 · p. 22, Table 11 · p. 23
  • On four CBraMod diagnostic datasets, DEFT decreases normalized training loss faster and reaches a lower final loss than vanilla fine-tuning.The curves normalize each method’s loss by its first-epoch loss; the figure attributes additional optimization signals to DEFT’s auxiliary constraints.Source: E Complete Main Results · p. 25, Figure 5 · p. 26

F.1 Content/Style Probing Diagnostic

  • Across six settings, content has higher task-probe B-ACC and style higher subject-probe B-ACC; on Mumtaz2016, task probes are 0.9224/0.9759/0.9298 for original/content/style.The corresponding subject probes are 0.3720/0.0633/0.1999.Source: §F.1 · p. 27, Table 14 · p. 27

F.2 Parameter and Sampler Controls

The controls compare DEFT with matched-capacity, subject-aware, and sequential baselines, and examine Gaussian refinement and objective components.

F.2 Parameter and Sampler Controls

  • On CBraMod–FACED, DEFT reached 0.6005 B-ACC, versus 0.5387 for Vanilla, 0.5363 for a doubled classifier, and 0.5441 with the subject-balanced sampler.The parameter-matched Generic Adapter retains the task loss and uses the same subject-disjoint split, sampler, and backbone update policy as DEFT.Source: §F.2 · p. 27

F.3 Subject-Aware and Sequential Baselines

  • On BCIC-IV-2a, DEFT reached 0.5579±0.0310 with CBraMod and 0.5671±0.0032 with LaBraM, exceeding the reported GRL and FHVAE-token values.The baselines use the same subject-disjoint splits and samplers.Source: §F.3 · p. 27

F.4 RGD Isolation and Sensitivity

  • On BCIC-IV-2a, DEFT reached 0.5579/0.5671 with CBraMod/LaBraM, versus 0.4395/0.5208 for the attention control; refinement alternatives reduced FACED B-ACC.Fixed gate, fixed precision, static transition, and shared transition reduced FACED B-ACC by 0.0104–0.0305 for CBraMod and 0.0226–0.0404 for LaBraM; four refinement steps gave the highest mean B-ACC in tested settings.Source: §F.4 · p. 28

F.5 Objective and Cross-Backbone Controls

  • The objective contributions varied by task: variance-only improved FACED, while the full objective gave the highest result on both datasets.Source: §F.5 · p. 28

F.6 Additional Backbone and Dataset Checks

Additional checks cover another EFM, an independent MDD dataset, a non-EEG time-series benchmark, subject probes, and temporal diagnostics.

F.6 Additional Backbone and Dataset Checks

  • The additional checks cover REVE compatibility, OpenNeuro Depression REST, and MOMENT representations on UCI HAR, spanning one additional EFM, one independent MDD dataset, and one pretrained time-series benchmark pair.Table 21 reports three subject-disjoint REVE compatibility checks; Table 22 covers additional subject-disjoint checks.Source: §F.6 · pp. 28–29, Table 21 · p. 29, Table 22 · p. 29

F.7 Probe and Sampler Diagnostics

  • Frozen subject-probe results show that subject-identity accessibility depends on both backbone and dataset, while the subject-balanced sampler gives each anchor at least one same-subject positive in the six main datasets.Anchors without positives are excluded from the contrastive loss.Source: §F.7 · p. 29, Table 23 · p. 29, Table 24 · p. 29

F.8 Temporal and Statistical Diagnostics

  • Full samples accumulate more task evidence than one-second windows in both datasets, while subject information is accessible from short windows; measured RGD gradient ratios and task-gradient cosine similarities were also reported.At the measured checkpoint, Lord and Lsty had weighted RGD-gradient/task-gradient ratios of 0.0167 and 5.41 × 10−5, with task-gradient cosine similarities of 0.2420 and 0.1246.Source: §F.8 · p. 29

F.9 Additional CBraMod Ablation Results

The appendix reports seed-0 CBraMod ablation details and checklist information on reproducibility, limitations, and societal impacts.

F.9 Additional CBraMod Ablation Results

  • Table 27 provides complete seed-0 CBraMod ablations, while the fixed-seed comparison in Table 2 supports the main text’s task-dependent component analysis.Table 27 separates style-contrast and variance constraints in Panel A from Gaussian-disentangler and reconstruction ablations in Panel B.Source: §F.9 · pp. 30–31, Table 27 · p. 30

NeurIPS Paper Checklist

  • The authors identify validation-selected loss weights, a temporal-variance prior, and training-time subject identities, with fully unsupervised disentanglement fine-tuning remaining future work.Source: NeurIPS Paper Checklist · p. 32
  • The checklist states that the main comparison is reproducible from the described setup, with split statistics, hyperparameters, implementation details, and code access provided across the paper and appendices.It also reports mean and standard deviation over available seeds for the main comparison and GPU-hour accounting for reproducing evaluations and diagnostics.Source: NeurIPS Paper Checklist · pp. 33–35
  • The checklist flags risks of inappropriate clinical interpretation and privacy-sensitive use of EEG/MDD-related data.Source: NeurIPS Paper Checklist · p. 36

Related papers

Loading 2610.14497v1…