PaperlayerView exact revision on arXiv

Source-linked AI summary

EEG-Fusion: Failure-Informed Source-Free Expert Routing for Robust Motor Imagery EEG Decoding

Abdul Basit, Saim Rehman, Muhammad Shafique

arXiv:2609.33962v1cs.HCcs.AIcs.LGeess.SP

TL;DR

Subject-independent MI-EEG decoders can fail for individual users under subject shift, even when average performance looks acceptable, while source-free deployment lacks target labels for adaptation and expert selection. EEG-Fusion estimates candidate reliability from unlabeled stream diagnostics and routes among experts; across completed LOSO protocols it improves mean subject macro-F1 and reduces collapse versus a NoAlign EEGNet anchor.

  • Problem

    Source-free MI-EEG decoding can look acceptable on average while producing a collapsed prediction distribution for a held-out subject, with target labels unavailable during deployment.

    Source: §I · p. 1

  • Method

    EEG-Fusion predicts adapted-decoder quality from unlabeled target diagnostics, penalizes observed collapse, and routes to the highest-scoring candidate.

    Source: §I · p. 1

  • Results

    Across completed LOSO protocols, EEG-Fusion improves mean subject macro-F1 by 0.080–0.166 and lowers collapse versus the NoAlign EEGNet source-free anchor.

    Source: §I · p. 2

  • Takeaways & Limitations

    EEG-Fusion contributes an auditable reliability layer above adaptation, using prediction-stream diagnostics to expose failures hidden by aggregate validation under known class balance.

    Source: §VI · pp. 6–7

    The three core protocols contain only nine held-out subjects, and collapse cannot distinguish model failure from legitimate target imbalance.

    Source: E. Limitations and Deployment Scope · p. 6

Abstract

from arXiv · show

Subject-independent motor-imagery (MI) EEG decoding can exhibit subject-level failures even when average performance appears acceptable: under subject shift, a decoder can become an overconfident near-one-class predictor. This is especially problematic in source-free deployment, where target-user labels are unavailable during adaptation and expert selection. We present \textit{EEG-Fusion}, a failure-informed decision-level fusion framework that treats source-free MI decoding as label-free reliability estimation over heterogeneous experts. EEG-Fusion applies subject-wise Euclidean alignment and normalization-only test-time adaptation, then routes each target subject to a neural, covariance-based, or physiological-feature expert using a reliability gate trained on source-held-out folds to predict expert performance and collapse risk from label-free stream diagnostics. The gate uses confidence, entropy, prediction diversity, expert agreement, and predicted class balance; collapse is measured as the maximum predicted class fraction. In 9-fold leave-one-subject-out (LOSO) evaluation with three seeds, relative to a no-alignment raw EEGNet source-free anchor, EEG-Fusion improves subject macro-F1 from 0.417 to 0.529 on BCI IV-2a local protocol, from 0.314 to 0.482 on BNCI2014-001, and from 0.607 to 0.708 on BNCI2014-004; corresponding collapse-index reductions are 0.199, 0.227, and 0.169. In a 9-subject Cho2017 external subset, EEG-Fusion improves macro-F1 from 0.516 to 0.630. These results suggest that label-free reliability estimation can reduce subject-level failure modes in source-free MI-EEG deployment.

I. INTRODUCTION

Source-free MI-EEG decoding can look acceptable on average yet collapse to one predicted class for a held-out subject. EEG-Fusion frames this as label-free reliability estimation and routes among experts using stream diagnostics; across completed LOSO protocols it improves mean subject macro-F1 and reduces collapse relative to the NoAlign EEGNet anchor.

I. INTRODUCTION

  • A decoder can appear acceptable on average while producing a collapsed prediction distribution for a held-out subject; the class-collapse index is the maximum predicted class fraction.The index is read from the unlabeled prediction histogram.Source: §I · p. 1
  • EEG-Fusion runs three adapted decoders, predicts likely performance from unlabeled target diagnostics, penalizes observed collapse, and selects the highest-scoring candidate.Candidates include EEGNet, SimpleConv, and a multi-kernel temporal CNN after the same safety BN-TTA.Source: §I · p. 1
  • A source-held-out reliability gate predicts candidate performance and combines it with observed label-free collapse to route unseen target subjects without target labels.A controlled extension also evaluates architecture–adaptation-policy candidates.Source: §I · pp. 1–2
  • The deployment protocol combines subject-wise Euclidean alignment with normalization-only test-time adaptation and records diagnostics, selected states, and routing decisions before target labels are accessed.The evaluation compares routed neural architectures with filter-bank, covariance-based, and physiological-feature comparators under LOSO.Source: §I · p. 2
  • Across completed LOSO protocols, EEG-Fusion improves mean subject macro-F1 by 0.080–0.166 and lowers class collapse versus the NoAlign EEGNet source-free anchor.Full 52-subject Cho2017 is identified as the primary external dataset check.Source: §I · p. 2

II. RELATED WORK

Prior MI-EEG work offers classical and deep decoding methods, cross-subject adaptation, and foundation models, but primarily evaluates aggregate performance. EEG-Fusion instead treats source-free deployment as subject-level reliability estimation and expert selection, a layer designed to remain distinct from the choice of raw-EEG backbone.

II. RELATED WORK

  • Classical pipelines and compact deep models offer different inductive biases, while benchmark rankings depend strongly on dataset, preprocessing, and protocol.EEG-Fusion routes among compact neural architectures and retains classical families as comparators.Source: §II · p. 2
  • Existing domain-generalization and test-time adaptation methods address unseen-subject invariance or target-time prediction optimization, whereas EEG-Fusion learns source-held-out subject-level quality estimates for candidate selection.The gate uses unlabeled stream diagnostics to predict candidate quality and observe collapse.Source: §II · p. 2
  • EEG-Fusion is a deployment-level reliability layer rather than a new raw-EEG architecture, and adding a pretrained model requires source-held-out reliability records and controlled channel/preprocessing evaluation.Large EEG representation models may change which raw-EEG backbone is strongest.Source: §II · p. 2

A. Problem Setup

The source-free, batch-transductive LOSO setup evaluates an unseen subject using an unlabeled target stream before final scoring. Its safety objective includes reducing concentrated prediction distributions, with collapse measured from the prediction histogram.

A. Problem Setup

  • The target stream is available without labels for alignment, adaptation, diagnostics, and routing before final scoring.Source-side candidates and reliability gates are prepared before target-time deployment.Source: A. Problem Setup · p. 2
  • The evaluation objective is to preserve subject-level macro-F1 while reducing degenerate prediction distributions.At target time, this objective is approximated with label-free diagnostics and source-held-out reliability estimates.Source: A. Problem Setup · p. 2
  • Class-collapse index ρ is the maximum predicted class fraction; ρ = 1 indicates a one-class predictor, while lower values indicate more diverse predictions.For unknown or naturally imbalanced priors, ρ is a concentration alert rather than evidence of error.Source: A. Problem Setup · pp. 2–3

B. Alignment-First Source-Free Preprocessing

EEG-Fusion applies unsupervised Euclidean alignment independently to each subject or session, using that stream’s trials without labels. The reference covariance includes identity shrinkage, and the final runs use trace normalization.

B. Alignment-First Source-Free Preprocessing

  • The subject/session reference covariance uses a small identity-shrinkage coefficient γ.The passage specifies the reference covariance as a combination of the average trial covariance and an identity term.Source: B. Alignment-First Source-Free… · p. 3
  • Each subject/session is aligned using only its own trials and no labels, with alignment computed independently for source and target streams.Final runs use trace normalization, γ = 10^-3, and ϵc = ϵ = 10^-6.Source: B. Alignment-First Source-Free… · p. 3

C. Candidate and Comparator Bank

EEG-Fusion’s primary routed bank consists of three neural candidates, while classical, covariance, and physiological models serve as independent comparators. Neural candidates use restricted normalization-only test-time adaptation on unlabeled target batches.

C. Candidate and Comparator Bank

  • The routed neural candidates are raw-input EEGNet, SimpleConv, and a compact multi-kernel temporal CNN, each after safety BN-TTA.The policy study treats architecture–adaptation pairs as distinct candidates.Source: C. Candidate and Comparator Bank · p. 3
  • CSP+LDA, covariance logistic regression, physiological-feature logistic regression, physiological-plus-covariance logistic regression, and LogEuclidean MDM are independent comparators, not routed candidates or gate inputs.Conventional features are retained as comparator inputs, not as a standalone fusion claim.Source: C. Candidate and Comparator Bank · p. 3

D. Normalization-Only Test-Time Adaptation

  • Neural-candidate TTA updates batch-normalization affine parameters and running statistics rather than the full backbone, using unlabeled target mini-batches.The paper calls this update scope “parameter-scoped” without claiming a measured runtime advantage.Source: D. Normalization-Only Test-Time… · p. 3
  • For the feature-gated diagnostic baseline, the adaptation objective includes a gate regularizer; it is zero for non-gated candidates.The negative marginal-entropy term encourages class diversity.Source: D. Normalization-Only Test-Time… · pp. 3–4
  • TTA state selection uses the full unlabeled target stream and restores the highest-scoring state, with early stopping and rollback if raw ρ exceeds a protocol-local threshold.The confidence term is weak and is considered jointly with histogram entropy and collapse penalties.Source: D. Normalization-Only Test-Time… · p. 4

E. Failure-Informed Reliability Gate

The reliability gate estimates candidate macro-F1 from unlabeled diagnostics using source-held-out pseudo-target records, then penalizes observed collapse when ranking candidates. If the top-ranked candidate crosses the collapse threshold, the system uses an adapted raw-input EEGNet fallback.

E. Failure-Informed Reliability Gate

  • The gate predicts each candidate’s macro-F1 from unlabeled diagnostics, observes collapse directly, and selects the highest score after a collapse penalty.The reliability records come from source-held-out pseudo-target folds; their labels are used only to compute the quality target.Source: E. Failure-Informed Reliability Gate · p. 4
  • A source-standardized ridge regressor predicts candidate quality using diagnostics and one-hot model identity, with target-fold records excluded from gate fitting and hyperparameter selection.Diagnostics include confidence, entropy, class fractions, observed collapse, diversity, and cross-candidate agreement.Source: E. Failure-Informed Reliability Gate · p. 4
  • If the top-ranked candidate’s observed collapse reaches κ, routing switches to the aligned raw-input EEGNet after safety BN-TTA and best-state restoration.Final runs use λ = 0.25, τ = 0.60, and κ = 0.85.Source: E. Failure-Informed Reliability Gate · p. 4

F. Auditability and Label Hygiene

Target labels are withheld until predictions are fixed, separating adaptation and routing from post-hoc evaluation.

F. Auditability and Label Hygiene

  • Target labels enter only after predictions are written; manifests and saved predictions, diagnostics, routing choices, and metrics make adaptation, routing, and evaluation auditable.Each fold/seed/model produces a manifest and records pre/post-TTA predictions, adaptation diagnostics, model records, choices, and metrics.Source: F. Auditability and Label Hygiene · p. 4

IV. EXPERIMENTAL SETUP

The experiments use subject-independent LOSO evaluation with fixed source-only splits and target labels withheld until predictions are fixed. The protocols span local and MOABB IV-2a variants and Cho2017, with dataset-specific preprocessing.

A. Datasets and Protocol

  • In each LOSO fold, one subject is the target, the next cyclic subject validates, and the remaining subjects train; target labels are used only after predictions are fixed.Local BCI IV-2a and BNCI2014-001 differ in loader, sessions, and preprocessing, so they serve as a protocol/reproducibility check rather than independent biological evidence.Source: A. Datasets and Protocol · p. 4
  • All protocols use 4–40 Hz signals resampled to 128 Hz; Table II covers 9-fold core protocols and 52-subject Cho2017, with representative trial counts.The local IV-2a path uses 22 EEG channels and a 0.5–3.5 s cue-locked window; MOABB preprocessing follows dataset-specific defaults without added notch or ICA.Source: A. Datasets and Protocol · pp. 4–5, Table II · p. 5

B. Baselines and EEG-Fusion Configurations

EEG-Fusion routes among three adapted neural candidates, with classical and feature-gated models as comparators. The setup logs reproducibility artifacts and evaluates subject-level performance and collapse using paired statistical procedures.

B. Baselines and EEG-Fusion Configurations

  • Three neural candidates—EEGNet, SimpleConv, and a multi-kernel temporal CNN—form the routed set after safety BN-TTA; classical and feature-gated models are independent comparators.The fallback is EA+Safe-BN EEGNet after best-state restoration; the temporal CNN uses multiple temporal kernels on the same 4–40 Hz input, not separate sub-bands.Source: B. Baselines and EEG-Fusion… · p. 5

C. Reproducibility Artifacts

  • Each run logs a manifest, predictions, reliability records, routing choices, and fold metrics; an uncontended three-fold IV-2a GPU profile takes 4.66±0.30 s across three candidates.Routing overhead is negligible; Q10 BN-TTA accounts for 4.42±0.24 s of the measured runtime.Source: C. Reproducibility Artifacts · p. 5

D. Metrics and Statistics

  • Primary metrics are per-subject macro-F1 and raw collapse index, with paired subject-level bootstrap and Wilcoxon procedures; Holm-adjusted tests support primary inference.The collapse index is the maximum predicted-class fraction; source-pool stability and selection regret are also evaluated.Source: D. Metrics and Statistics · p. 5

V. RESULTS AND DISCUSSION

EEG-Fusion improves F1 and reduces collapse relative to the NoAlign EEGNet anchor, while routing adds smaller gains beyond alignment and adaptation. Routing behavior is useful but imperfect, with performance depending on dataset and source-pool composition.

A. Main LOSO Results

  • Relative to NoAlign EEGNet, EEG-Fusion increases F1 by 0.080 [0.051, 0.110] and reduces collapse by 0.183 [0.149, 0.216]; no n = 9 pipeline-versus-anchor macro-F1 test remains significant after Holm correction.On full Cho2017, routing over EA+Safe-BN EEGNet changes F1 by +0.0004 (adjusted p = 0.755) and reduces collapse by 0.014 [0.0067, 0.0223] (adjusted p = 0.0026). Fig. 3 depicts the accuracy–reliability tradeoff.Source: A. Main LOSO Results · p. 5, Figure 3 · p. 6, Table III · p. 6

B. What Actually Drives the Gain?

  • Alignment and BN-TTA supply most F1 gain, while routing adds 0.001–0.046 F1 and reduces collapse by 0.020–0.101 across core protocols.Fallback activates only on BNCI2014-001/004; CSP/LDA has the lowest BNCI2014-004 collapse.Source: B. What Actually Drives the Gain? · p. 6

C. Routing Behavior and Failure Cases

  • Replacing diagnostic F1 prediction with identity while retaining collapse control shifts F1 by –0.001 to +0.007; predicted quality correlates with post-hoc F1 (r = 0.75, MAE 0.079), but selection remains imperfect.Oracle top-1 is 44%–74%, regret is 0.004–0.021, and regret begins to plateau with 6–8 source subjects.Source: C. Routing Behavior and Failure Cases · p. 6

D. Adaptation and Robustness Checks

The section reports robustness checks, deployment boundaries, and the conclusion that EEG-Fusion adds an auditable reliability layer above adaptation. Its evidence favors routing over the NoAlign anchor while showing that policy performance depends on dataset and class-prior conditions.

D. Adaptation and Robustness Checks

  • Under an imposed majority fraction of 0.85, F1 falls to 0.593 on BNCI2014-004 and 0.533 on Cho9, motivating prior-aware thresholds.The routing audit covers selection rates, mean regret, and imperfect predicted candidate F1; the fixed 34-routing grid spans at most 0.014 F1, 0.040 collapse, and 0.014 regret.Source: D. Adaptation and Robustness Checks · p. 6, Figure 4 · p. 7
  • Q15 costs 0.31 s more than Q10 for +0.005 F1 [–0.008, 0.016] (p = 0.419), making Q10 a practical elbow in the IV-2a step-budget sweep.Q3/Q5 are faster but less reliable; adaptation dominates target-time cost, and deployment remains batch-transductive.Source: Table IV · p. 6, D. Adaptation and Robustness Checks · p. 6

E. Limitations and Deployment Scope

  • With unknown class priors, collapse cannot distinguish model failure from legitimate target imbalance, and probability-based diagnostics may be affected by calibration shift.The three core protocols contain only nine held-out subjects; full Cho2017 provides the strongest external evidence.Source: E. Limitations and Deployment Scope · p. 6

VI. CONCLUSION

  • Across LOSO evaluations, EEG-Fusion yields higher mean macro-F1 and lower collapse than NoAlign EEGNet; on Cho52, it preserves EA+Safe-BN F1 while significantly reducing collapse.The conclusion attributes most gain to alignment and test-time adaptation; matched TENT’s rank reverses across datasets, so no tested policy uniformly dominates.Source: §VI · p. 6, D. Adaptation and Robustness Checks · p. 6
  • EEG-Fusion’s contribution is an auditable reliability layer above adaptation, with label-free diagnostics exposing failures under known class balance.Unknown priors and calibrated policy banks remain important steps toward online deployment.Source: §VI · pp. 6–7

Related papers

Loading 2609.33962v1…