EEG-Xplain: Decoding Neural Black-Boxes of EEG Foundation Models

Motivation

• Foundation models can classify EEG accurately while leaving latent representations difficult to map onto established neurophysiological principles.Existing attribution methods provide local sensitivity maps but do not distill latent decision paths into clinically interpretable evidence.Source: §1 · p. 1
• Models trained on heterogeneous data may acquire statistical shortcuts, so accurate predictions alone do not establish reliance on physiological evidence.The paper cites electrode impedance imbalance as an artifact a model could use for accurate classification without clinical credibility.Source: §1 · pp. 1–2
• Attribution maps remain separated from semantic reasoning, limiting their usefulness for clinicians and neuroscientists.The paper gives alpha-wave suppression in the occipital lobe as an example of evidence needing a higher-level clinical interpretation.Source: §1 · p. 2

Contributions

• The framework supports BIOT, LaBraM, EEGPT, and EEGMamba alongside input-gradient, perturbation, and feature-activation attribution methods.It is intended to compare feature-capture preferences, including spatial localization and time-frequency sensitivity.Source: §1 · p. 2
• Population-level progressive occlusion and AOPC quantify confidence decline after removing highly ranked features to assess attribution fidelity.The authors use this analysis to examine neurophysiologically aligned features and possible dependence on erroneous patterns.Source: §1 · p. 2
• Structured multidimensional attribution evidence is supplied to an LLM to produce natural-language reports linking physical representations with clinical semantics.The authors describe these reports as intuitive and transparent decision-making reports for neuroscientists.Source: §1 · p. 2

2.1 Evolution of EEG Foundation Models

• Task-specific CNNs extract spatiotemporal features but were constrained by dataset scale, while EEG foundation models use large-scale pretraining and varied architectures.Examples include BIOT's channel-level spectral tokenization, LaBraM's neural tokenizer, and EEGMamba's selective state-space modeling.Source: §2.1 · p. 2

2.2 Post-hoc Attribution and XAI Applications in EEG

• EEG attribution research spans relevance propagation, feature visualization, perturbation analysis, SHAP, LIME, and concept activation vectors.Most prior studies focus on single-model, task-specific designs rather than standardized auditing across heterogeneous foundation models.Source: §2.2 · pp. 2–3

2.3 Neuro-Symbolic Integration and Semantic Reasoning via LLMs

• Prior EEG work has explored LLM interpretation of feature descriptors, while this paper proposes transforming structured spatial and time-frequency evidence into natural-language expert reports.The reported gap is the difficulty of deriving logically consistent diagnostic conclusions from attribution visualizations.Source: §2.3 · p. 3

EEG Signal Modeling and Characteristics

• An EEG foundation model maps a multichannel recording through a pretrained backbone and task-specific head to produce a classification probability.The input has C electrodes and T time samples, and the backbone maps it into a high-dimensional representation space.Source: §3.1 · p. 3
• Foundation models segment signals into time patches and use Transformer attention or selective state-space models, leaving channel and segment contributions implicit in latent representations.The passage identifies this lack of direct interpretability as a consequence of high-dimensional hidden representations.Source: §3.1 · p. 3

Post-hoc Attribution Paradigms

• Input-gradient methods estimate feature importance from prediction sensitivity, with IG integrating gradients from a baseline and Gradient-SHAP averaging gradients across sampled paths.The passage notes that IG satisfies the Completeness axiom and Gradient-SHAP is intended to provide smoother, more stable maps.Source: §3.1 · p. 3
• Perturbation methods estimate importance from model responses to altered inputs, including confidence drops after masking and feature contributions across subsets.The framework also includes KernelSHAP as an efficient approximation and LIME as a local surrogate trained on perturbed samples.Source: §3.1 · pp. 3–4

Attribution Analysis

• The framework projects attribution results onto spatial, temporal, and frequency-band dimensions and supports single-sample and population-level analyses.These analyses are presented as a way to examine attribution reliability.Source: Figure 1 · p. 4

Post-hoc Attribution Paradigms

• Grad-CAM uses feature-map activations and class-specific importance weights to produce localization information for models with convolutional or self-attention layers.The weight is derived from globally pooled gradients of the final bottleneck layer.Source: §3.1 · p. 4

Unified Model-Task Adaptation and Explanation Interface

• A shared model-task interface standardizes processing across architectures and exposes a common attribution API.The framework describes its design goal as distilling model parameters into a small set of auditable spatial-temporal-frequency signatures.Source: §3.2 · p. 5

Spatiotemporal-Frequency Attribution Analysis

• Gradient-, perturbation-, and activation-based methods produce spatial channel maps, temporal heatmaps, and frequency-band contributions.Together these form an evidence chain from brain regions to finer spatiotemporal-frequency features.Source: §3.2 · p. 5

Fidelity Evaluation

• Population-level fidelity evaluation uses measures such as AOPC to quantify confidence decline during progressive masking.The framework presents this evaluation as a basis for validating attribution outcomes.Source: §3.2 · p. 5

LLM-assisted Clinical Interpretation

• Structured attribution results and assessment metrics are provided to an LLM, which generates natural-language interpretations in neurophysiological contexts.The resulting evidence statements are intended for clinical expert review.Source: §3.2 · p. 5

3.3 Unified Model-Task Adaptation and Explanation Interface

• A unified adaptation mapping converts the original EEG input space into each model’s input space, addressing differences in channels, segmentation, and representations.This shared abstraction is intended to make explanations more comparable across models and tasks.Source: §3.3 · p. 5
• For each sample and target class, attribution methods produce a joint channel-by-time-block attribution matrix within the shared output space.The underlying EEG network is denoted by f(·).Source: §3.3 · p. 5
• Marginal aggregation derives channel importance s_c and time-block importance t_p, enabling comparisons across attribution methods on matching dimensions.s_c measures a channel’s overall contribution to the prediction, while t_p characterizes the significance of information around time block p.Source: §3.3 · p. 5

3.4.1 Spatial Attribution Analysis

• Channel importance is mapped to the standard 10-10 scalp layout and can be aggregated by anatomical region to compare regional contributions and hemispheric lateralization.The layout contains approximately 60 electrode locations across frontal, central, temporal, parietal, and occipital regions.Source: §3.4.1 · pp. 5–6, Figure 2 · p. 6

3.4.2 Temporal Attribution Analysis

• Patch-level scores map to channel-specific time intervals, while population aggregation summarizes temporal importance across channels and samples.For the illustrated healthy Mumtaz2016 sample, high-contribution patches on O1 and O2 are evenly distributed, with no single local segment dominating the decision; the text contrasts this with expected event-focused patterns and more uniform resting-state contributions.Source: §3.4.2 · p. 6, Figure 3 · p. 6

3.4.3 Frequency Band Attribution Analysis

• Frequency-band attribution removes each rhythm with a band-stop filter and estimates its contribution to prediction confidence, producing population-level band-by-channel maps.The bands are δ (0.5–4 Hz), θ (4–8 Hz), α (8–13 Hz), β (13–30 Hz), and γ (30–45 Hz).Source: §3.4.3 · pp. 6–7

3.5.1 Perturbation-Based Faithfulness

• Faithfulness is tested by progressively masking features in descending attribution order and checking whether their removal lowers model output more than random removal.Spatial masking uses channel-wise mean replacement, while temporal masking uses Gaussian noise matched in mean and variance.Source: §3.5 · p. 7, §3.5.1 · p. 7
• AOPC summarizes confidence changes across cumulative masking, and AOPCgain compares attribution-ranked masking with the random-order baseline.A steeper confidence decline indicates that the ranking more precisely identifies decision-critical features.Source: §3.5.1 · p. 7
• Independent single-feature masking measures each feature’s logit drop, which is compared with attribution scores using Spearman rank correlation.A positive correlation indicates monotonic consistency between the attribution ranking and model feature dependencies, supporting comparisons across methods for a model-task combination.Source: §3.5.1 · p. 7

3.5.2 Cross-Method Consistency

• Cross-method consistency uses pairwise Spearman correlations between population attribution vectors; high agreement indicates method-insensitive conclusions.When methods disagree, the framework uses faithfulness metrics to select the method with the best faithfulness for that model-task combination.Source: §3.5.2 · p. 7

3.6 LLM-assisted Clinical Interpretation

• The LLM assesses physiological plausibility by relating attribution patterns to brain-region functions and neural-rhythm significance, integrating dimensions and assigning confidence to uncertain inferences.Inputs may be structured attribution data or visualizations, with domain-specific prompt constraints guiding the reasoning.Source: §3.6 · p. 7
• The module aims to make model-decision rationales more understandable to clinicians without computational backgrounds, while providing supplementary evidence about physiological alignment.Its output does not influence fidelity assessments or alter attribution rankings.Source: §3.6 · pp. 7–8

4 EXPERIMENTAL SETUP

• Experiments fine-tuned five EEG backbones on TUAB, Mumtaz2016, and TUEV, evaluating six post-hoc methods with population-level fidelity and cross-method consistency measures.The models comprised four Transformer architectures and EEGMamba; analyses averaged true-positive samples with target-class probability of at least 0.7.Source: §4 · p. 8

4.1.1 Channel-Level Spatial Fidelity

• On Mumtaz, CBraMod IG highlighted occipital O1, O2, and Pz for HC and frontal F3/F8 plus central C4 for MDD; attribution-guided curves declined faster than random baselines.The HC association was reported at ρ = 0.981, p < .001; EEGMamba also showed spatial fidelity for both classes.Source: §4.1.1 · p. 8, Figure 4 · p. 9, Table 3 · p. 10
• On TUAB, CBraMod and EEGMamba showed high spatial fidelity; CBraMod methods agreed on abnormal-class temporal channels, while LaBraM, BIOT, and GradCAM generally performed worse.CBraMod IG and GradientSHAP each reached ρ = 0.979, p < .001 for the abnormal class; normal-class channels included central-parieto-occipital regions.Source: §4.1.1 · pp. 8, 10, Table 4 · p. 11

4.1.2 Frequency-Band-Level Attribution Analysis

• For HC, alpha and low-beta contributions concentrated in occipito-parietal regions, while diffuse gamma attribution may include non-neural components.The authors note that scalp gamma is susceptible to EMG and microsaccade-related interference under standard preprocessing.Source: §4.1.2 · p. 10, Figure 5 · p. 11
• For MDD, positive δ/θ contributions appeared frontotemporally and β hotspots fronto-centrally; frontal α attribution was near-symmetrical, with the overall maps aligning with reported physiological findings.The authors suggest the model may implicitly integrate information related to frontal alpha asymmetry rather than displaying it as a distinct attribution pattern.Source: §4.1.2 · p. 10

4.2 Event-based Tasks: Temporal Faithfulness Analysis

• On TUEV, EEGPT showed significant event anchoring at finer granularity, BIOT had positive correlations across methods, and CBraMod had consistent but nonsignificant ρ = 0.800 results.Absolute ρ values cannot be directly compared across patch granularities; EEGMamba methods disagreed, while LaBraM temporal faithfulness was low and nonsignificant.Source: §4.2 · pp. 10, 12, Table 5 · p. 12

4.3 Discussion

• No attribution method performed optimally across every model-task pairing, supporting the framework’s use of multiple methods and a recommendation mechanism.For example, Occlusion reached ρ = 0.991 on EEGMamba (TUAB) but −0.412 on BIOT.Source: §4.3 · p. 12
• Across tasks, spatial attribution fidelity generally exceeds temporal fidelity, while stable patch-level temporal fidelity still requires finer evaluation.The discussion contrasts larger TUAB than TUEV spatial-fidelity radar radii with generally low temporal fidelity across most model-method combinations.Source: §4.3 · p. 12

• The framework supports multidimensional attribution analysis independent of specific model architectures by abstracting architectures and integrating attribution methods.Source: §5 · p. 12
• No attribution method is optimal across models and dimensions; method selection is coupled with model architecture and task dimensions.Source: §5 · p. 12
• Attribution consistency is generally higher spatially than temporally, and patch-level temporal fidelity requires more granular evaluation.Source: Figure 6 · p. 13, §5 · p. 13
• Cross-model comparisons reveal both common neurobiomarkers and model-specific biases.Source: §5 · p. 13