NOMAD-EEG: Native Omni-Montage Adaptive Diffusion

1 Introduction

• EEG events can be brief, while existing models may require fixed channel vocabularies or order-dependent channel mixing and may cap input length through temporal patches.The introduction notes spike durations of 20–70 ms and sharp waves of 70–200 ms.Source: §1 · p. 3

1.1 Related work

• Earlier generative EEG models use fixed input formats, while montage-agnostic models use position information but rely on masked or tokenised reconstruction and temporal patches or windows.The related-work discussion describes these as separate design choices that NOMAD-EEG combines.Source: §1 · p. 3

1 Introduction

• NOMAD-EEG combines diffusion pre-training, 3D-position-based electrodes, and patch-free temporal modelling at 20 ms resolution without a maximum input length.Its S4D layer spans the full input, and its parameter count does not depend on electrode or sample counts.Source: §1 · pp. 3–4
• NOMAD-EEG provides both a frozen feature extractor based on windowed gate-activation statistics and a token-based fine-tuning head whose output size is independent of montage and recording length.The related-work discussion also contrasts fixed-vocabulary, channel-index, and patch-based approaches across existing models.Source: §1 · p. 4, §1.1 · pp. 4–5
• The paper re-runs seven released foundation models under one protocol, evaluates transfer to two unseen populations, and tests pre-training, electrode-order, missing-electrode, and scaling effects.The scaling study uses a model 5.6 times larger and 47 times more data.Source: §1 · p. 4
• Across eight benchmarks, NOMAD-EEG was never ranked below fourth among eight foundation models, ranked first on Bonn and EEGMMIDB, and was best of five on unseen-montage autism P300 detection.The P300 evaluation used an eight-electrode montage not seen in pre-training.Source: §1 · p. 4

2.1 Problem setting and input representation

• Each recording is represented as E channels by T samples plus a 3D position for each channel; the model receives the signal and positions, not channel indices.Bipolar derivations are assigned the midpoint of their two electrodes, and channels without standard positions are dropped.Source: §2.1 · pp. 6–7

2.2 Architecture

• The model has a shared temporal stem, 16 residual and skip-connected blocks, and a pre-training-only output head; its feature width is 256.The model contains 11,998,977 parameters.Source: §2.2 · p. 7, Figure 2 · p. 8

2.2.1 Stem

• A shared one-dimensional convolution downsamples each electrode signal, producing one feature column every 20 ms at 200 Hz.The resulting tensor has one row per electrode and one column per time step.Source: §2.2.1 · pp. 7, 9, Figure 3 · p. 9

2.2.2 Conditioning

• During pre-training, the noise level and corpus identifier condition the network; downstream inputs are clean and use the clean end of the noise schedule.The conditioning vector is applied through feature-wise scaling and shifting after layer normalisation.Source: §2.2.2 · p. 9, §2.2.3 · p. 9

2.2.3 The NOMAD block

• Each block filters each electrode’s sequence with a bidirectional S4D layer, then uses attention across electrodes at each time step with position features re-injected into attention.The S4D kernel is generated for the input length, spans the full segment, and has no learned position table or maximum length; gated outputs feed residual and skip paths and supply downstream features.Source: §2.2.3 · pp. 9–10

2.2.4 Output head

• The pre-training output head maps summed skip outputs back to the input sample rate, while downstream representations read gates from selected blocks.The frozen probe summarizes gates in time windows; the fine-tuning head pools tokens by attention.Source: Figure 2 · p. 8, §2.2.4 · p. 10

• At each uniformly sampled diffusion step, NOMAD-EEG corrupts a clean EEG window and predicts its noise using t=k/1000.Source: §2.3 · p. 11
• The diffusion process uses a cosine noise schedule with s=0.008 and 1,000 steps.Source: §2.3 · p. 11
• Noise prediction requires modeling EEG rhythms, transients, artefacts, and how signals co-vary across the scalp.Source: §2.3 · p. 11

Flow-matching variant

• The rectified-flow variant uses xt=(1−t)ε+tx0 and target x0−ε, with architecture, data, optimiser, epochs, and task heads unchanged.Source: §2.3 · p. 11

• Permuting electrodes permutes every intermediate tensor and the output correspondingly, while outputs pooled over electrodes remain invariant.Source: §2.4 · p. 11
• Shared per-electrode operations and attention without positional indices produce permutation equivariance, with each electrode's position embedding moving with it.Source: §2.4 · p. 11
• NOMAD-EEG's parameters do not depend on electrode count or input length, allowing absent-from-training electrodes and any input length.Source: §2.4 · p. 11
• Per-block cost is roughly linear in time and quadratic in electrode count: O(E d T′ log T′ + T′E²d).Source: §2.4 · p. 11
• The frozen head averages gate activations across electrodes into 10,240 features, while fine-tuning pools electrode-by-window tokens with learned-query attention.Source: Figure 4 · p. 12

Long context

• Pre-training uses 30 s windows with whole-window context; longer inputs need no architectural change, but inputs assume 200 Hz and known channel positions.Source: §2.4 · p. 12

• For downstream tasks, clean EEG is evaluated at the clean trajectory endpoint using gates from blocks 4, 8, 12, and 16; the generative output head is unused.Source: §2.5 · p. 12

Frozen feature extraction

• Frozen probing averages each gate over electrodes, then takes mean and standard deviation in five time windows to form a 10,240-dimensional vector.Source: §2.5 · pp. 12–13
• The five fixed windows preserve coarse temporal order, and within-window standard deviations can capture brief events diluted by global averaging.Source: §2.5 · p. 13

Fine-tuning head

• Fine-tuning forms 25 time-window tokens per electrode and pools them with a learned-query attention head whose parameters do not depend on electrode count or input length.Source: §2.5 · p. 13
• Fine-tuning trains the head alone for two epochs before the full network, then selects the checkpoint with best balanced accuracy on subject-disjoint validation.Source: §2.5 · p. 13

Data

• Pre-training uses 17,678 non-overlapping 30 s windows from TUEV recordings, totaling 147 h; ROC is dropped because it lacks a standard coordinate and is absent from 40% of samples.Source: §2.6 · p. 13
• TUEV downstream evaluation is in-domain transfer, whereas all other downstream datasets are external to pre-training.Source: §2.6 · p. 13

Optimisation

• Both models train for about 74 epochs and 40,000 updates with effective batches of 32, using AdamW, warm-up and cosine decay, gradient clipping, and EMA weights.Source: §2.6 · p. 13
• Each model trains on a single NVIDIA A100 GPU; the S4D FFT remains in float32 because its kernel tail underflows in bfloat16.Source: §2.6 · p. 13

2.7 Downstream datasets

• Nine datasets are compared with baselines, while SEED-V is evaluated for NOMAD-EEG alone; the tasks and recording formats vary widely.TUEV overlaps with NOMAD-EEG’s unlabelled pre-training data, whereas Siena, Bonn, Mumtaz, EEGMMIDB, Sleep-EDF, and SEED-V are external.Source: §2.7 · p. 14
• TUEP evaluates epilepsy diagnosis using routine EEG from 100 patients with and 100 without epilepsy, with 60 random 5 s windows per patient.Equal class sizes make accuracy equivalent to balanced accuracy and weighted F1 equivalent to macro F1 on this task.Source: §2.7 · p. 14
• Most splits are subject-disjoint, but TUAB, TUSZ, Siena, and Bonn reuse fixed splits, while TUEP uses five-fold cross-validation over 200 patients.SEED-V follows a film-clip split protocol with all 16 subjects in each split.Source: Table 2 · p. 14

2.8 Baselines and overlap with their pre-training data

• Seven EEG foundation models are run from released weights on common inputs in their native formats; B[FM]2 is excluded because its code and weights are unavailable.Some fixed channel vocabularies are adapted by mapping unrepresentable channels to the nearest entry by 3D position.Source: §2.8 · p. 14

2.9 Evaluation protocol

• Frozen encoders are evaluated with class-balanced, L2-regularised logistic-regression probes, with task-specific procedures for TUEV and patient-level cross-validation for TUEP.Fine-tuned NOMAD-EEG is compared with frozen-probe baselines, and TUAB/TUSZ fine-tuning selects epochs using validation windows from training patients.Source: §2.9 · p. 15

2.10 Statistical analysis

• Metrics receive 95% bootstrap confidence intervals, and paired bootstrap comparisons are used for specified same-sample model contrasts; significance requires the interval to exclude zero.Where only unpaired intervals are available, the authors report overlap without treating non-overlap as a paired test.Source: §2.10 · p. 15

2.11 Transfer to unseen populations

• Transfer is tested on BCIAUT-P300 and Parkinson’s disease using frozen encoders and subject-held-out linear probes, with no training or fine-tuning on either corpus.BCIAUT-P300 tests P300 detection in 15 participants with autism using eight electrodes; the Parkinson’s task separates 100 participants with Parkinson’s disease from 49 controls.Source: §2.11 · pp. 16–17

2.12 Ablation experiments

• A random-weight version of NOMAD-EEG is evaluated with the same feature extraction, probe, and fine-tuning protocols on five datasets.This comparison is designed to separate pre-training contributions from those of the architecture and task head.Source: §2.12 · p. 17
• NOMAD-EEG-F differs from NOMAD-EEG only in using rectified flow rather than diffusion, and the models are compared on every downstream dataset with paired bootstrap testing.Architecture, data, optimiser, and training epochs are held identical.Source: §2.12 · p. 17
• Electrode-order tests compare activations and predictions across five random channel orders, while missing-electrode tests apply a full-montage-trained probe to subsets of 14, 10, or 5 electrodes.The position-shuffle control leaves signals in place, and no missing electrode is imputed.Source: §2.12 · p. 17

2.13 Scaling experiment: NOMAD-EEG-Large

• NOMAD-EEG-Large scales model size and pre-training data together: it has 67.6 M parameters and uses 6,984 h from seven corpora, 47 times Base’s data.Large is evaluated with frozen probes after 1.0 and 1.9 epochs; its 30 s-window-based epoch count is an upper bound because training samples are crops.Source: §2.13 · pp. 17–19, Table 3 · p. 18

2.14 Model interpretation

• Integrated Gradients attributes the frozen model’s TUSZ seizure score to the input, aggregating positive seizure evidence by electrode and 0.25 s time bins.The analysis uses one confidently classified seizure segment and one confidently classified non-seizure segment selected by a pre-specified seeded-random rule.Source: §2.14 · p. 19

2.15 Use of AI tools

• The authors used Claude for language and readability improvements, then reviewed and edited the content and took responsibility for it.Source: §2.15 · p. 19

3.1 Overview

• Across eight benchmarks, NOMAD-EEG was never ranked below fourth and was first on Bonn and EEGMMIDB, ranking ahead of five models on average.REVE and EEGDM led most often.Source: §3.1 · p. 19

3.2 TUH benchmarks (TUAB, TUSZ, TUEV, TUEP)

• On TUEV, fine-tuning reached κ 0.635 and weighted F1 0.814, while frozen-probe macro F1 was second on TUSZ and patient-level accuracy was highest on TUEP among models not pre-trained on it.TUSZ macro F1 was 0.620, behind EEGDM’s 0.642; TUEP accuracy was 0.640. On TUAB, NOMAD-EEG ranked fourth by AUROC at 0.838.Source: §3.2 · pp. 19–20, Table 4 · p. 21, Table 5 · p. 21

3.3 Seizure detection on external corpora (Siena, Bonn)

• On Siena, NOMAD-EEG had the highest macro F1 at 0.622; on Bonn, its AUROC was about 0.998–0.999, level with REVE.Using only Bonn’s signal-carrying electrode, it reached AUROC 0.995.Source: §3.3 · p. 21, Table 8 · p. 23, Table 9 · p. 23

3.4 New montages and tasks (Mumtaz, EEGMMIDB, Sleep-EDF, SEED-V)

• On EEGMMIDB, the best frozen model reached balanced accuracy 0.442 despite 45 unseen electrode positions; it ranked third on Sleep-EDF and Mumtaz by the reported primary metrics.Fine-tuning raised EEGMMIDB balanced accuracy to 0.596; on SEED-V, balanced accuracy rose from 0.315 frozen to 0.384 fine-tuned, against chance 0.20.Source: §3.4 · pp. 22–23, Table 11 · p. 24, Table 12 · p. 24

3.5 Transfer to unseen populations

• On unseen BCIAUT-P300, NOMAD-EEG was best of five models on every metric, reaching AUROC 0.718; on Parkinson’s diagnosis, EEGDM’s AUROC 0.795 significantly exceeded NOMAD-EEG’s 0.684.The P300 test used a montage with eight electrodes and subjects unseen during evaluation training; Parkinson’s results came from frozen features.Source: §3.5 · pp. 23, 25, Table 14 · p. 25, Table 15 · p. 25

3.6 Ablations

• Pre-training improved TUEV balanced accuracy, TUAB AUROC, and Siena macro F1; diffusion beat flow matching on four datasets, while AUROC stayed high with five electrodes but fixed-threshold accuracy fell.Reported gains over random weights were 0.107 on TUEV frozen-probe balanced accuracy, 0.040 on TUAB AUROC, and 0.071 on Siena macro F1. With five electrodes, mean AUROC was 0.891; balanced accuracy was 0.696. Diffusion’s largest reported advantages were 0.086 macro F1 on Siena, 0.055 balanced accuracy on TUEV, and 0.041 on Sleep-EDF.Source: §3.6 · pp. 25–27, Figure 6 · p. 27

3.7 Scaling model size and pre-training data

• After 1.9 epochs, the 67.6 M-parameter model trained on 6,984 hours was not significantly better than the 12.0 M-parameter model trained on 147 hours on any dataset.Its confidence interval was entirely below Base’s on EEGMMIDB, Sleep-EDF, and SEED-V.Source: §3.7 · pp. 27–28, Table 18 · p. 28

3.8 Model interpretation

• In the seizure segment, evidence concentrated on sharp, high-amplitude discharges and peaked where they occurred; the normal segment showed almost no evidence.Source: §3.8 · p. 28

4.1 Principal findings

• Across eight benchmarks, NOMAD-EEG was never ranked below fourth among eight foundation models and was first on EEGMMIDB and Bonn; on Siena it had the highest accuracy, weighted F1, and macro F1.It also performed best on BCIAUT-P300 with an unseen eight-electrode montage, detecting attended objects in 75.5% of blocks versus 61.2% for the next-best model.Source: §4.1 · pp. 28–30

4.7 What the model responds to

• In two typical correct TUSZ decisions, seizure attribution concentrated on sharp, high-amplitude discharges, but these examples do not establish quantitative seizure localisation.A localisation claim would require comparison with annotated seizure onsets and channels.Source: Figure 7 · p. 29, §4.7 · p. 32

4.2 Where NOMAD-EEG trails, and why

• NOMAD-EEG trailed on several tasks, while competitors’ pretraining overlap with evaluation corpora and greater scale are plausible contributors, not established causes.On Siena, its advantage was fewer false alarms rather than better ranking of seizure windows.Source: §4.2 · p. 30

4.3 Transfer to unseen populations

• On Parkinson’s disease diagnosis, NOMAD-EEG was not the best model, and its selected probe strength varied by five orders of magnitude across folds.The authors suggest the small validation set may make the probe unstable; a spectral-versus-transient explanation for the gap remains untested.Source: §4.3 · p. 30

4.4 Pre-training, objective and fine-tuning

• Pretraining significantly improved TUEV, TUAB, and Siena, but random-weight fine-tuning matched pretraining on Mumtaz and both models saturated on Bonn; diffusion beat flow matching on four datasets and lost on one.Fine-tuning helped on TUEV, EEGMMIDB, and SEED-V, but not on Mumtaz, Sleep-EDF, TUAB, or TUSZ, where validation-based checkpoint selection was problematic.Source: §4.4 · p. 31

4.5 Electrode order and missing electrodes

• Removing electrodes largely preserved test-window ranking but shifted scores relative to the full-montage probe threshold, so balanced accuracy fell faster than AUROC.The authors identify threshold recalibration or training with randomly dropped electrodes as possible remedies.Source: §4.5 · p. 31

4.6 Scaling model size and pre-training data

• Scaling model size and data together did not significantly improve any benchmark; the authors caution that fixed compute may leave the larger model under-trained relative to its data.Large had 1.63 M samples versus Base’s 1.28 M and at most about two epochs over its pool versus about 74 for Base over TUEV; which factor explains the result was not tested.Source: §4.6 · pp. 31–32

4.8 Limitations

• The evidence is bounded by single-corpus pretraining, untested inputs longer than 30 s, 200 Hz and known-position assumptions, reduced-montage threshold mismatch, and frozen-only baseline comparisons.Channels without standard positions are dropped, and bipolar derivations are approximated by a sensor at their midpoint.Source: §4.8 · pp. 32–33

Declarations

• The work was supported by the LAU President’s Intramural Research Fund, grant I0084, and the authors declare no conflict of interest.Source: Declarations · p. 33
• The study used publicly available, de-identified EEG datasets collected under ethics approvals reported by their original authors; consent was obtained by the original data collectors.Source: Declarations · p. 33
• The datasets are available from their original sources, and the code and model weights will be shared once the manuscript is published.Source: Declarations · pp. 33–34