Source-linked AI summary
MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation
Xiangyu Liu, Zeting Yan, Zhitong Yin, Boyang Li, Xi Zhang
TL;DR
EEG-to-fMRI translation must represent delayed, distributed, and frequency-specific EEG histories in a form that preserves their target-relative relationships. MEL addresses this gap with coordinate-preserving lag-channel-frequency tokens anchored to each fMRI target and evaluated with capacity-controlled readouts. Across VU and external Oddball evaluations, the paper reports improved prediction over strong baselines, with controls attributing gains to structured representation rather than leakage or decoder capacity.
Problem
EEG-to-fMRI prediction lacks representations that explicitly preserve the delayed, distributed, and spectral relationships linking EEG histories to fMRI responses.
Method
MEL anchors each target fMRI response to preceding EEG, organizes calibrated lag, channel, and frequency measurements into deterministic tokens, and decodes them with controlled readouts.
Results
MEL outperforms strong neural and traditional baselines across intra-scan and inter-subject evaluation, with Oddball transfer, ablations, and controls supporting structured temporal, spectral, and spatial representation.
Takeaways & Limitations
Representation design can make EEG evidence directly readable for EEG-to-fMRI decoding without relying exclusively on increased decoder capacity.
Takeaways & Limitations
External transfer depends on a usable EEG-fMRI alignment substrate; tokenization does not recover structure from severely degraded correspondence.
Abstract
from arXiv · showhide
Translating electroencephalography (EEG) into functional magnetic resonance imaging (fMRI) is important for medical neuroimaging, clinical brain-state monitoring, and multimodal neural decoding, because it aims to infer spatially organized hemodynamic activity from fast and accessible electrophysiological recordings. Existing EEG-to-fMRI studies mainly pursue stronger decoders, but the problem is also constrained by a representation-interface mismatch: fMRI responses are delayed, temporally integrated, and spatially distributed, whereas generic EEG encodings often entangle temporal lag, channel identity, and frequency-band structure. We propose Multi-band EEG Latent-state Tokenization (MEL), a coordinate-preserving EEG representation framework that anchors each target fMRI response to its preceding EEG history and organizes it into lag-channel-frequency neural-state tokens. By explicitly capturing hemodynamic latency and spectral-spatial dynamics, MEL aligns fMRI-pertinent EEG representations with capacity-controlled readouts without depending entirely on model scaling. Experiments on VU EEG-fMRI benchmarks and external Oddball data show that MEL improves prediction over strong NeuroBOLT baselines. Ablations and controls further indicate that the gains come from structured EEG representation rather than leakage, shortcut statistics, or decoder capacity.
Introduction
EEG-to-fMRI prediction is constrained by delayed, distributed neurovascular coupling and by representations that leave lag, channel, and frequency relationships implicit. MEL addresses this interface problem by organizing target-relative EEG histories into explicit coordinates for capacity-controlled decoding.
- Motivation: EEG-to-fMRI prediction seeks spatially distributed BOLD responses from millisecond-resolution electrophysiological recordings, linking complementary temporal and spatial information.The mapping could support multimodal neural decoding, brain-state monitoring, and study of large-scale neural dynamics.
- Representation gap: Delayed coupling, distributed electrode evidence, and frequency-specific rhythms make each fMRI target the outcome of a structured EEG history over lag, channel, and frequency coordinates.Without target-relative semantics, decoders must infer these relationships from limited paired EEG-fMRI data.
- Representation gap: Existing approaches model temporal, spatial, and spectral dependencies, but typically learn the target-relative meanings of lag, channel, and frequency implicitly.This leaves physiological relationships less directly accessible to the decoder.
- MEL: MEL is a deterministic coordinate-preserving framework that anchors preceding EEG histories, partitions temporal lags, computes channel-wise canonical bandpower, calibrates on training data, and constructs lag-channel-frequency tokens.The resulting representation can be decoded with Ridge regression or a compact multilayer perceptron.
- Evaluation: The study evaluates MEL across intra-scan, strict leave-one-subject-out, and external-dataset settings using capacity-controlled readouts, correspondence-breaking controls, and temporal and spectral ablations.These experiments are designed to characterize when and why the representation is effective.
Method
MEL presents EEG history as a coordinate-preserving interface for fMRI decoding by anchoring targets to preceding activity and organizing calibrated lag-channel-frequency tokens. The deterministic representation is passed to capacity-controlled readouts without relying on large decoder capacity.
- Task anchoring: MEL anchors each fMRI target to a preceding EEG history and organizes it into lag-channel-frequency tokens for ROI-response decoding.The framework uses train-only calibrated bandpower extraction before decoding.
- Lag-channel grid: The representation divides the pre-target EEG history into lag bins and preserves channel identity relative to the fMRI target.This coordinate system makes temporal delay and electrode location explicit rather than leaving them implicit in generic sequence patches.
- Bandpower token construction: MEL computes bandpower for canonical frequency bands at each lag-channel coordinate, producing a token cube whose elements encode channel, lag, and frequency.The token cube is then vectorized in a fixed coordinate order.
- Train-only calibration: Normalization statistics are estimated only from training data, and evaluation samples use the same train-set transformation.This design is intended to avoid normalization leakage across validation, test, and external samples.
- Capacity-controlled decoding: Ridge regression and compact MLP readouts test whether the calibrated token representation exposes predictive structure before decoding.MEL is presented as model-agnostic and capacity-controlled rather than as a high-capacity predictor.
- Coordinate preservation: With fixed coordinates and vectorization order, MEL re-indexes the token cube without lossy pooling, preserving coordinate identity through the readout stage.The method changes the coordinate system in which EEG history is presented rather than introducing a new optimization objective.
Experiments
Experiments evaluate MEL across intra-scan, inter-subject, external-transfer, ablation, and representation-validity settings. MEL performs best when lag, channel, and frequency coordinates are preserved, with gains concentrated in non-sensory targets and bounded by alignment quality.
- Intra-scan evaluation: MEL-MLP Ensemble achieves an intra-scan Avg. R of 0.543, improving NeuroBOLT’s 0.531, with the largest gains on cognitive, subcortical, and global targets.NeuroBOLT remains strongest on cuneus and Heschl’s gyrus, so the improvement is structured rather than uniform across ROIs.
- Capacity-controlled comparisons: Under matched readouts, MEL-coordinate tokens retain the strongest performance, indicating that the useful signal comes from the coordinate interface rather than bandpower alone.The audit compares classical lagged bandpower, uniform STFT lag-band, flattened PSD, and MEL tokens using the same MLP protocol and train-only calibration.
- Inter-subject evaluation: MEL-MLP Ensemble reaches an inter-subject Avg. R of 0.499 versus NeuroBOLT’s 0.473 under strict leave-one-subject-out evaluation.MEL is strongest on high-level cognitive, subcortical, and global targets, while pretrained encoders remain strongest on sensory ROIs.
- External transfer: External transfer improves on Oddball as longer ordered EEG histories are retained, whereas NODDI remains near zero under severely degraded alignment.These results support transfer when a usable EEG-fMRI alignment substrate is present, but not when external correspondence is severely degraded.
- Representation-validity controls: Label and target shuffling collapse Avg. R to 0.0014 and −0.0021, while channel permutation, band reversal, random lag assignment, and lag reversal reduce it to 0.3945, 0.2355, 0.1977, and 0.1022.Train-only and all-split normalization produce the same score, 0.4521, excluding normalization leakage as the source of improvement.
- Component attribution: With the same Ridge decoder, preserving lag coordinates with three bands reaches Avg. R 0.407, while the full five-band representation reaches 0.452.Time-mean and HRF-weighted aggregation reach 0.292 and 0.325; 2-second bins and the full pre-target window provide the strongest temporal design.
Conclusion
MEL organizes pre-target EEG histories into explicit lag-channel-frequency coordinates for EEG-to-fMRI translation. Across intra-scan, inter-subject, and Oddball evaluations, its gains depend on ordered temporal, spectral, and spatial structure rather than leakage or decoder capacity.
- MEL organizes pre-target EEG histories into explicit lag-channel-frequency coordinates, reducing the mismatch between fast electrophysiological activity and delayed hemodynamic responses.
- Across intra-scan and inter-subject evaluation, MEL outperforms strong neural and traditional baselines.
- Ridge results show that MEL is directly readable, while lightweight nonlinear readouts capture additional cross-coordinate interactions.
- Oddball transfer, ablations, and validity controls indicate that MEL gains depend on ordered temporal, spectral, and spatial structure rather than leakage or decoder capacity.
- MEL provides a simple, interpretable, and model-compatible interface for exposing fMRI-relevant EEG structure despite cross-subject variability and imperfect multimodal alignment.