FRIST: FMRI Representation Informed Shared-space Training Improves EEG-only Individual-Finger BCI Decoding

• FRIST improves EEG-only individual-finger decoding using fMRI-informed spectral projections and shared-space training.Source: Highlights · p. 2
• Decoding gains persist when the target participant’s own fMRI data are unavailable.Source: Highlights · p. 2
• The framework enhances four EEG decoders in finger execution and imagery.Source: Highlights · p. 2

• Individual-finger decoding remains challenging because finger-related neural patterns overlap and vary across participants.Scalp EEG further blurs spatially adjacent finger representations through volume conduction.Source: §1 · p. 3
• fMRI offers high spatial resolution for finger representations, but its low temporal resolution and limited portability hinder routine real-time BCI control.Source: §1 · p. 3
• Prior work had not established whether fMRI could select EEG feature directions and provide finger-specific training anchors for EEG-only decoding.The question is especially relevant when trial-level correspondence is unavailable across separate recording sessions.Source: §1 · pp. 4–5
• Because the modalities lack shared trials, FRIST uses supervised contrastive learning based on shared finger labels rather than paired observations.Source: §1 · p. 5
• FRIST learns fMRI-informed spectral projections and aligns EEG features with an fMRI-derived finger-class space, then decodes using EEG alone.Its two stages use simultaneous EEG–fMRI data for training-time guidance.Source: §1 · p. 5

2. METHODS

• The fMRI guidance dataset used block-design ME and MI finger tasks during simultaneous EEG-fMRI scanning; the EEG-only dataset contained multiple sessions for session-held-out evaluation.The protocol separates fMRI guidance from EEG-only decoding.Source: §2 · p. 5, Figure 1 · p. 6
• FRIST combines fMRI-informed spectral projection features with a 40-dimensional fMRI-guided representation and residual correction of Stage 1 predictions.Validation-based calibration is applied before held-out decoding, which requires only EEG.Source: §2 · p. 5

2.1. Dataset Description and Task Design

• Each participant completed nine EEG-only sessions and two simultaneous EEG-fMRI sessions, with the latter used to obtain fMRI-derived spatial representations.Source: §2.1 · p. 6
• EEG-only recordings included three ME and six MI sessions, with two-class and three-class tasks and later online sessions reserved for chronological testing.The two-class task contrasted thumb with pinky; the three-class task added index.Source: §2.1 · p. 7
• Simultaneous EEG-fMRI data were collected in separate block-design ME and MI sessions covering all five fingers.The simultaneous recordings supplied the fMRI guidance data rather than EEG decoder training data.Source: §2.1 · p. 7

2.2. EEG Preprocessing and Input Construction

• EEG-only signals were common-average referenced, downsampled to 100 Hz, filtered from 4–40 Hz, and segmented into overlapping 1-s windows with 0.1-s strides.Source: §2.2 · p. 7
• A spherical-spline mapping aligned EEG-only recordings to the simultaneous EEG-fMRI montage before band-power extraction.Four inferior temporal channels were excluded, leaving 59 channels for the simultaneous recordings.Source: §2.2 · p. 8

2.3 fMRI Preprocessing and Activation Analysis

• fMRI volumes were realigned, distortion-corrected, coregistered to anatomical images, normalized to MNI space, and smoothed with a 4-mm kernel.Source: §2.3 · p. 8
• Finger-versus-rest contrasts were analyzed at first and second levels, with age, sex, and handedness included as group-analysis covariates.ME-versus-MI comparisons used a repeated-measures model with subject, task, and finger factors.Source: §2.3 · p. 8
• Finger-related spatial organization was summarized using weighted activation centroids and pairwise correlations between finger-specific activation maps.Source: §2.3 · p. 8

• Stage 1 uses temporally aligned, volume-level EEG band-power and BOLD observations, whereas Stage 2 uses block-level fMRI activation patterns to define finger-class geometry.Source: §2.4 · p. 9
• Stage 2 standardizes block-level patterns, selects 4,000 discriminative voxels, and applies whitened PCA to obtain a fixed 40-dimensional fMRI representation.Voxel features come from bilateral S1, M1, and V1; separate references are made for ME and MI and for two- and three-class settings.Source: §2.4 · p. 9
• The full-cohort reference uses simultaneous EEG–fMRI data from all 12 participants, while the participant-excluded reference is re-estimated from the other 11 participants.Source: §2.4 · p. 9

2.5.1. Stage 1: fMRI informed spectral representation

• The Stage 1 projections map band-specific EEG power to BOLD-related spatial patterns, and applying the frozen projections produces 4,500 FISP features per EEG-only trial.Ridge regression uses a fixed penalty of α=1000; the frozen 30×59 projections are applied to five bands and 30 time bins.Source: §2.5.1 · p. 9
• FISP features are reduced using training-only PCA, classified with shrinkage LDA, and selected by validation accuracy; Stage 1 fuses calibrated Base and FISP class scores.Static5 is an fMRI-free spectral control using whole-trial band-power features and the same two-branch fusion setup.Source: §2.5.1 · pp. 9–10

2.5.2. Stage 2: Residual shared-space training

• Stage 2 jointly learns a residual correction to Stage 1 scores and a 40-dimensional EEG representation aligned with fMRI-derived finger-class geometry.Within each fold, the EEG input concatenates a standardized 96-dimensional Base feature with a training-reduced 128-component FISP vector; the corresponding fixed fMRI input is 40-dimensional.Source: §2.5.2 · p. 10, §2.5.2.1 · p. 10

2.5.2.2. Residual shared-space network

• The residual network maps the 224-dimensional EEG input to a 40-dimensional embedding and K residual scores; during training, the tanh-bounded, class-centered residual is weighted by Stage 1 prediction entropy.The residual head is zero-initialized, so the initial corrected scores recover Stage 1.Source: §2.5.2.2 · pp. 10–11

2.5.2.3. Training Objective

• Stage 2 combines five losses: corrected-score cross-entropy, residual-direction fitting, EEG–fMRI class-mean and fMRI-LDA alignment, and cross-modal supervised contrastive alignment.The five coefficients are 1, 0.5, 0.25, 0.25, and 0.1 and are held constant across tasks, reference conditions, and backbones.Source: §2.5.2.3 · p. 11

2.5.2.4. Validation-only Residual Calibration

• A validation-fitted affine calibrator combines Stage 1 scores with Stage 2 residuals, and all fitted quantities are fixed before test inference.The parameter-averaged Stage 2 network supplies residual scores; the held-out trial passes through the frozen EEG paths and calibrator to produce final class scores.Source: §2.5.2.3 · p. 11, §2.5.2.4 · p. 12

• Each eligible online session serves once as the held-out test set, with only earlier sessions available for development and 20% of the historical trials reserved for validation.Source: §2.6 · p. 12
• The participant-excluded analysis rebuilds both fMRI reference banks from the other 11 participants while keeping the target participant’s EEG partitions unchanged.The all-participant and participant-excluded conditions use the same fold-local, validation-only fusion protocol.Source: §2.6 · p. 12
• Backbone evaluation replaces EEGNet with EEG Conformer, a residual dilated temporal convolutional network, or a 1D-CNN-Transformer while leaving other FRIST settings unchanged.Source: §2.6 · pp. 12–13

• The correct-centroid margin tests whether a held-out EEG embedding is closer to its true-finger fMRI centroid than to competing centroids.Source: §2.7 · p. 13
• The margin compares a normalized held-out EEG embedding with normalized class prototypes formed from layer-normalized fMRI-bank rows.The fMRI rows are averaged within class before each resulting class mean is L2-normalized.Source: §2.7 · p. 13
• A positive margin indicates greater similarity to the matching fMRI centroid than to the mean of competing centroids; margins are averaged equally across classes and then across sessions within participant.Source: §2.7 · p. 13

2.8. Statistical Analysis

• Group-level EEG decoding and representation analyses treated the 12 participants as statistical units and reported mean ± sample SD.Accuracy changes were expressed as absolute percentage-point differences.Source: §2.8 · p. 13
• Paired differences used two-sided t tests with Cohen’s d4, while prototype margins were tested against zero with two-sided one-sample t tests.Benjamini–Hochberg false-discovery-rate correction was applied for multiple comparisons.Source: §2.8 · p. 13
• Adjusted p values were used for statistical interpretation after Benjamini–Hochberg correction.Source: §2.8 · p. 13
• Group-level fMRI maps used an uncorrected voxel threshold of p < 0.001 followed by cluster-level family-wise-error correction at p789 < 0.05.Source: §2.8 · p. 13

3.1. fMRI Reveals Finger-Specific Spatial Representations

• Finger activations overlapped in shared motor networks, yet execution showed finger-dependent spatial shifts and stronger, broader activation than imagery.Imagery had a more spatially restricted shared network, with less consistent primary sensorimotor involvement; neighboring fingers had more similar activation patterns than distant fingers.Source: §3.1 · p. 14, Figure 2 · p. 15

3.2. FRIST Improves EEG-Only Individual-Finger Decoding Through Two Stages

• 74.53% vs 66.93% (two-class ME), 56.58% vs 44.83% (three-class ME), 85.63% vs 80.78% (two-class MI), and 69.90% vs 60.93% (three-class MI) for FRIST vs EEGNet Base.Source: §3.2 · p. 16, Figure 3 · p. 16
• Stage 2 added significant gains in three-class ME (Δ = 1.85%), two-class MI (Δ = 0.77%), and three-class MI (Δ = 2.00%), but not two-class ME.Across the four settings, Stage 1 provided significant gains over Base of 8.12, 9.91, 4.07, and 6.96 percentage points, respectively.Source: §3.2 · pp. 16–17

3.3. fMRI-Informed Spectral and Cross-Modal Representation Analyses

• FRIST-S1 exceeded the Static5 EEG-only control by 5.49, 6.82, 4.39, and 4.67 percentage points across two-class ME, three-class ME, two-class MI, and three-class MI.All four comparisons remained significant after multiple-comparison correction.Source: §3.3 · p. 17, Figure 4 · p. 17
• Held-out EEG embeddings had positive fMRI-centroid prototype margins in every setting, with means of 0.198, 0.115, 0.322, and 0.225; all exceeded zero significantly.The settings were two-class ME, three-class ME, two-class MI, and three-class MI, respectively.Source: Figure 4 · p. 17, §3.3 · p. 18

3.4. FRIST Retains EEG Decoding Gains Without Target-Participant fMRI

• Using fMRI from only the other 11 participants, FRIST retained gains over Base in all four settings; seven of eight cohort comparisons were nonsignificant.Final three-class ME accuracy was 1.88 points higher with the full-cohort reference.Source: §3.4 · p. 18, Figure 5 · p. 18

3.5. FRIST Generalizes Across Multiple EEG Deep Learning Decoders

• FRIST beat Base in all 16 matched comparisons; EEG Conformer led all four settings, reaching 87.40% (two-class MI) and 72.54% (three-class MI).Source: §3.5 · p. 19, Figure 6 · p. 19

4. DISCUSSION

• FRIST exceeded the matched EEG-only Base by up to 11.75 percentage points across the four movement-execution and motor-imagery settings.The largest total gains occurred in the two three-class settings.Source: §4 · p. 20
• Finger labels provide a biologically meaningful correspondence between EEG and fMRI representations despite their different temporal scales and signal-generating mechanisms.The group-level analysis found overlapping sensorimotor activation with finger-dependent spatial organization, and stronger, more extensive ME responses than MI responses.Source: §4 · p. 20
• Stage 1 improved accuracy by 4.07–9.91 percentage points across four conditions, while Stage 2 added significant gains in three-class ME and both MI settings, but not two-class ME.Stage 2 changed accuracy by −0.52, +1.85, +0.77, and +2.00 percentage points across the four settings, respectively.Source: §4 · p. 21
• fMRI guidance remained effective when the target participant’s fMRI data were excluded, using finger-specific representations learned from a population EEG-fMRI dataset.The authors note that larger cohorts are needed to characterize generalizability across broader populations.Source: §4 · p. 22
• FRIST significantly exceeded the corresponding Base model in all 16 matched comparisons across EEGNet, EEG Conformer, TCN, and 1D-CNN–Transformer.The reported results suggest the framework is not tied to a particular EEG architecture.Source: §4 · p. 22
• An exploratory inverse correlation between baseline performance and gain was consistent with stronger encoders already recovering part of the information supplied by the fMRI prior.Source: §4 · p. 23
• The evaluation covered only two- and three-finger tasks and remained offline, so five-finger scalability and closed-loop performance under feedback, nonstationarity, and user adaptation remain untested.The authors also describe the EEG-fMRI cohort as relatively modest and call for larger, more diverse cohorts.Source: §4 · pp. 22–23

5. CONCLUSION

• FRIST transfers EEG–BOLD spectral coupling and finger-discriminative fMRI structure from a separate simultaneous EEG–fMRI reference dataset to improve EEG-only individual-finger decoding.Source: §5 · p. 23
• Fixed fMRI-informed spectral projections and residual shared-space alignment with contrastive learning provide complementary benefits across two- and three-class ME and MI tasks.The conclusion also reports performance enhancement across EEG decoders and retention of most gains when target-participant fMRI was excluded.Source: §5 · p. 23
• The evaluation was performed offline using chronological held-out sessions, supporting further prospective evaluation in closed-loop EEG finger-level BCI systems.Source: §5 · p. 24