PaperlayerView exact revision on arXiv

Source-linked AI summary

FOCUS: A Lightweight Mamba-2 Model for EEG-Based Cognitive Workload Classification

Ioannis Kyprakis, Stefanos Gkikas, Chariklia Chatzaki, Zongyi Jiang, Randy Gomez, Raul Fernandez Rojas

arXiv:2610.32361v1cs.AI

TL;DR

EEG-based workload assessment faces variation across people, tasks, and experimental contexts, despite its relevance to monitoring in high-risk settings. The paper presents FOCUS, a compact Mamba-2 model using fixed frequency bands for binary workload classification. In a controlled pilot, its selected configuration achieves 74.75% ALL accuracy with 0.04 M parameters and 0.11 GFLOPs.

  • Problem

    EEG workload patterns vary across individuals, tasks, and experimental contexts, making generalization across task conditions an important challenge.

    Source: §I · p. 2

  • Method

    FOCUS classifies binary workload from preprocessed multi-channel EEG using fixed FFT band decomposition and a compact Mamba-2 architecture.

    Source: C. Proposed Architecture: FOCUS · p. 3, 1) Input Normalisation · p. 3

  • Results

    74.75% ALL accuracy with 0.04 M parameters and 0.11 GFLOPs for the selected FOCUS configuration.

    Source: §IV · p. 5

  • Takeaways & Limitations

    The selected configuration provides a favorable performance–cost trade-off in the evaluated pilot setting.

    Source: §VI · p. 6

    The 11-participant dataset was recorded in one controlled laboratory environment, and generalizability to clinical or operational settings has not been tested.

    Source: §VI · p. 6

Abstract

from arXiv · show

Cognitive workload estimation from physiological signals is increasingly relevant in high-risk environments, where continuous and unobtrusive monitoring can inform adaptive systems and safety interventions. This paper presents FOCUS (Frequency-band Optimized Cognitive-load Ultralight SSM), a compact Mamba-2-based model for EEG-based binary workload classification under constrained data conditions. Rather than learning frequency sensitivity from data, FOCUS encodes preprocessed multi-channel EEG using a fixed FFT decomposition that covers the five canonical EEG bands. The proposed model combines depthwise-separable convolution and a weight-shared encoder-bottleneck Mamba-2 pair, with 0.04 million parameters and 0.11 GFLOPs. Evaluated in a pilot study with 11 participants under leave-one-subject-out cross-validation across three cognitively distinct tasks, FOCUS achieves 74.75% accuracy on the full dataset at 0.08 ms GPU model-forward latency. In the weight-sharing ablation, the shared configuration achieves 4.55 percentage points higher accuracy on the full dataset than the non-shared variant while reducing the SSM parameter budget; however, this difference does not reach statistical significance under a two-sided Wilcoxon signed-rank test ($p=0.083$). Overall, the selected FOCUS configuration provides a favorable trade-off between classification performance and computational cost within this controlled pilot setting.

I. INTRODUCTION

Cognitive workload monitoring can support safety and performance in high-risk domains, but EEG-based assessment must contend with neural patterns that vary across people, tasks, and contexts. Prior work applies EEG workload estimation across multiple architectures and health-related settings.

I. INTRODUCTION

  • Cognitive workload is the mental effort working memory requires to perform a task; subjective tools such as NASA-TLX apply after task completion, unlike continuous physiological monitoring.Source: §I · p. 1
  • EEG captures cortical activity and rapid changes associated with cognitive engagement, including oscillatory activity linked to cognitive and memory processes.Source: §I · pp. 1–2
  • EEG workload patterns vary across individuals, tasks, and experimental contexts, making generalization across task conditions an important challenge.Source: §I · p. 2

II. RELATED WORK

  • Prior EEG workload studies use CNNs and LSTMs, while Transformers and Conformers have also achieved competitive results at low computational cost.Source: §II · p. 2
  • EEG workload monitoring has been explored in surgical and training contexts, as well as adaptive learning and cognitive training applications.Source: §II · p. 2

III. METHODOLOGY

The study evaluates workload classification using controlled easy and hard conditions across three tasks in an 11-participant EEG dataset. Its methodology describes preprocessing and a fixed FFT-based frequency-band embedding.

A. Data Collection

  • Eleven participants completed reasoning, numerical problem-solving, and video-game tasks at easy and hard difficulty levels while 14-channel EEG was recorded.Source: A. Data Collection · p. 2
  • Each task condition lasted two minutes and was divided into twelve non-overlapping 10-second EEG samples, yielding 264 samples per task.Source: A. Data Collection · p. 2

B. Pre-processing

2) D1 — FFT Band Embedding:

C. Proposed Architecture: FOCUS

FOCUS is a compact Mamba-2 model for binary workload classification from preprocessed multi-channel EEG. Its architecture uses fixed EEG bands and parameter-efficient convolution and shared SSM blocks.

C. Proposed Architecture: FOCUS

  • FOCUS combines fixed FFT band decomposition, depthwise-separable temporal convolution, shared encoder-bottleneck Mamba-2 blocks, and minimal model dimensions.Source: C. Proposed Architecture: FOCUS · p. 3

1) Input Normalisation:

  • Five fixed masks cover delta, theta, alpha, beta, and gamma bands, and inverse FFT recovers each band signal at the full 1280-point temporal resolution.Source: 1) Input Normalisation · p. 3
  • The five-band signals are stacked across channels and projected into the model dimension; the embedding stage has only one learnable component, while its band masks have none.Source: 1) Input Normalisation · p. 3
  • The model has 0.04 M parameters and 0.11 GFLOPs, with 0.08 ms model-forward latency per 10-second, 14-channel sample on an NVIDIA GeForce RTX 3090 GPU.The latency excludes signal acquisition and offline ICA artifact removal.Source: 1) Input Normalisation · p. 3

3) Mamba-2 Encoder Block (D4):

The Mamba-2 encoder processes the embedded EEG sequence using an input-dependent state-space recurrence. A pre-norm residual connection wraps the SSM.

3) Mamba-2 Encoder Block (D4):

4) D2 — Depthwise-Separable Temporal Convolution:

FOCUS uses depthwise-separable convolution to capture local temporal patterns efficiently, then reduces temporal resolution through max-pooling.

4) D2 — Depthwise-Separable Temporal Convolution:

5) D3 — Bottleneck Block with Weight Sharing:

The bottleneck Mamba-2 pass refines the downsampled sequence while reusing the encoder SSM parameters, reducing the SSM parameter budget.

5) D3 — Bottleneck Block with Weight Sharing:

6) Classification Head:

The classification head pools the bottleneck output into a single descriptor, applies dropout, and maps it to class logits.

6) Classification Head:

  • Global temporal average pooling produces one descriptor, which is regularized with dropout (p = 0.3) before a linear layer maps it to class logits.Source: 6) Classification Head · p. 4

D. Training Configuration

Training uses AdamW with a cosine learning-rate schedule, per-channel augmentation, and participant-held-out evaluation across 11 folds.

D. Training Configuration

  • Models train for 190 epochs with AdamW (η=10^-4, weight decay λ=0.05), cosine annealing, 20 warm-up epochs, 10 cool-down epochs, batch size 32, and label smoothing 0.10.Source: D. Training Configuration · p. 4
  • Per-channel training augmentation applies polarity inversion, Gaussian noise, and temporal masking, each with probability drawn uniformly from [0.01, 0.10].Temporal masking uses a window size of 15–30% of the sequence.Source: D. Training Configuration · p. 4
  • Each LOSO fold trains from scratch on 10 participants and evaluates on the held-out participant; reported performance is mean accuracy across 11 folds.Each fold's result is its maximum held-out accuracy over the fixed 190-epoch horizon.Source: D. Training Configuration · p. 4

IV. EXPERIMENTAL EVALUATION & RESULTS

The experiments select A1 as the final FOCUS configuration based on its cost–performance tradeoff. Ablations examine weight sharing, convolution type, model width, and SSM state dimension under leave-one-subject-out evaluation.

Experimental evaluation

  • Evaluation used LOSO across 11 participants; ALL was a separate full-dataset experiment, not an average of the three task-specific results.Binary labels represented predefined easy and hard conditions, not direct subjective workload measurements.Source: §IV · p. 4

Weight sharing (D3)

  • 4.55 percentage points higher ALL accuracy for A1 than A2 (74.75% vs 70.20%) was descriptive; the Wilcoxon test was not significant (p = 0.083).A2 increases parameters from 0.04 M to 0.07 M while GFLOPs remain 0.11.Source: §IV · p. 4

Depthwise-separable convolution (D2)

  • 2.9× fewer convolutional parameters for depthwise-separable convolution (4,288 vs 12,288) motivated its retention; the ALL accuracy difference was 0.13 percentage points and not significant (p = 0.888).Replacing it with standard Conv1d adds 0.02 GFLOPs; task-level accuracy differences were small descriptively.Source: §IV · pp. 4–5

Model width (D4 — d)

  • 75.00% ALL accuracy at d = 128 was only 0.25 percentage points above A1, while increasing cost to 0.14 M parameters and 0.40 GFLOPs; d = 64 was retained.Reducing width to d = 32 lowered ALL accuracy to 69.07%, a descriptive 5.68 percentage-point difference from A1.Source: §IV · p. 5

SSM state dimension (D4 — dstate)

  • 70.45% ALL accuracy with dstate = 8 compared with 74.75% for A1 was not a statistically significant difference (p = 0.102).The results suggest sensitivity to state dimension in this configuration but establish neither a significant performance difference nor a specific temporal mechanism.Source: §IV · p. 5

Final configuration

  • A1 achieves 74.75% ALL accuracy with 0.04 M parameters, 0.11 GFLOPs, and 0.08 ms model-forward latency per sample.A5 reaches 75.00% ALL accuracy, but its difference from A1 is not statistically significant and it uses 0.14 M parameters and 0.40 GFLOPs.Source: §IV · p. 5

V. COMPARISON WITH EXISTING WORK

FOCUS is compared with published EEG models, a multimodal framework, and a plain Mamba-2 baseline. Its performance varies by task, while its mean-score and computational profile compare favorably with several alternatives.

Comparison setup

  • Published EEGNet and 1BT results use the same EEG data and subject-independent LOSO protocol; the multimodal framework adds five peripheral modalities to EEG.The plain Mamba-2 baseline and FOCUS were evaluated in this study.Source: §V · p. 5

Task and pooled accuracy

  • 74.75% ALL accuracy for FOCUS versus 62.88% for the multimodal framework; FOCUS leads on MATH and GAME, while the multimodal framework leads on IQ.FOCUS reaches 75.76% accuracy on MATH and 76.14% on GAME; the multimodal framework reaches 72.35% on IQ.Source: §V · p. 5

Cost–performance comparison

  • 73.57% mean score makes FOCUS highest among the five models, with 0.04 M parameters and 0.11 GFLOPs.Compared with plain Mamba-2, FOCUS has a 4.79 percentage-point higher mean score while using approximately 33% fewer parameters and 21% fewer GFLOPs.Source: §V · p. 5

VI. DISCUSSION & CONCLUSION

The selected A1 configuration offers the best observed performance–cost trade-off among the evaluated variants. The results remain pilot evidence, with limited generalizability and the fixed FFT front-end untested against alternatives.

VI. DISCUSSION & CONCLUSION

  • A1’s 74.75% ALL accuracy exceeds A2’s 70.20% by 4.55 points, but the paired Wilcoxon test is not significant (p = 0.083).The task-level effect is uneven, and the results do not establish weight sharing as a significant improvement or an implicit-regularisation mechanism.Source: §VI · pp. 5–6
  • Depthwise-separable convolution reduces convolutional parameters 2.9×; halving dstate yields 4.30 points lower ALL accuracy, while d=128 gains 0.25 points at substantially higher computational cost.The reported accuracy differences for the convolution, state-dimension, and width comparisons are not statistically significant; A1 is selected on observed cost–performance trade-off.Source: §VI · p. 6
  • 74.75% ALL accuracy with 0.04 M parameters and 0.11 GFLOPs makes A1 the selected performance–cost trade-off; its GPU model-forward latency is 0.08 ms per sample.A5 reaches 75.00% ALL accuracy, but its 0.25-point difference is not statistically significant and it uses 0.14 M parameters and 0.40 GFLOPs.Source: §VI · p. 6, Table I · p. 6
  • The pilot uses 11 participants in one controlled laboratory environment; generalizability to clinical or operational settings, different headsets, and longer sessions remains untested.The fixed FFT decomposition was not compared with alternative EEG front-ends, and the binary easy/hard targets are not direct estimates of subjective workload.Source: §VI · pp. 6–7
  • Table II compares FOCUS with published baselines and a plain Mamba-2 baseline under the same dataset and subject-independent LOSO protocol.The supplied discussion passages do not report the table’s cell values.Source: Table II · p. 7
Loading 2610.32361v1…