PaperlayerView exact revision on arXiv

Source-linked AI summary

A Brain Wave Encodes a Thousand Tokens: Modeling Inter-Cortical Neural Interactions for Effective EEG-based Emotion Recognition

Nilay Kumar, Priyansh Bhandari, G. Maragatham

arXiv:2511.13954v1q-bio.NCcs.LG

TL;DR

Existing EEG emotion-recognition attention models generally treat electrode signals independently, despite the relevance of inter-electrode interactions. RBTransformer models those interactions in latent space using BDE tokens and electrode identity embeddings. The paper reports state-of-the-art results across three datasets and the evaluated dimensions and classification settings.

  • Problem

    Many attention-based EEG emotion-recognition models attend to individual spatial, spectral, or temporal inputs but do not model inter-electrode interactions.

    Source: §I · p. 2

  • Method

    RBTransformer processes BDE tokens with electrode identity embeddings and inter-cortical attention blocks that model electrode interactions in latent space.

    Source: §I · p. 3

  • Results

    RBTransformer achieves state-of-the-art performance across SEED, DEAP, and DREAMER, across the reported dimensions and binary and multi-class settings.

    Source: §I · p. 3

  • Takeaways & Limitations

    The paper reports that t-SNE visualizations and confusion matrices show emotional-class segregation in latent space and consistent discriminative performance across classes.

    Source: §I · p. 3

Abstract

from arXiv · show

Human emotions are difficult to convey through words and are often abstracted in the process; however, electroencephalogram (EEG) signals can offer a more direct lens into emotional brain activity. Recent studies show that deep learning models can process these signals to perform emotion recognition with high accuracy. However, many existing approaches overlook the dynamic interplay between distinct brain regions, which can be crucial to understanding how emotions unfold and evolve over time, potentially aiding in more accurate emotion recognition. To address this, we propose RBTransformer, a Transformer-based neural network architecture that models inter-cortical neural dynamics of the brain in latent space to better capture structured neural interactions for effective EEG-based emotion recognition. First, the EEG signals are converted into Band Differential Entropy (BDE) tokens, which are then passed through Electrode Identity embeddings to retain spatial provenance. These tokens are processed through successive inter-cortical multi-head attention blocks that construct an electrode x electrode attention matrix, allowing the model to learn the inter-cortical neural dependencies. The resulting features are then passed through a classification head to obtain the final prediction. We conducted extensive experiments, specifically under subject-dependent settings, on the SEED, DEAP, and DREAMER datasets, over all three dimensions, Valence, Arousal, and Dominance (for DEAP and DREAMER), under both binary and multi-class classification settings. The results demonstrate that the proposed RBTransformer outperforms all previous state-of-the-art methods across all three datasets, over all three dimensions under both classification settings. The source code is available at: https://github.com/nnilayy/RBTransformer.

I. INTRODUCTION

EEG offers a direct view of emotional activity, but existing attention-based approaches largely treat electrode signals independently. RBTransformer models inter-cortical interactions, and the paper reports state-of-the-art results across three datasets and multiple classification settings.

I. INTRODUCTION

  • EEG provides a direct, involuntary window into emotional responses, and dimensional models describe emotions along Valence, Arousal, and Dominance.Source: §I · p. 1
  • Earlier EEG emotion-recognition work used handcrafted features with shallow classifiers, followed by neural models for spatial, temporal, and inter-electrode dependencies.Source: §I · p. 2
  • Prior attention-based models emphasize individual spatial, spectral, or temporal features, while the authors identify inter-electrode interactions as an unaddressed modeling gap.Source: §I · p. 2

II. METHODOLOGY

  • RBTransformer uses BDE tokens, electrode-identity embeddings, and electrode × electrode attention to model inter-cortical interactions in latent space.The methodology section covers the preprocessing pipeline and model architecture.Source: §I · p. 3, §II · p. 3

I. INTRODUCTION

  • RBTransformer achieves state-of-the-art performance across SEED, DEAP, and DREAMER, across the reported dimensions and binary and multi-class settings.Source: §I · p. 3

A. Dataset Preprocessing

The preprocessing pipeline standardizes baseline and stimulus-response EEG segments, extracts band differential entropy features, and corrects trial features against baseline activity before model input.

A. Dataset Preprocessing

  • Recordings contain baseline and stimulus-response trial segments; trial data are divided into overlapping 512-sample windows with stride 117, while baseline data are averaged per trial and divided into non-overlapping 128-sample windows.Source: A. Dataset Preprocessing · p. 3
  • Each trial chunk and baseline segment is normalized per channel using its mean and standard deviation across time points.Source: A. Dataset Preprocessing · p. 4
  • Band Differential Entropy features are computed for each channel in the theta, alpha, beta, and gamma bands, yielding four features per channel.Source: A. Dataset Preprocessing · p. 4
  • Baseline BDE features are subtracted from corresponding trial BDE features to reduce subject-specific trends and remove static background activity.Source: A. Dataset Preprocessing · p. 4

B. Model Architecture

RBTransformer processes baseline-normalized BDE tokens through electrode-aware attention blocks to model inter-cortical interactions, then classifies the resulting representations. Its architecture is outlined in Figure 4.

B. Model Architecture

  • Baseline-normalized BDE tokens are the model’s input, with each token encoding an electrode’s spectral-energy deviation from baseline across EEG bands.A shared projection maps each 4-dimensional token to the model embedding dimension.Source: B. Model Architecture · p. 4
  • Learnable electrode identity embeddings are added after projection to retain each electrode’s spatial information in the attention inputs.The embedding is broadcast across the batch, followed by dropout.Source: B. Model Architecture · p. 4
  • Inter-cortical multi-head self-attention computes electrode-by-electrode weights, allowing each electrode to attend to every other electrode.The weights come from scaled query-key dot products and are applied to values; the heads are concatenated and projected.Source: B. Model Architecture · pp. 4–5
  • Each attention block follows self-attention with a feedforward network that refines per-electrode representations through nonlinear transformations and a residual connection.The feedforward network uses linear transformations, dropout, and GELU activation.Source: B. Model Architecture · pp. 5–6
  • The classification head averages representations across electrodes, applies layer normalization, and maps the pooled vector to class logits.Figure 4 schematically depicts RBTransformer’s inter-cortical attention architecture.Source: Figure 4 · p. 5, B. Model Architecture · p. 6

III. EXPERIMENTAL SETUP

The experimental setup section describes the datasets, performance metrics, and RBTransformer training configuration.

III. EXPERIMENTAL SETUP

  • The experimental setup covers the datasets used to train and benchmark RBTransformer.The passage identifies dataset details as one component of the section.Source: §III · p. 6
  • The section describes the metrics used to evaluate model performance.Source: §III · p. 6
  • The section also presents RBTransformer’s training configuration.Source: §III · p. 6

A. Datasets

The study benchmarks RBTransformer on SEED, DEAP, and DREAMER, which differ in participants, elicitation procedures, and EEG channel counts.

A. Datasets

  • SEED includes 15 participants who viewed 15 emotional movie clips per session, with EEG recorded using 62 channels.Participants completed three sessions at least one week apart; BDE features cover five canonical frequency bands.Source: A. Datasets · p. 6
  • DEAP contains EEG and peripheral recordings from 32 participants who rated responses to 40 one-minute music videos.Its EEG was recorded with a 32-channel system; ratings include Valence, Arousal, Dominance, and liking.Source: A. Datasets · p. 6
  • DREAMER contains EEG and ECG recordings from 23 participants who self-assessed emotions while viewing 18 audiovisual clips.EEG was recorded using a 14-channel device, and BDE features were extracted from overlapping time windows.Source: A. Datasets · p. 6

B. Metrics

The evaluation reports accuracy, precision, recall, and F1-score to characterize classification performance.

B. Metrics

  • Accuracy is the proportion of correct predictions across all samples, computed from true and false positives and negatives.Source: B. Metrics · p. 6
  • Precision is the proportion of predicted positives that are true positives.It is computed as TP divided by TP + FP.Source: B. Metrics · p. 6
  • Recall is the proportion of actual positives correctly identified by the model.The paper also calls recall sensitivity or true positive rate.Source: B. Metrics · p. 7
  • F1-score is the harmonic mean of precision and recall, balancing the two metrics in one measure.Source: B. Metrics · p. 7

C. Training Configurations

RBTransformer uses AdamW with scheduled learning-rate decay, label-smoothed cross-entropy, and training-fold-only SMOTE, alongside five-fold cross-validation.

C. Training Configurations

  • AdamW trained RBTransformer for 300 epochs with weight decay of 1 × 10−3 and a cosine-annealing learning rate from 1 × 10−3 to 1 × 10−6.Batch size was 256 for the first 150 epochs and 64 for the remaining epochs.Source: C. Training Configurations · p. 7
  • Cross-entropy loss used label smoothing set to 0.12, while SMOTE was applied only to training folds to handle class imbalance.Performance was evaluated with five-fold cross-validation.Source: C. Training Configurations · p. 7

IV. RESULTS AND DISCUSSION

The evaluation compares RBTransformer with existing state-of-the-art models across three EEG datasets in subject-dependent binary and multi-class tasks. It also reports extended metrics, class-wise confusion matrices, t-SNE visualizations, and ablations.

IV. RESULTS AND DISCUSSION

  • RBTransformer is evaluated on SEED, DEAP, and DREAMER in subject-dependent binary and multi-class classification tasks against existing state-of-the-art models.The evaluation also covers extended metrics, confusion matrices, t-SNE plots, and component ablations.Source: §IV · p. 7

A. Subject-Dependent Emotion Recognition Evaluation

Under binary classification, RBTransformer is compared with prior state-of-the-art models on DEAP and DREAMER across Valence, Arousal, and Dominance. Evaluation uses subject-dependent data splits with five-fold cross-validation.

A. Subject-Dependent Emotion Recognition Evaluation

  • Each dataset was evaluated separately by subject using an 80–20 train–validation split and five-fold cross-validation, with subjects represented in both splits as distinct subsets.Binary labels divided DEAP ratings into High above 5 and Low at or below 5, and DREAMER ratings into High above 3 and Low at or below 3.Source: A. Subject-Dependent Emotion… · p. 7, B. Binary-Class Classification… · p. 7

B. Binary-Class Classification Evaluation Results

C. Multi-Class Classification Evaluation Results

RBTransformer is evaluated in multi-class classification on SEED, DEAP, and DREAMER, with results compared against prior state-of-the-art models. The ablations examine DREAMER binary Arousal, while t-SNE visualizations cover both task settings across datasets and dimensions.

C. Multi-Class Classification Evaluation Results

D. Ablation Study

  • Removing Inter-Cortical Attention drops all four metrics by around 30% in DREAMER binary Arousal; the other ablated configurations also rank below the final model.Figure 6 covers attention and the training and regularization ablations.Source: Figure 6 · p. 8, D. Ablation Study · pp. 8–9
  • The t-SNE visualization spans SEED, DEAP, and DREAMER, covering binary and multi-class tasks across Valence, Arousal, and Dominance.Source: Figure 7 · p. 9

E. Extended Metric Evaluation for Performance Validation

RBTransformer is reported to outperform existing models across the evaluated datasets, dimensions, and classification settings. Extended metrics and visual analyses describe its performance across classes and in learned feature space.

E. Extended Metric Evaluation for Performance Validation

  • Across datasets and evaluated dimensions, RBTransformer shows balanced, stable, and consistently high Accuracy, Precision, Recall, and F1-Score over five cross-validation folds.Results are reported as mean and standard deviation; the evaluation covers binary and multi-class tasks.Source: E. Extended Metric Evaluation for… · p. 9

G. Confusion Matrix Analysis

F. Visualization of Learned Feature Representations by RBTransformer

  • t-SNE projections show separated emotion-class clusters for SEED, and for DEAP and DREAMER across their evaluated dimensions and classification settings.The reported groupings include three SEED classes, nine DEAP multi-class classes, and five DREAMER multi-class classes.Source: F. Visualization of Learned Feature… · p. 10

V. CONCLUSION

  • RBTransformer outperforms existing models across SEED, DEAP, and DREAMER, across the evaluated dimensions and binary and multi-class tasks.Source: §V · p. 11
Loading 2511.13954v1…