PaperlayerView exact revision on arXiv

Source-linked AI summary

EEG-Based Emotion Recognition Using Regularized Graph Neural Networks

Peixiang Zhong, Di Wang, Chunyan Miao

arXiv:1907.07835v4cs.CVcs.AIcs.HCcs.LG

TL;DR

EEG emotion-recognition methods often underuse channel topology and must handle cross-subject variation and noisy labels. The paper proposes RGNN with biologically inspired local/global graph relations plus NodeDAT and EmotionDL, and reports superior performance over competitive methods in most settings. Its analyses also identify informative brain regions and inter-channel relations, while noting boundaries for smaller-channel graphs and current adjacency assumptions.

  • Problem

    Existing EEG emotion-recognition methods do not fully exploit channel topology and face cross-subject EEG variation and noisy emotion labels.

  • Method

    RGNN models EEG channels as graph nodes with a sparse biologically inspired adjacency matrix capturing local and global relations, regularized by NodeDAT and EmotionDL.

  • Results

    RGNN achieves superior performance over competitive baselines and BiHDM in most experimental settings, with its adjacency matrix and two regularizers producing consistent and significant gains.

  • Takeaways & Limitations

    The analyses identify pre-frontal, parietal, and occipital regions and left-right hemispheric relations as informative for EEG-based emotion recognition.

  • Takeaways & Limitations

    The model may require simpler architectures or advanced regularization to avoid over-smoothing when applied to EEG signals with fewer channels.

Abstract

from arXiv · show

Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes. Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels. In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition. RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels. Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization. In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively. Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings. Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model. Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition.

1 INTRODUCTION

EEG-based emotion recognition is motivated by EEG’s access to emotional information and practical advantages, but existing methods underuse channel topology and face cross-subject and noisy-label challenges. RGNN addresses these issues with biologically inspired local/global relations and two regularizers, achieving strong reported performance and identifying informative regions and connections.

  • Motivation: EEG records cortical voltage fluctuations through scalp electrodes and can reveal information about human emotional states.Physiological signals are difficult to hide or disguise, and EEG devices are increasingly noninvasive, easy to use, and inexpensive.
  • Challenges: Existing EEG methods extract channel-wise features and train classifiers but generally do not model the topological structure of EEG channels.Examples include PSD, wavelet, nonlinear dynamical, differential entropy, and PSD asymmetry features paired with conventional classifiers.
  • Challenges: CNN-, RNN-, and nearest-channel GNN approaches can lose spatial information or distant inter-channel dependencies, including informative left-right frontal asymmetry.Flattening 3D channel arrangements into 2D representations may cause information loss, while local-only relations omit distant channels.
  • Challenges: Subject-independent recognition requires handling cross-subject variation, while noisy emotion labels remain an unaddressed challenge in prior EEG-based work.Prior domain adaptation methods improved subject-independent accuracy by around 10%, but noisy emotion labels had not been addressed in this setting.
  • Proposed approach: RGNN models EEG channels as graph nodes and uses biologically inspired sparse topology to capture local anatomical and global functional relations.Local connections represent nearby neuronal groups, while global connections link distant groups across hemispheres.
  • Proposed approach: NodeDAT regularizes node-level source-target feature discrepancies, while EmotionDL learns label distributions to improve robustness to cross-subject variation and noisy labels.The paper evaluates RGNN on SEED and SEED-IV in subject-dependent and subject-independent settings and reports superior performance in most settings.

2 RELATED WORK

Related work spans EEG feature extraction, topology-aware classification, graph neural networks, domain adaptation, and learning with noisy labels. These studies motivate RGNN’s topology-aware graph representation and node-level domain regularization.

  • EEG emotion recognition: EEG emotion-recognition features include single-channel statistics, PSD, differential entropy, wavelets, and multi-channel measures such as hemispheric asymmetry and functional connectivity.The literature therefore includes both channel-specific and inter-channel representations.
  • EEG emotion recognition: Topology-invariant classifiers ignore EEG feature topology, whereas CNNs and GNNs aggregate information from neighboring channels to learn topology-aware representations.Topology-aware methods operate in Euclidean or non-Euclidean spaces.
  • Graph neural networks: GNNs process graph-domain data by learning representations over nodes and their connections, supporting applications such as molecular, social, and knowledge graphs.The paper builds on this graph representation framework for EEG channels.
  • Domain adaptation: Unsupervised domain adaptation mitigates source-target domain shifts through instance reweighting or domain-invariant feature learning, including adversarial training.NodeDAT extends adversarial domain adaptation to minimize source-target discrepancies for each graph node.
  • Learning with noisy labels: Noisy-label methods commonly use transition matrices or robust loss functions to reduce the impact of corrupted supervision.The paper’s EmotionDL addresses noisy labels through label-distribution learning rather than these conventional approaches.

3 PRELIMINARIES

The preliminaries formulate graph convolution as topology-aware feature aggregation and describe normalized, spectral, polynomial, and simplified graph convolutions. These operations progressively motivate SGC as an efficient linear graph representation method.

  • Graph neural networks: A graph represents nodes and weighted edges with a feature matrix X and adjacency matrix A, while GNNs transform X into output features Z.Self-loops are included through Aii = 1.
  • Graph convolution: A basic graph-convolution layer aggregates each node’s neighborhood, including itself, using weighted sums followed by a nonlinear transformation.Repeated application can make hidden-state values overly large because of summation.
  • Graph convolution: GCN normalizes neighborhood aggregation using the diagonal degree matrix D, preventing hidden representations from growing overly large.The normalized operator uses degree information from A.
  • Graph convolution: Applying L graph-convolution layers aggregates node features within an L-sized neighborhood.Each layer expands the receptive field through successive neighborhood aggregation.
  • Simple graph convolution: SGC removes nonlinearities and combines layer transformations into one matrix, computing a topology-aware transformation followed by a final linear transformation.This simplification accelerates training while retaining comparable performance.
  • Spectral graph convolution: Spectral graph convolution decomposes the normalized graph Laplacian into eigenvectors and eigenvalues, then applies graph filters in the Fourier domain.The filter uses element-wise multiplication with spectral coefficients.
  • Spectral graph convolution: Polynomial and Chebyshev approximations reduce the computational cost of spectral filtering by expressing filters through learnable polynomial parameters.Chebyshev polynomials provide a recursive approximation to the filtering operation.
  • Spectral graph convolution: GCN’s simplified spectral operation behaves like a low-pass filter by smoothing each node’s features with neighboring node features.The formulation uses K = 1, λmax = 2, and θ1 = −θ0.

4 REGULARIZED GRAPH NEURAL NETWORK

RGNN models EEG channels as a biologically inspired graph, combining local distance-based and global inter-channel relations. NodeDAT and EmotionDL regularize the model for cross-subject variation and noisy labels.

  • 4.1 Adjacency Matrix in RGNN: The adjacency matrix combines local connections initialized from inverse-square physical distance with selected global connections between symmetric electrode pairs.The global pairs include FP1–FP2, AF3–AF4, F5–F6, and other corresponding channels across the scalp.
  • 4.2 Dynamics of RGNN: RGNN extends simple graph convolution and computes graph representations from EEG features using normalized adjacency operations and sum pooling.The model uses an L1 penalty on the adjacency matrix and absolute adjacency values when computing node degrees.
  • 4.2.1 Node-wise Domain Adversarial Training: NodeDAT aligns source and target domains at each node representation before pooling to improve robustness in subject-independent classification.A gradient reversal layer confuses the domain classifier, with its scaling factor increasing from 0 to 1 during training.
  • 4.2.2 Emotion-aware Distribution Learning: EmotionDL converts single training labels into emotion-aware class distributions and optimizes their agreement with model outputs using KL divergence.The distributions encode dataset-dependent emotion relationships and assume participants are unlikely to generate opposite emotions.

5 EXPERIMENTAL SETTINGS

The experiments evaluate RGNN on SEED and SEED-IV under subject-dependent and subject-independent settings. The study uses established EEG datasets, pre-computed features, cross-validation protocols, and tuned model hyperparameters.

  • Datasets: Experiments use the public SEED and SEED-IV datasets, each containing 62-channel EEG recordings from 15 subjects.SEED includes negative, neutral, and positive emotions; SEED-IV includes neutral, sad, fear, and happy emotions.
  • Datasets: The study uses pre-computed differential entropy features, with SEED features smoothed by linear dynamic systems across five frequency bands.The bands are delta, theta, alpha, beta, and gamma, computed for each channel at one-second intervals.
  • Classification Settings: Both subject-dependent and subject-independent classification settings are evaluated on both datasets using protocols aligned with prior studies.Subject-independent evaluation uses leave-one-subject-out cross-validation, training on 14 subjects and testing on the remaining subject.
  • Classification Settings: Subject-dependent evaluation separates early trials for training from later trials for testing within each subject.SEED uses the first 9 of 15 trials for training; SEED-IV uses the first 16 of 24 trials and tests on the remaining 8.
  • Model Settings: RGNN uses two convolutional layers, 0.7 output-layer dropout, batch size 16, and Adam optimization, while tuning feature dimension and regularization parameters.NodeDAT is used only in subject-independent experiments, and comparisons use published baseline results.

6 PERFORMANCE EVALUATIONS

RGNN is evaluated across subject-dependent and subject-independent settings on SEED and SEED-IV, with performance varying by dataset, frequency band, and emotion. The model generally outperforms baselines, while subject-independent recognition remains more difficult.

  • Subject-Dependent Classification: RGNN outperforms all baselines, including BiHDM and DGCNN, when all frequency bands are used in subject-dependent classification on SEED and SEED-IV.The paper attributes the improvement partly to global inter-hemispheric asymmetry modeling and the simpler SGC-based architecture.
  • Subject-Independent Classification: RGNN is nearly 5% better than BiHDM on SEED-IV but marginally worse on SEED in subject-independent classification.RGNN also achieves the lowest accuracy standard deviation among baselines on both datasets.
  • Subject-Independent Classification: Subject-independent accuracy is consistently 5% to 30% lower than subject-dependent accuracy across models.The paper relates this gap to variability in EEG signals across subjects.
  • Frequency-Band Analysis: RGNN performs best in the gamma band for subject-dependent classification and in the delta, beta, and gamma bands for subject-independent classification on SEED.Most models perform better in beta and gamma than in lower-frequency bands.
  • Confusion-Matrix Analysis: RGNN recognizes positive and neutral emotions better than negative emotion on SEED, while sad emotion is recognized best on SEED-IV.Negative and sad emotion detection becomes relatively worse in subject-independent settings; fear is the exception on SEED-IV.

7 DISCUSSION

The discussion examines how adjacency design, regularization, hyperparameters, and learned neural patterns affect RGNN. Global connections, NodeDAT, EmotionDL, and biologically aligned graph structure are consistently linked to model behavior and performance.

  • Ablation Study: Distance-based adjacency initialization slightly outperforms functional-connectivity initialization, while random initialization performs worst.The results indicate that properly initializing the adjacency matrix benefits performance.
  • Ablation Study: A symmetric adjacency matrix reduces overfitting and improves accuracy.This design is one of the evaluated adjacency-matrix components.
  • Ablation Study: Removing global connections causes a noticeable performance drop on both datasets.Alternative global-connection sets perform slightly worse than the proposed model but much better than removing global connections.
  • Ablation Study: NodeDAT improves cross-subject performance and robustness by learning subject-invariant representations at the node level.Replacing it with graph-level DAT creates a clear performance gap, while removing NodeDAT increases accuracy variance.
  • Ablation Study: EmotionDL improves accuracy by around 3% on both datasets.The authors interpret this gain as supporting the presence of unintended emotions in emotion-eliciting trials.
  • Sensitivity Analysis: Increasing α helps subject-dependent classification up to 0.1 but reduces subject-independent performance beyond a threshold of 0.01.The paper attributes the difference to training-data size and the trade-off between regularization and bias.
  • Sensitivity Analysis: Increasing ϵ initially improves performance and then reduces it, with the model usually performing best at ϵ = 0.2.Excessive noise weakens true learning signals.
  • Brain-Region Analysis: Learned adjacency activations are strong in pre-frontal, parietal, and occipital regions across frequency bands.Temporal activation is additionally strong in the beta and gamma bands.

8 CONCLUSION

The RGNN captures local and global EEG inter-channel relations and improves emotion-recognition performance across most tested settings. Brain-region analyses identify informative regions and relations, while future work targets domain adaptation and smaller-channel graphs.

  • 8 CONCLUSION: RGNN captures local and global inter-channel relations and outperforms competitive baselines and BiHDM in most experimental settings.Its biologically inspired adjacency matrix and two regularizers provide consistent and significant performance gains.
  • 8 CONCLUSION: Pre-frontal, parietal, and occipital regions may be the most informative regions for EEG-based emotion recognition.
  • 8 CONCLUSION: Global relations between left and right hemispheres, plus local pairs (FP1, AF3), (F6, F8), and (FP2, AF4), may provide useful information.
  • 8 CONCLUSION: Future work includes more discriminative domain classifiers and techniques for handling imbalanced training and test samples.These directions aim to help the model learn more domain-invariant EEG representations.
  • 8 CONCLUSION: Applying RGNN to EEG signals with fewer channels may require simpler models or advanced regularization to avoid over-smoothing on small graphs.Improved spatial-resolution processing, such as spatial filtering, is also identified as a direction for exploration.
Loading 1907.07835v4…