PaperlayerView exact revision on arXiv

Source-linked AI summary

Identifying Stable Patterns over Time for Emotion Recognition from EEG

Wei-Long Zheng, Jia-Yi Zhu, Bao-Liang Lu

arXiv:1601.02197v1cs.HCcs.AI

TL;DR

Prior studies reported emotion-related EEG patterns, but their stability over time was not fully investigated. This paper systematically evaluates machine-learning methods on DEAP and a newly developed dataset to identify stable patterns across sessions. The results indicate relatively stable neural patterns within and between sessions, with emotion-specific frequency-band and brain-region signatures.

  • Problem

    The stability of EEG neural patterns and emotion-recognition performance across time and sessions had not been systematically evaluated.

  • Method

    The study compares feature extraction, feature selection, feature smoothing, and classification methods on DEAP and a newly developed SEED subset, using GELM and cross-session evaluation.

  • Results

    Neural patterns are relatively stable within and between sessions, with positive, neutral, and negative emotions showing distinct frequency-band and brain-region signatures.

  • Takeaways & Limitations

    Stable EEG patterns can support emotion recognition across repeated sessions, including lateral temporal, parietal, occipital, and prefrontal signatures.

  • Takeaways & Limitations

    Existing public datasets lacked repeated-session EEG recordings for the same subjects, motivating development of a new dataset for stability analysis.

Abstract

from arXiv · show

In this paper, we investigate stable patterns of electroencephalogram (EEG) over time for emotion recognition using a machine learning approach. Up to now, various findings of activated patterns associated with different emotions have been reported. However, their stability over time has not been fully investigated yet. In this paper, we focus on identifying EEG stability in emotion recognition. To validate the efficiency of the machine learning algorithms used in this study, we systematically evaluate the performance of various popular feature extraction, feature selection, feature smoothing and pattern classification methods with the DEAP dataset and a newly developed dataset for this study. The experimental results indicate that stable patterns exhibit consistency across sessions; the lateral temporal areas activate more for positive emotion than negative one in beta and gamma bands; the neural patterns of neutral emotion have higher alpha responses at parietal and occipital sites; and for negative emotion, the neural patterns have significant higher delta responses at parietal and occipital sites and higher gamma responses at prefrontal sites. The performance of our emotion recognition system shows that the neural patterns are relatively stable within and between sessions.

1 INTRODUCTION

The paper addresses the underexplored stability of EEG emotion-recognition patterns and models over time. It develops a dataset and systematically evaluates methods to identify patterns consistent across subjects and sessions.

  • Motivation: Emotion recognition from EEG remains challenging because emotions vary across individuals and lack a theoretical ground-truth label.Emotional states depend on time, context, space, language, culture, and race.
  • Research gap: Stable EEG patterns are defined as neural activities in critical brain areas and frequency bands shared across individuals and sessions under different emotional states.The paper examines whether these patterns remain consistent across repeated sessions.
  • Research questions: The study asks whether EEG discriminates emotions, whether stable neural oscillation or brain-region patterns exist, and whether models perform consistently across days.
  • Contributions: The authors develop a novel publicly available emotion EEG dataset, a subset of SEED, for evaluating stability across subjects and sessions.They state that no public EEG dataset was previously available for this purpose.
  • Contributions: The study systematically compares feature extraction, feature selection, feature smoothing, and pattern classification methods using DEAP and SEED.It also adopts discriminative GELM and cross-session schemes to evaluate model stability over time.
  • Contributions: Experiments indicate that neural signatures for positive, neutral, and negative emotions are relatively stable within and between sessions.

2 RELATED WORK

Prior EEG emotion-recognition research identifies neural correlates and evaluates subject-dependent or subject-independent performance, but largely overlooks temporal stability. The paper motivates a systematic cross-session analysis and a new dataset because existing public datasets lack repeated-session EEG for the same subjects.

  • Emotion recognition methods: Physiological signals such as EEG provide noninvasive measures of emotional activity, complementing external expressions that may mask internal emotional states.
  • Emotion recognition methods: Affective-computing studies have used EEG features to estimate emotional states, while public datasets such as DEAP support emotion-classification research.
  • EEG patterns associated with emotions: Prior neuroscience work suggests emotion-related neural signatures may appear as distributed brain-activity patterns rather than dedicated processing modules.
  • EEG patterns associated with emotions: Reported correlates include increased gamma power for negative valence over the left temporal region and frontal asymmetry associated with approach and withdrawal tendencies.
  • Stability gap: Existing EEG emotion studies commonly use subject-dependent and subject-independent evaluations but extract activated patterns without considering the time factor.
  • Stability gap: Earlier EEG stability studies often quantify reliability statistically rather than through pattern-classifier performance.
  • Stability gap: A pilot stability study examined individual features from 14-channel EEG in a small subject group, whereas this paper evaluates neural-pattern stability systematically over time.
  • Datasets: DEAP and MAHNOB HCI lack EEG recordings from different sessions for the same subject, preventing direct investigation of stable patterns over time.The authors therefore develop a new emotional EEG dataset as a subset of SEED1.

3 EMOTION EXPERIMENT DESIGN

The experiment collects EEG during repeated film-clip emotion tasks to examine neural signatures and stability across sessions and individuals. Participants complete three sessions separated by at least one week, with EEG recording and post-clip emotional questionnaires.

  • Experimental setting: The experiment uses film clips with scene and audio to elicit strong subjective and physiological emotional changes.The clips are intended to expose participants to more real-life scenarios.
  • Participants and sessions: Fifteen participants complete the emotion experiment in three sessions separated by one week or longer.The participants are seven males and eight females, with mean age 23.27 and standard deviation 2.37.
  • Data acquisition: EEG is recorded from 62 active AgCl electrodes using an ESI NeuroScan System at a 1000 Hz sampling rate.Electrodes follow the international 10-20 system.
  • Trial protocol: Each experiment contains 15 trials, followed by feedback in which participants report emotional reactions and rate subjective arousal.The questionnaire asks about experienced emotion, emotion during viewing, prior movie exposure, and clip comprehension.
  • Preprocessing: Raw EEG is downsampled to 200 Hz and bandpass-filtered between 0.5 Hz and 70 Hz for signal processing.These steps are used to prepare the EEG data after collection.

4 METHODOLOGY

The methodology combines multiple EEG feature representations, dimensionality-reduction or feature-selection methods, temporal smoothing, and classifiers. It uses a linear dynamic system to model gradual emotional changes and GELM to incorporate class-based graph regularization.

  • Feature extraction: Six EEG feature types are extracted across five frequency bands: PSD, DE, DASM, RASM, ASM, and DCAU.The bands are delta, theta, alpha, beta, and gamma; PSD uses STFT with a 1 s window and no overlap.
  • Feature extraction: DASM and RASM capture hemispheric asymmetry, while DCAU captures spectral differences along the frontal-posterior direction.DASM and RASM use 27 electrode pairs, whereas DCAU uses 23 frontal-posterior pairs.
  • Feature smoothing: The linear dynamic system models observed EEG features as generated from hidden, gradually changing emotional states and is compared with moving-average smoothing.The model parameters are estimated by maximum likelihood through EM, and latent states are inferred from observation sequences.
  • Feature selection: PCA projects high-dimensional features into a lower-dimensional space, while MRMR selects features using relevance and redundancy criteria.PCA minimizes information loss under orthogonal transformation; MRMR uses mutual information and seeks near-optimal feature subsets through incremental search.
  • Classification: The study compares KNN, logistic regression, SVM, and discriminative GELM classifiers for emotion recognition.GELM constrains outputs from samples in the same class to be similar through graph regularization, while its output weights can be calculated directly.

5 EXPERIMENT RESULTS

Experiments compare EEG features, smoothing, classifiers, and dimensionality reduction, then test emotion-recognition stability across sessions and subjects. Results identify discriminative neural patterns and show that performance declines across more distant sessions and unseen subjects.

  • Neural patterns: Beta and gamma features performed better than other frequency bands, while lateral temporal activity was stronger for positive than negative emotion.Negative emotion showed enhanced prefrontal beta and gamma energy relative to positive emotion.
  • Feature selection: Dimensionality reduction had limited effect: PCA accuracy fell from 91.07% to 88.46% at dimension 210, then reached 89.57% at dimension 160.MRMR was preferred because it preserves channel and frequency information while selecting relevant, minimally redundant features.
  • Stability over time: A 79.28% comparative mean accuracy was achieved across different sessions, with nearer-time sessions outperforming more separated sessions.The authors report that performance may worsen over time and propose model adaptation as future work.
  • Cross-subject stability: Subject-independent features reached 60.93% mean accuracy with a 13.95% standard deviation, supporting the possibility of a common model while retaining individual differences.Cross-subject and cross-session performance was worse than performance on single experiments.

6 CONCLUSIONS AND FUTURE WORK

The study finds that machine-learning methods can recognize emotions from EEG while revealing neural patterns that remain relatively stable across sessions. Its conclusions are based on systematic evaluations across DEAP and SEED, with future work extending the approach beyond the three studied emotions and accounting for additional individual and temporal factors.

  • GELM with differential entropy features outperformed other evaluated methods on both DEAP and SEED datasets.The best average accuracies were 69.67% for DEAP and 91.07% for SEED under 5-fold cross-validation.
  • 79.28% average classification accuracy was achieved when training and testing used data from different SEED sessions.These cross-session results were used to investigate stability over time.
  • Stable EEG patterns associated with positive, neutral, and negative emotions were consistent across repeated measurements within the same subject.The reported emotion-specific patterns included band- and region-specific responses across sessions.
  • The study examined only positive, neutral, and negative emotions, leaving broader emotion-category generalization for future evaluation.Future work also proposes considering gender, age, race, individual differences, and temporal evolution.
Loading 1601.02197v1…