PaperlayerView exact revision on arXiv

Source-linked AI summary

Affective EEG-Based Person Identification Using the Deep Learning Approach

Theerawit Wilaiprasitporn, Apiwat Ditthapron, Karis Matchaparn, Tanaboon Tongbuasirilai, Nannapas Banluesombatkul, Ekapol Chuangsuwanich

arXiv:1807.03147v3eess.SPcs.CV

TL;DR

Affective EEG-based person identification has received limited attention compared with task-based EEG identification. This paper evaluates CNN-RNN cascades, comparing CNN-GRU with CNN-LSTM on DEAP, and reports high recognition rates across affective states, with CNN-GRU faster than CNN-LSTM.

  • Problem

    EEG-based person identification has rarely been studied while people are in different affective states, despite EEG-based identification being influenced by mood, stress, and mental state.

  • Method

    The paper combines CNNs for spatial EEG information with GRUs or LSTMs for temporal information and evaluates the cascades on DEAP.

  • Results

    CNN-GRU and CNN-LSTM reach up to 100% mean CRR across affective EEG conditions and significantly outperform the SVM baseline in higher-frequency and all-band settings.

  • Takeaways & Limitations

    CNN-GRU reaches 99.17% CRR with five electrodes, is faster than CNN-LSTM, and reportedly overcomes affective-state influence in EEG-based identification.

Abstract

from arXiv · show

Electroencephalography (EEG) is another mode for performing Person Identification (PI). Due to the nature of the EEG signals, EEG-based PI is typically done while the person is performing some kind of mental task, such as motor control. However, few works have considered EEG-based PI while the person is in different mental states (affective EEG). The aim of this paper is to improve the performance of affective EEG-based PI using a deep learning approach. \textcolor{red}{We proposed a cascade of deep learning using a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)}. CNNs are used to handle the spatial information from the EEG while RNNs extract the temporal information. \textcolor{red}{We evaluated two types of RNNs, namely, Long Short-Term Memory (CNN-LSTM) and Gated Recurrent Unit (CNN-GRU). } The proposed method is evaluated on the state-of-the-art affective dataset DEAP. The results indicate that CNN-GRU and CNN-LSTM can perform PI from different affective states and reach up to 99.90--100\% mean Correct Recognition Rate (CRR), significantly outperforming a support vector machine (SVM) baseline system that uses power spectral density (PSD) features. Notably, the 100\% mean \emph{CRR} comes from only 40 subjects in DEAP dataset. To reduce the number of EEG electrodes from thirty-two to five for more practical applications, the frontal region gives the best results reaching up to 99.17\% CRR (from CNN-GRU). Amongst the two deep learning models, we find CNN-GRU to slightly outperform CNN-LSTM, while having faster training time. \textcolor{red}{Furthermore, CNN-GRU overcomes the influence of affective states in EEG-Based PI reported in the previous works.

I. INTRODUCTION

The paper addresses limited EEG-based person-identification research across affective states by proposing deep-learning methods that combine spatial and temporal signal modeling. It evaluates CNN-GRU and CNN-LSTM approaches on DEAP, including reduced-electrode settings.

  • EEG-based person identification research using deep learning remains scarce, motivating investigation of affective EEG for identification.
  • The study evaluates affective EEG-based identification on the DEAP dataset and investigates whether EEG frequency bands differ in performance.
  • The proposed cascade uses CNNs for spatial information and GRUs for temporal information, performing both feature extraction and classification.
  • The experiments assess feasibility with five EEG electrodes from different scalp areas for more practical scenarios.
  • CNN-GRU converges faster and achieves slightly higher mean CRR than CNN-LSTM, especially with fewer electrodes, while reportedly overcoming affective-state influence.

II. THE DEEP LEARNING APPROACH TO EEG

The deep-learning approach uses CNN layers to learn local EEG patterns before recurrent layers model temporal information. The paper compares CNN-GRU and CNN-LSTM cascades, emphasizing GRU simplicity and training efficiency.

  • CNN features are supplied to recurrent layers because EEG contains spatial structure and temporal dynamics.
  • CNN layers extract local patterns and features through convolution and pooling operations.
  • GRUs use reset and update gates, whereas LSTMs use input, output, and forget gates with an additional memory component.
  • The CNN-GRU cascade is explored against CNN-LSTM because GRUs may require less training data and train faster than LSTMs.
  • The recurrent comparison evaluates both accuracy and training speed for EEG-based person identification.

III. METHODOLOGY

The methodology uses the DEAP affective EEG dataset and prepares EEG inputs for the proposed deep-learning approach. The section also introduces a figure comparing GRU and LSTM operations.

  • The methodology uses DEAP and describes preprocessing, data partitioning, and implementation of the proposed deep-learning approach.
  • Figure 1 compares the structures and operations of GRU and LSTM units.

A. Affective EEG Dataset

DEAP provides affective EEG recordings from 32 participants who watched music videos and rated valence and arousal. The study preprocesses and segments these recordings for person identification.

  • 32 healthy participants watched 40 affective music videos while EEG and subjective valence and arousal ratings were collected.
  • The EEG data were down-sampled to 128 Hz and cleaned of EOG artifacts using independent component analysis.
  • Signals were bandpass-filtered from 4.0–45.0 Hz and additionally separated into Theta, Alpha, Beta, Gamma, and all-band ranges.
  • The data were rereferenced, segmented into 60-second trials, and had 3-second pre-trial segments removed.

B. Subsampling and Cross Validation

The study subsampled affective EEG trials into short segments and used stratified cross-validation with independent trial assignments across training, validation, and test sets.

  • Subsampling and Cross Validation: Four affective states were defined from valence and arousal scores using 5 as the threshold between low and high levels.States were based on subjective scores from 1–9 for valence and arousal.
  • Subsampling and Cross Validation: Five EEG trials per affective state were randomly selected for each person to simulate a short registration procedure.The selected trials corresponded to EEG recordings from five video clips.
  • Subsampling and Cross Validation: 30 subsamples were produced per person by dividing each 60-second EEG trial into six 10-second segments.The 10-second segments were intended to support person identification from short-length EEG recordings.
  • Subsampling and Cross Validation: Each subsample contained 1280 EEG data points from 10 seconds recorded at a 128 Hz sampling rate.The data structure was organized as participants × 30 subsamples × 1280 EEG data points.
  • Subsampling and Cross Validation: Stratified 10-fold cross-validation allocated 80% of subsamples to training and 10% each to validation and testing, while keeping trial subsamples within one split.This made the training, validation, and test subsamples independent at the trial level.

C. Experiment I: Comparison of affective EEG-based PI among different affective states

Experiment I compared affective EEG-based person identification across affective-state settings using deep learning and conventional SVM approaches. The deep-learning setup converted multi-channel EEG into 2D mesh sequences before spatial-temporal modeling, while the study selected all affective states after finding that affective mixing did not reduce deep-learning performance.

  • Experiment I: EEG-based person identification was evaluated across affective states using deep learning and conventional machine learning on 4–40 Hz EEG.The experiment investigated which affective-state setting provided the highest correct recognition rate.
  • Deep Learning Approach: Multi-channel EEG was mapped into 9×9 2D meshes over time, producing a sequence suitable for the cascade CNN-recurrent model.Unallocated mesh positions were set to zero, and each mesh was normalized individually.
  • Deep Learning Approach: The deep-learning model applied TimeDistributed 2D CNN layers to each temporal window before recurrent processing.The CNN layers extracted spatial features from each mesh frame while preserving the sequence structure.
  • Conventional Machine Learning Approach: The SVM baseline used Welch power spectral density features, Z-score normalization, and validation-based selection of the capacity parameter C.The SVM formulation used a one-against-one multi-class strategy elsewhere in the described baseline procedure.
  • Experiment I: Deep-learning models maintained their performance when affective states were mixed, so the all-states setting was used in subsequent experiments.The study explicitly reports that affective states did not affect person-identification performance for the deep-learning models.

D. Experiment II: Comparison of affective EEG-based PI among EEGs from different frequency bands

Experiment II compared EEG frequency bands for affective person identification and selected the full 4–40 Hz range for the remaining experiments because it provided the best correct recognition rate.

  • Experiment II: EEG was divided into Theta (4–8 Hz), Alpha (8–15 Hz), Beta (15–32 Hz), Gamma (32–40 Hz), and all bands (4–40 Hz).Butterworth bandpass filtering was used to extract the frequency-band signals.
  • Experiment II: The all-bands setting (4–40 Hz) provided the best CRR and was retained for the remainder of the study.This selection was based on the results of Experiment II.
  • Sparse-electrode comparison: Figure 3 organizes five-electrode scalp configurations into five groups for evaluating sparse-electrode EEG-based person identification.The configurations correspond to different scalp regions and are labeled (a–e).

E. Experiment III: Comparison of affective EEG-based PI among EEGs from sets of sparse EEG electrodes

Experiment III evaluated sparse-electrode person identification and compared CNN-GRU and CNN-LSTM with conventional baselines. It varied recurrent and convolutional configurations, while also adapting Mahalanobis-distance baselines using PSD or coherence features.

  • Sparse-electrode configurations: Five-electrode configurations were defined for frontal, central-parietal, temporal, occipital-parietal, and frontal-parietal scalp regions.The experiment tested whether reducing the electrode count from 32 to 5 could preserve acceptable CRR.
  • Model comparison: The study varied CNN-layer counts and GRU/LSTM unit counts before comparing the resulting cascade models.Selected CNN configurations were subsequently cascaded with recurrent layers for evaluation.
  • Baseline comparison: The Mahalanobis baseline used PSD or COH features and classified samples from distances to participant class distributions.The study used all-element classification with stratified 10-fold cross-validation.
  • Baseline comparison: CNN-GRU and CNN-LSTM significantly outperformed the traditional SVM in every affective-state setting, including the all-states setting.The comparison used EEG in the 4–40 Hz range.

IV. RESULTS

The study compares mean CRR across affective states and recognition approaches using stratified 10-fold cross-validation. Deep-learning approaches significantly outperform SVM, while affective state does not significantly alter performance.

  • Affective state did not significantly affect PI performance for CNN-GRU, CNN-LSTM, or SVM.For CNN-GRU, CNN-LSTM, and SVM, the reported tests gave p=0.530, p=0.557, and p=0.457, respectively.
  • CNN-GRU and CNN-LSTM significantly outperformed SVM in every affective state, including all states.Pairwise comparisons reported p<0.01.
  • 100% mean CRR was reached by CNN-GRU in all states and by CNN-LSTM in LH.

B. Results II: comparison of affective EEG-based PI among EEGs from different frequency bands

The experiments assess frequency bands, sparse electrode sets, and model configurations. Deep-learning performance is largely stable across frequency bands, frontal electrodes perform best, and CNN-GRU generally matches or exceeds CNN-LSTM while training faster.

  • Frequency bands: CNN-GRU and CNN-LSTM showed no significant CRR differences across frequency bands, whereas SVM favored Theta and Alpha over higher-frequency and all-band inputs.For CNN-GRU and CNN-LSTM, p=0.092 and p=0.964; SVM reported F(4)=1309.747 with p<0.01.
  • Frequency bands: CNN-GRU and CNN-LSTM significantly outperformed SVM in Beta, Gamma, and all bands, reaching 100% and 99.79% CRR, respectively, in all bands.No approach differences were found in Theta and Alpha.
  • Sparse electrode sets: 99.17 ± 0.34% mean CRR was achieved by CNN-GRU with five frontal electrodes, compared with 98.23 ± 0.52% for CNN-LSTM.The recommended electrodes were F3, F4, Fz, F7, and F8.
  • Sparse electrode sets: Frontal electrodes produced significantly higher CRR than other electrode sets, while occipital and parietal electrodes produced significantly lower CRR.The comparison used five electrode sets with p<0.05.
  • Model comparison: CNN-GRU mean CRR was higher or equal to CNN-LSTM across all tested configurations and significantly higher in one 3-CNN-layer, 16/8-unit configuration.The reported t-test gave p<0.01, and training-speed comparisons used 128, 64, and 32 CNN filters.

V. DISCUSSION

The discussion separates physical EEG factors from algorithmic factors. Deep-learning models maintain high identification performance across affective states and outperform conventional approaches, with CNN-GRU offering faster training and slightly higher CRR than CNN-LSTM.

  • Physical issues: CNN-GRU and CNN-LSTM reached up to 100% mean CRR across different affective states, while SVM using PSD did not reach 50%.SVM improved to up to 98% mean CRR when restricted to Theta or Alpha bands.
  • Physical issues: Deep-learning performance was little affected by frequency band, consistent with the models capturing hidden features beyond frequency-specific information.
  • Algorithmic issues: CNN-GRU and CNN-LSTM outperformed state-of-the-art and relevant algorithms on the same dataset.The comparison included Mahalanobis distance with PSD/COH features, DNN, and SVM.
  • Conclusion: CNN-GRU reached 99.90–100% mean CRR with 32 electrodes and 99.17% with five electrodes.
  • Algorithmic issues: CNN-GRU trained faster and achieved slightly higher mean CRR than CNN-LSTM, especially with a small number of electrodes.
Loading 1807.03147v3…