Classification of Hand Movements from EEG using a Deep Attention-based LSTM Network

Guangyi Zhang; Vandad Davoodnia; Alireza Sepas-Moghaddam; Yaoxue Zhang; Ali Etemad

Source-linked AI summary

Classification of Hand Movements from EEG using a Deep Attention-based LSTM Network

Guangyi Zhang, Vandad Davoodnia, Alireza Sepas-Moghaddam, Yaoxue Zhang, Ali Etemad

arXiv:1908.02252v2Aug 6, 2019cs.LGeess.SPstat.ML

TL;DR

The paper addresses EEG-based left/right hand-movement classification, particularly the challenge of generalizing across subjects. It extracts time- and frequency-domain features and uses an attention-based LSTM, reporting improved performance over benchmarks in both intra-subject and cross-subject evaluations.

Problem
Cross-subject EEG hand-movement classification is challenging because brain activity is highly subject-dependent, while intra-subject methods do not provide generalized off-the-shelf solutions.
Method
The method preprocesses EEG, extracts time- and frequency-domain features, and feeds them to an attention-based LSTM that learns temporal information.
Results
The proposed model significantly outperforms the best cross-subject benchmark by 5% accuracy and achieves near-perfect intra-subject performance compared with a previous best accuracy of 88.6%.
Takeaways & Limitations
The framework supports both hand-movement classification across subjects and analysis of sensor-level EEG information associated with movement-related brain activity.

Abstract

from arXiv · show

Classifying limb movements using brain activity is an important task in Brain-computer Interfaces (BCI) that has been successfully used in multiple application domains, ranging from human-computer interaction to medical and biomedical applications. This paper proposes a novel solution for classification of left/right hand movement by exploiting a Long Short-Term Memory (LSTM) network with attention mechanism to learn the electroencephalogram (EEG) time-series information. To this end, a wide range of time and frequency domain features are extracted from the EEG signals and used to train an LSTM network to perform the classification task. We conduct extensive experiments with the EEG Movement dataset and show that our proposed solution our method achieves improvements over several benchmarks and state-of-the-art methods in both intra-subject and cross-subject validation schemes. Moreover, we utilize the proposed framework to analyze the information as received by the sensors and monitor the activated regions of the brain by tracking EEG topography throughout the experiments.

I. INTRODUCTION

EEG-based hand-movement classification supports BCI applications, but cross-subject generalization remains more challenging than subject-specific classification. The paper addresses this challenge with an attention-based LSTM evaluated under both validation schemes.

I. INTRODUCTION: Cross-subject hand-movement classification is more desirable for off-the-shelf BCI systems but typically achieves lower accuracy than intra-subject approaches.Prior work has often focused on intra-subject validation, while cross-subject validation has been comparatively neglected.
I. INTRODUCTION: The proposed attention-based LSTM classifies left/right hand movements from EEG using artifact reduction, time- and frequency-domain features, and temporal attention.The model is evaluated with both intra-subject and cross-subject validation and compared with benchmarks and state-of-the-art methods.
I. INTRODUCTION: The model is trained on data from 103 subjects with 10-fold cross-subject validation and reportedly outperforms state-of-the-art hand-movement classification solutions.The same network is also evaluated separately for each subject in the intra-subject setting.
I. INTRODUCTION: The framework also analyzes EEG information flow across sensors to relate movement classification with brain activity during stimulus perception and hand movement.The authors report correspondence between sensor-pair information flow and expected neurological function.
I. INTRODUCTION: Prior studies predominantly used intra-subject validation and classical machine-learning models, leaving cross-subject generalization and deep-learning approaches less explored.Reported intra-subject results include 64.02% average accuracy with CSP and 88.69% with a QD-based method, while cross-subject performance could fall to chance level in one prior study.

III. PROPOSED METHOD

The proposed method section introduces the preprocessing, feature-extraction, and deep-learning components used for EEG hand-movement classification.

III. PROPOSED METHOD: The proposed method comprises preprocessing, time- and frequency-domain feature extraction, and a deep-learning solution for EEG classification.The section also establishes notation for scalars, vectors, and matrices.

A. Pre-processing

The preprocessing pipeline removes selected artifacts and represents each EEG segment with time- and frequency-domain features before classification.

A. Pre-processing: Artifact reduction uses selected differential EEG channels, a 50 Hz notch filter, and a 0.5–70 Hz band-pass filter.Ten of the 64 available channels were discarded because of their non-symmetric nature, leaving 27 differential channels.
A. Pre-processing: Each trial is divided into 2-second segments, from which features are extracted at each time-step.The effect of segment size is evaluated separately in the experiments.
A. Pre-processing: The feature representation combines seven time-domain statistics with relative power in four frequency bands.The frequency bands are delta, theta, alpha, and beta; 297 features are extracted from each time-step across 27 channels.

C. Proposed Deep Learning Solution

The deep-learning solution uses LSTM memory to model EEG dependencies over time and attention to emphasize the most discriminative time-steps.

C. Proposed Deep Learning Solution: The model is motivated by the need to remember and aggregate subtle spatio-temporal changes associated with intended movements.The LSTM-attention combination is selected to address both temporal memory and discriminative time-step weighting.
C. Proposed Deep Learning Solution: LSTM is used to capture both long- and short-term dependencies in the sequential EEG feature stream.The architecture uses a cell state and input, forget, and output gates to control information flow.
C. Proposed Deep Learning Solution: The LSTM cell updates its cell state from the previous state and current input, then derives the hidden state through the output gate.The equations define cell-state and hidden-state updates using trainable weights, biases, and nonlinearities.

2) Attention Mechanism:

The attention mechanism weights LSTM hidden states to emphasize time steps containing the most discriminative EEG information. The resulting attention-based LSTM feeds an attended representation to a classifier for left/right hand-movement prediction.

2) Attention Mechanism:: Attention weights emphasize LSTM time steps containing the most discriminative task-related EEG information.Unlike a conventional LSTM that uses only its last hidden state, the attention mechanism applies trainable weights to the hidden-state outputs.
2) Attention Mechanism:: The attention output is formed from weighted LSTM hidden states, with trainable parameters defining the attention transformation.The hidden state h_i corresponds to the i-th LSTM input, while v is the attention layer’s output and W_s and b_s are trainable parameters.
2) Attention Mechanism:: The proposed architecture stacks three seven-cell LSTM layers, followed by attention and a sigmoid-activated fully connected classifier.Each segment contributes 297 features across seven time steps, which are processed by the recurrent layers before class-probability prediction.

3) Proposed Network:

This section identifies the test material, hyper-parameter selection, validation protocols, and benchmark solutions used to evaluate the proposed method.

3) Proposed Network:: Evaluation covers test material, LSTM hyper-parameter tuning, validation protocols, and state-of-the-art and benchmark recognition solutions.

A. Dataset

The study uses a filtered 64-channel EEG movement dataset with left- and right-hand actions, segmented into overlapping time windows for LSTM classification. Performance is evaluated with separate intra-subject and cross-subject 10-fold protocols using precision, recall, and accuracy.

A. Dataset: The EEG Movement Dataset contains 109 subjects, with six low-signal-to-noise subjects removed before analysis.The resulting experiments use data from 103 subjects.
A. Dataset: Each movement segment is represented as a 2-second LSTM sequence with seven time steps and 50% overlap between adjacent windows.
A. Dataset: Both intra-subject and cross-subject evaluation use 10-fold cross-validation without overlap between training and testing segments.The protocol is intended to avoid inflated performance associated with overlapping evaluation segments.
A. Dataset: Precision, recall, and accuracy are calculated from true-positive, false-negative, false-positive, and true-negative counts.

1) Solutions for Cross-Subject Scheme:

The cross-subject benchmarking compares the proposed approach with prior methods and conventional machine-learning classifiers to assess generalization across users.

1) Solutions for Cross-Subject Scheme:: Cross-subject benchmarking includes PLV, ANN, SVM, Naïve Bayes, decision tree, logistic regression, and random forest solutions.The additional classical and deep-learning methods were implemented to strengthen comparison on the challenging dataset.

2) Solutions for Intra-Subject Scheme:

The section compares the proposed method with prior solutions for hand-movement classification, emphasizing intra-subject evaluation and related benchmark schemes.

2) Solutions for Intra-Subject Scheme:: The experiments distinguish intra-subject evaluation from the more challenging cross-subject generalization setting.The paper states that its main goal is cross-subject generalization while also performing comparisons under intra-subject validation.
2) Solutions for Intra-Subject Scheme:: The reported evaluations use tables covering segment-size effects and method comparisons under cross-subject and intra-subject schemes.The supplied table captions identify these evaluation scopes but do not provide the underlying cell values.
2) Solutions for Intra-Subject Scheme:: The evaluation compares the proposed method with CSP, QDA, rough set-based methods, and MDA from previous studies.These methods are identified as the principal comparison solutions in the supplied passages.

V. RESULTS AND DISCUSSION

The experiments select a 2-second feature-extraction segment and show strong classification performance across validation schemes, while also tracing feature importance and time-varying sensor activity.

Effect of Segment Size: A 2-second segment achieves the highest classification accuracy and minimum standard deviation among the tested segment sizes.The tested sizes range from 0.25 to 2.0 seconds, and 2 seconds is selected for feature extraction and classification.
Performance: 5% accuracy separates the proposed model from the best cross-subject benchmark, while intra-subject performance is described as near-perfect.The cross-subject comparison uses Table V, and the intra-subject comparison uses Table VI; prior intra-subject work reached 88.6% accuracy.
Performance: The proposed approach reaches an AUC of 0.908 and is reported as superior to the top three cross-subject benchmarks.Figure 4 presents ROC curves and corresponding AUCs for the proposed network and the three benchmarking solutions.
Discussion: Random Forest ranks skewness, mean, and area from the F7-F8 sensor pair among the three top features, followed by dominant frontal-temporal and temporal sensor pairs.The analysis selects the top 30 significant subject-independent features for subsequent sensor-flow analysis.
Information Flow: At stimulus onset, anterior-frontal, parietal-occipital, and occipital sensor pairs display the strongest features, consistent with visual-cortex involvement.The cited passages associate these regions with receiving and processing visual stimuli.
Information Flow: At 0.75s, temporal, frontal, and anterior-frontal sensor pairs have the highest discriminability, which remains concentrated in temporal and frontal regions through movement completion.Central-parietal pairs become more informative at 0.5s, whereas occipital pairs contribute little from 0.75s to 2.0s.

VI. SUMMARY AND FUTURE WORK

The proposed attention-based LSTM uses time- and frequency-domain EEG features to classify left/right hand movements, achieving effective results across intra-subject and cross-subject evaluations. Future work targets earlier prediction, automated feature extraction, and improved cross-subject adaptation.

VI. SUMMARY AND FUTURE WORK: The method achieved high performance and generalization across intra-subject and cross-subject validation on 103 subjects, outperforming benchmark and state-of-the-art methods.The framework also analyzed EEG information flow through sensors over time to reflect brain activity during the experiment.
VI. SUMMARY AND FUTURE WORK: The approach reduces signal artifacts, extracts time- and frequency-domain features, and feeds them into an attention-based LSTM network.The model was evaluated after studying optimal LSTM hyper-parameter settings.
VI. SUMMARY AND FUTURE WORK: Future work will develop deep models for early detection or prediction of hand movements rather than classification.
VI. SUMMARY AND FUTURE WORK: The use of hand-crafted features is identified as a limitation, motivating future CNN-based feature extraction for simpler and more robust solutions.
VI. SUMMARY AND FUTURE WORK: Future cross-subject work will use domain adaptation, including Wasserstein Generative Adversarial Network Domain Adaptation, to minimize differences among subjects.