PaperlayerView exact revision on arXiv

Source-linked AI summary

MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification

Phairot Autthasan, Rattanaphon Chaisaen, Thapanun Sudhawiyangkul, Phurin Rangpong, Suktipol Kiatthaveephong, Nat Dilokthanakul, Gun Bhakdisongkhram, Huy Phan, Cuntai Guan, Theerawit Wilaiprasitporn

arXiv:2102.03814v4eess.SPcs.AIcs.CVcs.LG

TL;DR

Subject-independent MI-EEG classification must handle subject-specific rhythms while reducing calibration demands for new users. MIN2Net integrates deep metric learning into a multi-task autoencoder to learn compact, discriminative EEG representations while classifying signals. It reports improved subject-independent performance across benchmark datasets and supports the practicality of calibration-free MI-BCI use.

  • Problem

    Subject-independent MI-EEG classification requires discriminative features that generalize across subjects so new users can operate BCIs without calibration.

  • Method

    MIN2Net integrates an autoencoder, deep metric learning, and a supervised classifier to compress, discriminate, and classify EEG simultaneously.

  • Results

    MIN2Net outperformed baseline methods in subject-independent experiments on SMR-BCI and OpenBMI, including three-class OpenBMI accuracy of 68.81 ± 12.44% and F1-score of 68.04 ± 12.97%.

  • Takeaways & Limitations

    The findings indicate that MIN2Net can support MI-based BCI applications for new users without calibration while improving discriminative information in latent representations.

Abstract

from arXiv · show

Advances in the motor imagery (MI)-based brain-computer interfaces (BCIs) allow control of several applications by decoding neurophysiological phenomena, which are usually recorded by electroencephalography (EEG) using a non-invasive technique. Despite great advances in MI-based BCI, EEG rhythms are specific to a subject and various changes over time. These issues point to significant challenges to enhance the classification performance, especially in a subject-independent manner. To overcome these challenges, we propose MIN2Net, a novel end-to-end multi-task learning to tackle this task. We integrate deep metric learning into a multi-task autoencoder to learn a compact and discriminative latent representation from EEG and perform classification simultaneously. This approach reduces the complexity in pre-processing, results in significant performance improvement on EEG classification. Experimental results in a subject-independent manner show that MIN2Net outperforms the state-of-the-art techniques, achieving an F1-score improvement of 6.72%, and 2.23% on the SMR-BCI, and OpenBMI datasets, respectively. We demonstrate that MIN2Net improves discriminative information in the latent representation. This study indicates the possibility and practicality of using this model to develop MI-based BCI applications for new users without the need for calibration.

I. INTRODUCTION

MI-based BCIs need subject-independent EEG decoding because subject-dependent systems require burdensome calibration and existing approaches generalize poorly to new users. MIN2Net addresses this challenge with an end-to-end multi-task autoencoder that integrates deep metric learning to preserve discriminative latent information.

  • EEG enables non-invasive BCI communication, while MI produces motor-cortex activity patterns without external stimuli.
  • Subject-independent MI classification aims to let new users operate BCI systems without calibration, but requires discriminative features that generalize across subjects.
  • Deep-learning MI classifiers often perform well only in subject-dependent settings, limiting generalization to new users.
  • MIN2Net combines a multi-task autoencoder with deep metric learning to learn latent representations that preserve discriminative EEG information.It jointly minimizes reconstruction, cross-entropy, and triplet losses.
  • The proposed end-to-end architecture extracts meaningful EEG features without high-complexity preprocessing and targets subject-independent classification.
  • The study visualizes learned latent features to interpret MIN2Net’s classification superiority over state-of-the-art algorithms.

II. RELATED WORK

Prior MI-EEG methods rely on handcrafted or separately trained representations and classifiers, with performance remaining limited for subject-independent use. Related work motivates an end-to-end multi-task approach that combines compression, feature learning, and classification.

  • Conventional MI pipelines use CSP-derived or spectral features with classifiers such as SVM and LDA, but their subject-independent performance still needs improvement.
  • B. Deep Metric Learning Model: Deep metric learning learns representations that measure data similarity by pulling same-class samples together and separating different-class samples.
  • C. Autoencoders: Autoencoders learn latent representations for compression, denoising, dimensionality reduction, and feature extraction, then reconstruct the input.
  • C. Autoencoders: Most prior EEG autoencoder studies used unsupervised feature extraction with classifiers trained separately, rather than end-to-end learning.
  • C. Autoencoders: ERPENet previously showed that a multi-task autoencoder can jointly learn unsupervised reconstruction and supervised classification features for ERP decoding.

III. METHODS

MIN2Net is evaluated with baseline methods on three benchmark MI datasets, using filtered single-trial EEG signals represented by channels, time points, and class labels. Its architecture combines autoencoding, deep metric learning, and supervised learning.

  • Data and evaluation: The study evaluates MIN2Net and baseline methods on the BCIC IV 2a, SMR-BCI, and OpenBMI datasets.BCIC IV 2a and SMR-BCI are described as benchmark datasets, while OpenBMI is identified as the largest public MI dataset.
  • Data and evaluation: The BCIC IV 2a evaluation uses right- and left-hand motor imagery from 20 motor-cortex channels after downsampling from 250 to 100 Hz.The dataset contains 9 healthy subjects and 288 trials collected across two offline sessions.
  • Data and evaluation: The SMR-BCI evaluation uses two-class motor imagery data from 14 subjects, downsampled from 512 to 100 Hz.The selected four-second interval spans 0–4 s after stimulus onset.
  • Data and evaluation: The OpenBMI evaluation uses left- and right-hand motor imagery, selecting 20 motor-cortex electrodes and downsampling signals from 1,000 to 100 Hz.The dataset includes offline and online sessions, with feedback provided in the online condition.
  • MIN2Net design: MIN2Net comprises autoencoder, deep metric learning, and supervised-learning modules operating on filtered single-trial EEG data.The input is x ∈ R^C×T, with C channels, T sampled time points, and labels y ∈ {1, 2, ..., k}.

1) Autoencoder:

The autoencoder compresses EEG into a latent vector and reconstructs the input, while convolutional blocks learn spatial and temporal representations before decoding.

  • Encoder: Two CNN blocks in the encoder combine convolution, batch normalization, ELU activation, and average pooling before fully connected latent mapping.The CNN approach performs channel mixing and convolves along time to extract spatial features from EEG inputs.
  • Autoencoder structure: The autoencoder consists of an encoder, latent vector, and decoder that respectively compress, represent, and reconstruct the input EEG data.The latent vector is produced by the encoder and passed to the decoder for reconstruction.
  • Latent representation: The latent vector is designed to preserve robust EEG-MI features while its size depends on the classification setting.Its reported size is C for 2-class classification and 256 for 3-class classification.
  • Decoder: The decoder mirrors the encoder, using a fully connected layer, reshape operation, and transpose convolutions with stride 4 to upsample the latent representation.Each decoder CNN block uses Conv2DTranspose and ELU layers.
  • Reconstruction objective: The autoencoder is trained with mean squared error between each input channel and its reconstructed signal.The reconstruction loss compares x_j with x̂_j across the C input channels.

2) Deep Metric Learning:

MIN2Net adds triplet-based deep metric learning and supervised softmax classification to the autoencoder, combining reconstruction, metric, and classification objectives.

  • Deep metric learning: The deep metric learning module extends the autoencoder latent vector to preserve distinguishable EEG patterns through a learned distance metric.Triplets contain an anchor, positive, and negative input, which are encoded concurrently into latent vectors.
  • Triplet loss: The triplet objective encourages the anchor-positive distance to be shorter than the anchor-negative distance by a margin α.The hinge term uses [z]+ = max(z, 0), and α controls the required separation.
  • Supervised learning: A supervised classifier feeds the latent vector into a fully connected layer with softmax activation to produce class importance weights.The classifier uses W as the weight matrix and b as the bias vector.
  • Supervised learning: The predicted class is the one with maximum classification probability, and training minimizes cross-entropy loss.The probabilities are produced from the latent representation by the supervised classifier.
  • Joint objective: MIN2Net combines mean-square reconstruction, triplet, and cross-entropy losses with hyperparameters β1, β2, and β3.Integrating the three losses allows unsupervised and metric-learning objectives to influence supervised learning.

E. Network Training

MIN2Net and comparison methods are implemented in a common deep-learning framework, with training controlled by task-specific optimization schedules and early stopping.

  • Implementation: MIN2Net is implemented with Keras using TensorFlow v2.2.0 as the backend and trained on an NVIDIA Tesla V100 GPU with 32GB memory.Adam optimization uses task-dependent learning-rate schedules.
  • Optimization: The learning-rate schedule is [10^-3, 10^-4] for binary classification and [10^-4, 10^-5] for multi-class classification.The schedules are applied during Adam optimization.
  • Comparison methods: Four state-of-the-art methods are implemented for comparison, including FBCSP, Deep Convnet, and EEGNet-8,2.The comparison implementations use Keras and TensorFlow v2.2.0 for deep-learning approaches.
  • Comparison methods: FBCSP extracts features from nine 4-Hz frequency bands spanning 4–40 Hz using four spatial filters and bandpass filtering.Its features are subsequently used with a support vector machine.

4) Spectral-spatial CNN:

The experiments evaluate MIN2Net across subject-dependent and subject-independent settings, using benchmark datasets, cross-validation, parameter search, and ablation comparisons. The ablation removes the triplet module or decoder to assess component contributions.

  • Experimental setup: MIN2Net was evaluated in subject-dependent and subject-independent manners on three benchmark datasets using accuracy and F1-score.The datasets were BCIC IV 2a, SMR-BCI, and OpenBMI.
  • Experimental setup: Subject-dependent evaluation used session-based training and testing splits with stratified 5-fold cross-validation for parameter search.The split differed by dataset, using offline sessions for BCIC IV 2a and offline versus online sessions for SMR-BCI and OpenBMI.
  • Experimental setup: Subject-independent evaluation used leave-one-subject-out cross-validation, training on all data from N_s−1 subjects and testing on the held-out subject.This protocol produced N_s classification results for a dataset with N_s subjects.
  • Parameter adjustment: The study searched loss weights β1, β2, and β3 and examined the margin α in MIN2Net’s deep metric learning module.The β parameters were grid-searched over {0.1, 0.5, 1.0}, while margin values were evaluated for their effect on performance.
  • Ablation study: The ablation study compared complete MIN2Net with variants without the triplet module and without the autoencoder decoder.Repeated-measures ANOVA with Bonferroni correction was used to assess performance differences.

2) Experiment II:

Experiment II compares MIN2Net with ablated variants and visualizes learned features to assess classification effectiveness and discriminative representation quality. The analyses use accuracy and F1-score with standard deviations and statistical testing.

  • Experimental design: Experiment II compares MIN2Net with versions without the triplet module or decoder using binary EEG-MI classification.The comparison covers subject-dependent and subject-independent settings across the considered datasets.
  • Experimental design: The experiment also evaluates a three-class OpenBMI task in pseudo-online conditions to compare MIN2Net with baseline methods.The classes are right-hand MI, left-hand MI, and resting EEG.
  • Feature analysis: Learned features were visualized with two-dimensional t-SNE using embeddings at the input of each model’s final fully connected layer.This visualization compares the discriminative features extracted by different deep learning methods.
  • Evaluation: Results are reported as accuracy ± SD and F1-score ± SD, with statistical analyses used to validate MIN2Net’s effectiveness.The results section covers Experiments I, II, and III and the discriminative power of learned EEG features.
  • Parameter adjustment: A margin value of 1.0 produced the best subject-independent MIN2Net performance for all datasets.The margin parameter significantly affected final classification performance.
  • Ablation results: MIN2Net outperformed both ablated variants on SMR-BCI and OpenBMI in accuracy and F1-score across both evaluation manners.For OpenBMI, the differences were significant with p < 0.05; on BCIC IV 2a, improvements were not found in either manner.

B. Experiment II: Binary MI classification

MIN2Net shows its strongest binary-classification advantages in subject-independent evaluation on SMR-BCI and OpenBMI, while performance varies by dataset and training-set size. Its learned embeddings form more compact and separated class clusters than those of the compared methods.

  • Binary classification results: In subject-independent evaluation, MIN2Net achieved the highest accuracy on OpenBMI and the highest F1-score on both SMR-BCI and OpenBMI.Differences were significant on OpenBMI, while SMR-BCI F1-score gains were significant except against Deep Convnet and Spectal-spatial CNN.
  • Binary classification results: MIN2Net was lower than some baseline methods on BCIC IV 2a in the subject-independent setting, and gains were not found in subject-dependent evaluation.The comparison used accuracy and F1-score across three datasets and multiple methods.
  • Feature visualization: The t-SNE visualization shows MIN2Net embeddings forming compact class clusters while preserving relative distances between clusters.The visualization covers all datasets using trials from one testing subject per dataset.
  • Training-set analysis: Increasing training samples from 100 to 21200 improved binary-classification F1-score across the datasets.With large training sets, MIN2Net’s F1-score exceeded those of the two compared baseline methods.
  • Multi-class classification: In three-class subject-independent OpenBMI classification, MIN2Net reached 68.81 ± 12.44% accuracy and 68.04 ± 12.97% F1-score.Both metrics were significantly different from all baseline methods at p < 0.05.
  • Multi-class feature visualization: For three-class OpenBMI embeddings, MIN2Net produced three compact clusters with separated class representations.The result was observed in comparison with the other baseline methods.

V. DISCUSSION

MIN2Net’s deep metric learning improves the discriminative structure of latent EEG representations and supports strong subject-independent classification with simple preprocessing. Its performance depends on training data availability and loss-weight choices.

  • Effectiveness of Deep Metric Learning: Triplet-loss deep metric learning significantly improves MIN2Net classification over the version without triplet loss across two benchmark datasets.It addresses the autoencoder’s tendency to recognize instances without sufficiently discriminating among classes.
  • Analysis of the Proposed Method: MIN2Net uses filtered time-domain EEG and achieves higher classification results than baseline methods with simplistic preprocessing.The method filters EEG once to reduce high- and low-frequency artifacts.
  • Analysis of the Proposed Method: All four monitored losses converged around 15 epochs during OpenBMI subject-independent training.The monitored losses were MSE, triplet, cross-entropy, and total loss over 60 epochs.
  • Analysis of the Proposed Method: Optimal loss weights vary with dataset size, with smaller weights helping small datasets avoid overfitting and values near 1 favoring large datasets.Equal β values can produce substantial differences on small datasets but only slight differences on large datasets.
  • Comparison Performance: MIN2Net outperforms all baseline methods on SMR-BCI and OpenBMI in subject-independent binary classification, including F1-score when accuracy is lower.The authors attribute this result to generalized EEG features learned through multi-task autoencoding and deep metric learning.
  • Comparison Performance: Subject-dependent performance is suboptimal when training uses few samples from one subject, reflecting an overfitting limitation of the setting.The discussion links improved performance to increasing training samples, including through augmentation.

D. Visualization of the Learned Latent Representation

Visualization shows that MIN2Net produces more discriminative latent EEG features than baseline representations, while experiments support multi-class and calibration-free online BCI use. The authors identify alternative metric losses, other EEG paradigms, and fast adaptation as future directions.

  • Visualization of the Learned Latent Representation: MIN2Net latent features show highly discriminative patterns, whereas baseline embeddings more often overlap across MI classes.This pattern appears in binary subject-independent visualization on SMR-BCI and OpenBMI and in OpenBMI multi-class visualization.
  • Feasibility in online BCI systems: MIN2Net significantly outperforms all baselines in subject-independent three-class OpenBMI classification with an acceptable misclassification rate.The three classes are resting EEG, left-hand MI, and right-hand MI.
  • Feasibility in online BCI systems: MIN2Net’s prediction time for all testing trials is 0.2373 s on BCIC IV 2a, 0.2966 s on SMR-BCI, and 0.1043 s on OpenBMI.The authors frame prediction time as the relevant latency in subject-independent testing.
  • Future Direction: Future work includes replacing triplet loss, applying MIN2Net to SSVEP, MRCP, and ERP, and investigating fast-adaptation transfer learning.These directions are presented as possible improvements and extensions.
  • Conclusion: The study concludes that MIN2Net may support real-world MI-EEG applications, including calibration-free use for new users.The conclusion covers both binary subject-independent results and promising three-class results.

APPENDIX

The appendix reports subject-dependent augmentation experiments and presents a table for comparing MIN2Net’s classification performance across subject-dependent settings.

  • Subject-Dependent Experiments: Data augmentation methods were applied to increase subject-dependent EEG training samples and address suboptimal performance.The methods include jittering, magnitude-warping, scaling, time-warping, and permutation.
  • Subject-Dependent Experiments: Table VIII reports MIN2Net classification accuracy and F1-score, with bold values marking the best numerical results and asterisks marking significant superiority at p < 0.05.The table compares different subject-dependent manners.
Loading 2102.03814v4…