PaperlayerView exact revision on arXiv

Source-linked AI summary

Transfer Learning for EEG-Based Brain-Computer Interfaces: A Review of Progress Made Since 2016

Dongrui Wu, Yifan Xu, Bao-Liang Lu

arXiv:2004.06286v4cs.HCcs.LGeess.SP

TL;DR

EEG-based BCIs require burdensome calibration because noisy, non-stationary signals vary across subjects, sessions, devices, and tasks. This paper reviews transfer-learning approaches published since 2016 across six applications and three transfer settings. The review organizes recent methods and identifies coverage patterns and future research directions, including a reported 90% calibration-time reduction in one driver-drowsiness approach.

  • Problem

    EEG variability and noise make universal BCI models difficult to build, so new subjects typically require time-consuming calibration.

  • Method

    The paper reviews journal publications on EEG-based BCI transfer learning since 2016, grouping approaches across six applications and cross-subject/session, cross-device, and cross-task settings.

  • Results

    The review finds that steady-state visual evoked potentials receive comparatively little recent transfer-learning attention, while affective BCIs and regression problems attract increasing interest.

  • Takeaways & Limitations

    Transfer learning is a broad strategy for reducing calibration effort by using relevant data or knowledge from other subjects, sessions, devices, or tasks.

Abstract

from arXiv · show

A brain-computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common non-invasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject non-stationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time-consuming and user unfriendly. Transfer learning (TL), which utilizes data or knowledge from similar or relevant subjects/sessions/devices/tasks to facilitate learning for a new subject/session/device/task, is frequently used to reduce the amount of calibration effort. This paper reviews journal publications on TL approaches in EEG-based BCIs in the last few years, i.e., since 2016. Six paradigms and applications -- motor imagery, event-related potentials, steady-state visual evoked potentials, affective BCIs, regression problems, and adversarial attacks -- are considered. For each paradigm/application, we group the TL approaches into cross-subject/session, cross-device, and cross-task settings and review them separately. Observations and conclusions are made at the end of the paper, which may point to future research directions.

I. INTRODUCTION

EEG-based BCIs are difficult to generalize across subjects, sessions, devices, and tasks because EEG is noisy and non-stationary, making calibration burdensome. This review addresses prior coverage gaps by surveying transfer-learning approaches since 2016 across six BCI paradigms and applications and three transfer settings.

  • Motivation: EEG signals are weak, noisy, and non-stationary across subjects and sessions, so universal BCI models are difficult to build.New subjects usually require calibration data, which is time-consuming and user unfriendly.
  • Motivation: Transfer learning uses data or knowledge from related subjects, sessions, devices, or tasks to facilitate learning for a new target.It can also be integrated with active learning for better performance.
  • Research gap: Previous reviews mainly considered cross-subject and cross-session transfer for motor imagery, event-related potentials, and steady-state visual evoked potentials.They did not cover more challenging cross-device and cross-task transfers, affective BCIs, regression, or adversarial attacks.
  • Scope: The review covers motor imagery, event-related potentials, steady-state visual evoked potentials, affective BCIs, regression problems, and adversarial attacks.These applications extend beyond the three classic classification paradigms.
  • Review organization: For each paradigm or application, the paper separately reviews cross-subject/session, cross-device, and cross-task transfer approaches.Approaches spanning multiple categories are introduced under the more challenging category.

II. TL CONCEPTS AND SCENARIOS

The paper frames transfer learning through domains and tasks that may differ in feature spaces, marginal distributions, label spaces, or conditional prediction distributions. It distinguishes inductive, transductive, and domain-adaptation settings according to available labels and distributional assumptions.

  • Domains: A domain consists of a feature space and its associated marginal probability distribution.Domains differ when their feature spaces or marginal feature distributions differ.
  • Tasks: A task consists of a label space and a prediction function, represented as the conditional probability distribution P(y|X).Tasks differ when their label spaces or conditional distributions differ.
  • Transfer learning: Transfer learning seeks a target prediction function with low expected error despite differences between source and target domains or tasks.The general assumptions allow differences in feature spaces, label spaces, marginal distributions, and conditional distributions.
  • Transfer settings: Inductive transfer learning has labelled target samples, whereas transductive transfer learning has unlabelled target samples and labelled source samples.In most inductive BCI scenarios, source samples are labelled, though they may also be unlabelled.
  • Domain adaptation: Domain adaptation is a special case of transductive transfer learning in which source and target feature spaces and label spaces are shared.Their marginal or conditional distributions may nevertheless differ.
  • Covariate shift: Covariate shift is a simpler domain-adaptation case where feature and label spaces and conditional distributions match, but marginal feature distributions differ.The defining distributional difference is Ps(X) ≠ Pt(X).

B. TL Scenarios

EEG transfer scenarios vary according to whether source data come from other subjects or sessions, devices, or tasks, and these scenarios can be combined. The paper also establishes notation for EEG trials and their covariance matrices in motor-imagery analyses.

  • Cross-subject/session TL: Cross-subject transfer uses other subjects’ data to reduce calibration for a new subject, usually with the same task and EEG device.Cross-session transfer similarly uses previous sessions to calibrate a new session.
  • Cross-device TL: Cross-device transfer uses data from one EEG device to facilitate calibration for another, usually with the same subject and task.The source and target devices define the transfer boundary.
  • Cross-task TL: Cross-task transfer uses labelled data from similar or relevant tasks to facilitate calibration for a new task, usually with the same subject and EEG device.Examples include transferring between hand and feet or tongue motor-imagery tasks.
  • Mixed scenarios: Simple transfer scenarios can be combined into more complex settings, including cross-subject plus cross-device or cross-task transfer.The paper treats these as mixed transfer scenarios.
  • MI notation: For motor-imagery analyses, each EEG trial is represented as an E × T matrix, where E is the number of electrodes and T is the number of time samples per channel.The corresponding covariance matrix is an E × E symmetric positive-definite matrix on a Riemannian manifold.

A. Cross-Subject/Session Transfer

Cross-subject/session transfer methods align EEG representations, share subject-invariant parameters, or adapt classifiers using limited target calibration data. The reviewed approaches include Riemannian, Euclidean, probabilistic, ensemble, and kernel-based strategies.

  • Model-based approaches: TKCSP integrates transfer kernel learning with kernel common spatial patterns to learn domain-invariant spatial filters for cross-subject MI classification.The method first computes a domain-invariant kernel, then identifies components with the largest energy difference between classes.
  • Model-based approaches: Other cross-subject/session methods share parameters across source subjects, weight source contributions, or combine transfer learning with active learning and fine-tuning.Shared parameters represent subject-invariant stimulus-prediction characteristics, while fine-tuning a dense layer on limited target calibration data improved cross-subject transfer.
  • Alignment-based approaches: Riemannian alignment centres EEG covariance matrices at the identity to reduce inter-subject/session variation in MI and ERP classification.Resting trials provide the reference state for MI, while ERP alignment uses non-target stimuli and requires labelled target trials.
  • Alignment-based approaches: Parallel transport maps covariance matrices from different domains into a common tangent space so source-trained classifiers can be applied directly to target data.The approach was demonstrated in cross-subject MI, sleep-stage, and mental-arithmetic classification.
  • Alignment-based approaches: EA aligns EEG trials in Euclidean space by transforming each domain with a reference matrix, after which Euclidean or Riemannian classifiers can be used.The reference matrix can use the arithmetic mean for greater flexibility and speed, or the more computationally intensive Riemannian mean.
  • Alignment-based approaches: EA outperformed RA in MI and ERP classifications in both offline and simulated online applications.This comparison supports EA as a flexible alignment choice across both evaluation settings.
  • Alignment-based approaches: RPA matches covariance distributions across domains through translation, scaling, and rotation before training a classifier on combined transformed data.RPA is semi-supervised, requiring at least one labelled sample from each target-domain class, and achieved promising results across MI, ERP, and SSVEP classification.

B. Cross-Device TL

Cross-dataset transfer studies evaluate deep learning when EEG devices, channels, and tasks differ. A common-channel selection and online Euclidean pre-alignment strategy was applied across eight public MI datasets.

  • Cross-device TL: Eight publicly available MI datasets were used to study deep learning in cross-dataset transfer despite differences in devices, channels, and MI tasks.The analysis retained only channels C3, CZ, and C4 and left-hand and right-hand MI tasks.
  • Cross-device TL: Online pre-alignment recursively computed each subject’s Riemannian mean and used it as the reference matrix in EA for every EEG trial.This provided the alignment step before applying the deep-learning transfer analysis.

C. Cross-Task TL

Cross-task transfer addresses cases where source and target tasks have different label spaces, beyond the assumptions of RA and EA. LA aligns source information to target classes and is described as the only reported cross-task EEG-BCI TL work in this review.

  • C. Cross-Task TL: RA and EA assume identical source and target feature and label spaces, so they may not apply to cross-task transfers.Cross-task settings can involve different label spaces even when EEG data are otherwise related.
  • C. Cross-Task TL: LA selects source channels similar to target channels, estimates class means, and re-centres each source class at the corresponding target class mean.Aligned trials can then be processed with Euclidean or Riemannian feature extraction and classification.
  • C. Cross-Task TL: LA requires as few as one labelled sample from each target-domain class and can precede different feature extraction and classification algorithms.The method is presented as applicable to simultaneous cross-subject, cross-device, and cross-task transfer.
  • C. Cross-Task TL: The paper identifies LA as the only cross-task TL work in EEG-based BCIs and the most complicated TL scenario considered in the literature so far.The stated scenario combines cross-subject, cross-device, and cross-task transfer.

IV. TL IN ERP-BASED BCIS

The ERP section reviews transfer-learning approaches for ERP-based BCIs while avoiding duplication with methods introduced earlier. It reports no cross-task ERP TL publications and therefore omits a cross-task subsection.

  • IV. TL IN ERP-BASED BCIS: The ERP section reviews recent ERP-based BCI TL approaches and excludes methods already introduced in the preceding section.Examples such as RA, EA, RPA, and EEGNet are mentioned as reusable approaches but are not repeated.
  • IV. TL IN ERP-BASED BCIS: No publications on cross-task TL in ERP-based BCIs were identified, so the review contains no cross-task subsection for ERP.The omission is explicitly attributed to the absence of publications.

A. Cross-Subject/Session TL

Cross-subject/session transfer methods reduce dependence on target-user calibration by combining source and limited target information across ERP and related settings.

  • ERP-based BCIs: STIG combines predictions from subject-specific MDRM classifiers for subject-independent ERP classification.It outperformed some calibration-free and traditional calibration approaches when limited data were available.
  • ERP-based BCIs: wAR directly learns a classifier using weighted source and target fitting losses, structural regularization, and marginal and conditional distribution distances.The sample weights balance positive and negative classes across domains.
  • ERP-based BCIs: Riemannian-distance selection identifies source ERP epochs most similar to a small target reference set for P300 classification.The selected samples train the classifier applied to the target subject.
  • ERP-based BCIs: A generic P300 model set clusters 116 participants into 10 groups, then matches a new subject’s online model to the closest cluster model.PCA reduces time-domain feature dimensionality before k-means clustering and WLDA training.
  • Scope across paradigms: Cross-subject/session transfer also spans SSVEP and affective settings, while SSVEP reviews report fewer studies than MI and ERP.The supplied passages describe calibration-free Compact-CNN, cross-device AwAR, and the absence of SSVEP cross-task publications.

A. Cross-Subject/Session TL

Cross-subject/session transfer in SSVEP and affective BCIs uses calibration-free models, feature or subspace alignment, and selective source information to address subject and session variability.

  • SSVEP-based BCIs: Compact-CNN performed 12-class SSVEP classification without user-specific calibration and outperformed hand-crafted CCA and Combined-CCA approaches.
  • SSVEP-based BCIs: Cross-device SSVEP transfer constructs spatial filters and combines source-trial Pearson correlations with canonical correlations to identify the target class.The method compares a target trial with filtered source class averages and computer-generated SSVEP models.
  • Affective BCIs: Affective-BCI literature uses both categorical emotion labels and continuous arousal-valence representations, but existing work focused almost exclusively on classification.
  • Affective BCIs: ASFM aligns source and target PCA subspaces, reduces marginal-distribution discrepancy, and refines pseudo-labels for target classification.
  • Affective BCIs: Other affective-BCI approaches include conditional transfer, robust-PCA filtering, prototype-based source mapping, and subspace-alignment autoencoders.These methods address feature relevance, deviant signal events, source selection, and distribution discrepancy across subjects or sessions.
  • Affective BCIs: Cross-day EEG variability can dramatically impair emotion classification, while training with data from more days significantly improves generalization.
  • Affective BCIs: Adaptive stacked denoising autoencoders update shallow hidden-layer weights during testing using augmented samples and pseudo-labels for cross-session mental-workload classification.

B. Cross-Device TL

Cross-device transfer addresses heterogeneity in datasets, electrode layouts, and signal modalities by aligning shared information or converting EEG into device-consistent representations.

  • Dataset and device heterogeneity: DEAP-to-SEED transfer uses only 32 common channels after accounting for different subjects, devices, and electrode counts.The study selected three trials from 14 DEAP subjects and extracted differential-entropy features across five frequency bands.
  • Dataset and device heterogeneity: RPCA-embedded transfer selects similar source subjects by Riemannian distance and combines their trials with target trials to train a personalized cross-day SVM.
  • Cross-modality transfer: Eye-tracking data can enhance cross-subject EEG affect classification through a three-step distribution-to-classifier mapping.The heterogeneous method achieved comparable performance with homogeneous EEG-based and scanpath-based models.
  • Deep cross-device transfer: Deep-learning approaches represent EEG as images or topographies to accommodate devices with different electrode placements, sampling rates, and channel counts.One approach uses theta-, alpha-, and beta-band PSD topographies; another pads channels with noisy copies for a fixed CNN input size.

VII. TL IN BCI REGRESSION PROBLEMS

The review extends transfer-learning coverage to EEG regression problems, including drowsiness, vigilance, and reaction-time estimation, which earlier reviews did not adequately address.

  • Scope of regression problems: EEG regression applications include driver drowsiness, vigilance, and user reaction-time estimation.
  • Scope of regression problems: No publications addressed cross-device or cross-task transfer for BCI regression problems in this review.

A. Cross-Subject/Session TL

Cross-subject/session transfer learning methods reduce calibration demands in EEG-based BCI regression and classification, while adversarial-attack work examines how perturbations target model components and test inputs.

  • A. Cross-Subject/Session TL: OwARR combines weighted adaptation regularization with source-domain selection to reduce subject-specific calibration and approximately halve computational cost.Its loss includes source and target fitting, marginal and conditional distribution distances, and a sample Pearson-correlation term.
  • A. Cross-Subject/Session TL: Cross-subject driver drowsiness estimation can select and combine source models by predicting their transferability from distribution-distance measures.The procedure ranks sources, fuses their models with learned weights, and applies target-specific recalibration.
  • A. Cross-Subject/Session TL: Feature selection and adaptation regularization are combined in ARTL classification, selecting features that balance class separability with domain similarity.CSDF-ReliefF and CSDF-mRMR extend feature-selection criteria to incorporate both objectives before ARTL classification.
  • A. Cross-Subject/Session TL: FWET uses feature weighting and episodic training to learn source-domain representations that generalize across subjects without calibration.It trains transformations to remain effective when applied to regressors learned from other source subjects.
  • A. Cross-Subject/Session TL: EEG adversarial-attack studies focus on test-input adversarial examples, generated by perturbations that can degrade classifier and regression-model performance.Attacks may target multiple machine-learning components, but EEG-based BCI research reviewed here investigates test inputs.

A. Cross-Model Attacks

Adversarial-attack transfer in EEG-based BCIs mainly concerns cross-model attacks, where adversarial examples generated for one model are applied to another. Existing work spans multiple attack settings and BCI paradigms, while transfer learning across domains may further improve black-box attacks.

  • Cross-model attacks: Cross-model attacks transfer adversarial examples generated from one machine-learning model to attack another model.This transferability is used even when transfer learning is not explicitly considered.
  • Cross-model attacks: Cross-model attacks are especially relevant to grey-box and black-box settings because the victim model is unknown and requires a substitute model.The substitute model approximates the victim model for attack construction.
  • Cross-model attacks: Unsupervised fast gradient sign methods have been demonstrated for white-box, grey-box, and black-box attacks across MI and ERP using EEGNet, Deep ConvNet, and Shallow ConvNet.The reported study spans two BCI paradigms and three deep-learning models.
  • Cross-model attacks: Cross-subject attacks use one model across subjects while designing adversarial examples on some subjects and applying them to another.This meaning differs from cross-subject transfer learning in the paper’s other sections.
  • Cross-model attacks: Transfer learning across domains may improve black-box attacks by using public datasets to reduce victim-model queries or improve approximation with the same query budget.The paper identifies this as a possible direction beyond existing cross-model transferability.
Loading 2004.06286v4…