PaperlayerView exact revision on arXiv

Source-linked AI summary

Transfer Learning for Motor Imagery Based Brain-Computer Interfaces: A Complete Pipeline

Dongrui Wu, Xue Jiang, Ruimin Peng, Wanzeng Kong, Jian Huang, Zhigang Zeng

arXiv:2007.03746v3eess.SPcs.HCcs.LGstat.ML

TL;DR

Long calibration for new MI-BCI subjects motivates transfer learning, but prior approaches typically used it in only one or two components. The paper proposes a pipeline with alignment and transfer across three components; experiments report improved classification performance from broader and more sophisticated transfer.

  • Problem

    New MI-BCI subjects can require long calibration, while prior transfer-learning approaches usually addressed only one or two system components.

    Source: §1 · p. 2, §5 · p. 23

  • Method

    The paper proposes adding data alignment before spatial filtering and considering transfer learning in spatial filtering, feature engineering, and classification.

    Source: §1 · p. 3

  • Results

    Generally, transfer learning in different pipeline components improved classification over using none, more sophisticated approaches outperformed simple ones, and integrating components could further improve performance.

    Source: §5 · p. 27

  • Takeaways & Limitations

    The authors identify data alignment as important preprocessing and report that transfer learning across components can be complementary.

    Source: §5 · p. 27

    The paper considers only binary MI classification problems in BCIs.

    Source: §5 · p. 27

Abstract

from arXiv · show

Transfer learning (TL) has been widely used in motor imagery (MI) based brain-computer interfaces (BCIs) to reduce the calibration effort for a new subject, and demonstrated promising performance. While a closed-loop MI-based BCI system, after electroencephalogram (EEG) signal acquisition and temporal filtering, includes spatial filtering, feature engineering, and classification blocks before sending out the control signal to an external device, previous approaches only considered TL in one or two such components. This paper proposes that TL could be considered in all three components (spatial filtering, feature engineering, and classification) of MI-based BCIs. Furthermore, it is also very important to specifically add a data alignment component before spatial filtering to make the data from different subjects more consistent, and hence to facilitate subsequential TL. Offline calibration experiments on two MI datasets verified our proposal. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.

1. Introduction

Earlier MI-based BCI transfer learning often addressed only one or two system components. This paper proposes a complete pipeline spanning data alignment, spatial filtering, feature engineering, and classification, and evaluates it on two MI datasets.

1. Introduction

  • A closed-loop EEG-based BCI processes signals through spatial filtering, feature engineering, and classification before a controller sends a command to an external device.The system also includes signal acquisition and temporal filtering.Source: §1 · p. 2
  • Long calibration sessions for new subjects, ranging from 20–30 minutes to hours or days, reduce the utility of MI-based BCIs.The paper relates this need to individual differences and EEG non-stationarity.Source: §1 · p. 2
  • Previous BCI transfer-learning approaches usually addressed only one or two components, particularly classification.Examples include transfer methods for spatial filtering and feature selection, sometimes combined with a transfer-learning classifier.Source: §1 · p. 3
  • Across two MI datasets, using TL in all components achieved better classification performance than using none or only a subset.The experiments compare the complete approach with approaches that omit TL or apply it in fewer components.Source: §1 · p. 4
  • The proposed pipeline adds data alignment before spatial filtering and applies TL to spatial filtering, feature engineering, and classification.Alignment makes source- and target-domain data more consistent; the authors describe it as important across traditional and deep learning, online and offline, and cross-subject and cross-session settings.Source: §1 · pp. 3–4

2. TL Approaches

The section frames transfer learning around using labeled EEG trials from source subjects to help classify a target subject’s trials. It introduces the offline binary setting and illustrates how source data can supplement sparse target training data.

2. TL Approaches

  • The setup considers offline binary classification, using labeled trials from a source subject to help classify a target subject’s trials; multiple sources can be combined or handled separately.The paper assumes source and target share the feature and label spaces but may differ in their distributions.Source: §2 · p. 4, §2.1 · p. 5

2.1. TL

  • For BCI calibration, transfer learning uses labeled EEG trials from an existing subject to help calibrate a new subject.The considered scenario has the same headset and MI types, but different marginal and conditional distributions.Source: §2.1 · p. 5
  • With only four labeled target samples, the simple classifier example combines source and target samples for training and may give source samples smaller weights.The source samples can be downweighted because the two domains may not be fully consistent.Source: §2.1 · p. 5
  • The illustrated simple classification approach may also be adapted to spatial filtering and feature engineering.Source: §2.1 · p. 5

2.2. Euclidean Alignment (EA)

Euclidean alignment (EA) addresses subject-to-subject differences in EEG trial distributions by aligning source and target data before later processing. After alignment, each subject’s trials are whitened and have an identity mean spatial covariance matrix.

2.2. Euclidean Alignment (EA)

  • EA makes EEG trials from different subjects more consistent by aligning their distributions before subsequent operations.The paper motivates alignment by individual differences in marginal trial distributions.Source: §2.2 · pp. 5–6
  • EA is completely unsupervised, requiring no labeled data from any subject.It is used in the paper as an example alignment approach.Source: §2.2 · p. 5
  • EA computes covariance means separately for the source and target subjects before aligning their EEG trials.For the target, the mean uses all Nl + Nu spatial covariance matrices.Source: §2.2 · p. 6, Figure 4 · p. 6

2.3. Pre-alignment Strategy (PS)

The pre-alignment strategy (PS) aligns EEG trials using a Riemannian mean rather than the Euclidean mean used by EA. It is unsupervised like EA, but costs more computationally because its mean requires iterative computation.

2.3. Pre-alignment Strategy (PS)

  • PS uses a Riemannian mean, defined as the matrix minimizing the sum of squared Riemannian distances.Riemannian distance is the minimum curve length connecting two symmetric positive-definite covariance matrices.Source: §2.3 · p. 6
  • PS aligns EEG trials directly and is completely unsupervised, like EA.Its formula is essentially identical to EA’s except for the mean used.Source: §2.3 · pp. 6–7
  • PS has higher computational cost than EA because its Riemannian mean has no closed-form solution and is computed iteratively.The described computation uses an iterative gradient descent algorithm.Source: §2.3 · pp. 6–7

2.4. CSP

CSP is supervised spatial filtering that learns filters to maximize the variance ratio between two classes. The resulting spatially filtered trials are represented by log-variance features.

2.4. CSP

  • CSP performs supervised spatial filtering to maximize the variance ratio between the two classes.Traditional CSP uses target-subject data only.Source: §2.4 · p. 7
  • CSP constructs 2f spatial filters by concatenating the f filters obtained for each class.The filters are derived from class mean spatial covariance matrices.Source: §2.4 · p. 7
  • CSP applies the filters to the EEG trials and extracts their log-variances as features.The paper identifies log as the logarithm operator.Source: §2.4 · p. 7

2.5. Combined CSP (CCSP)

CCSP combines labeled source- and target-subject samples to design spatial filters, while RCSP regularizes the class covariance estimates. RCSP includes traditional CSP and CCSP as parameter settings.

2.5. Combined CSP (CCSP)

  • CCSP concatenates labeled source and target samples to compute spatial filters, weighting all samples equally.It is motivated by the unreliability of filters estimated from the target subject’s few labeled samples.Source: §2.5 · p. 7

2.6. Regularized CSP (RCSP)

  • RCSP regularizes each class’s average spatial covariance matrix using parameters β and γ.Both parameters lie in [0, 1], and I is the identity matrix.Source: §2.6 · p. 8
  • RCSP becomes traditional CSP when β = 1 and γ = 0, and becomes CCSP when β = 0.5 and γ = 0.CCSP is described as a simple TL-based CSP approach.Source: §2.5 · p. 7, §2.6 · p. 8

2.7. ReliefF

ReliefF ranks features by comparing how well each distinguishes samples from different classes versus the same class. It selects features with the largest resulting importance weights.

2.7. ReliefF

  • ReliefF repeatedly samples x, finds k nearest same-class neighbors H and different-class neighbors M, and updates each feature's importance weight.The paper sets k = 10.Source: §2.7 · p. 8
  • Feature importance decreases with same-class discrimination and increases with different-class discrimination.The update contrasts the feature-wise differences between a sample and its same-class neighbors H and different-class neighbors M.Source: §2.7 · p. 8
  • After 100 iterations, ReliefF ranks feature weights and selects features with the largest values.Each iteration uses a randomly selected sample x.Source: §2.7 · p. 8

2.8. Combined ReliefF (CReliefF)

This section describes transfer-learning approaches for feature selection and classification, from combining labeled samples to regularizing a classifier across source and target distributions.

2.8. Combined ReliefF (CReliefF)

  • CReliefF selects x, H, and M from labeled samples in both domains, whereas traditional ReliefF uses only target-domain labeled samples.This extends ReliefF's sample selection to include source-domain data.Source: §2.8 · p. 8

2.9. LDA

  • LDA assumes full-rank, equal feature covariance matrices for the two classes, both denoted Σt, and computes class means from target-domain labeled samples.The means are computed from the Nl labeled target samples.Source: §2.9 · p. 9

2.10. Combined LDA (CLDA)

  • When Nl is small, CLDA concatenates labeled source- and target-domain samples to train LDA, treating all samples equally.Source: §2.10 · p. 9

2.11. Weighted Adaptation Regularization (wAR)

  • wAR minimizes a regularized loss combining source- and target-sample classification losses, structural risk, and marginal and conditional distribution distances.Its formulation uses sample weights, a kernel, and non-negative regularization parameters.Source: §2.11 · pp. 9–10

2.12. Online wAR (OwAR)

Online wAR applies the wAR regularized loss to online cross-subject EEG classification, using only labeled source and target samples to compute its kernel matrix.

2.12. Online wAR (OwAR)

  • OwAR uses the wAR loss but computes its kernel matrix only from labeled source and target samples, since unlabeled target samples are unavailable online.Source: §2.12 · p. 10

3. Experiments and Results

Offline cross-subject experiments on two MI datasets compared transfer learning (TL) across BCI pipeline components. Results favored broader and more sophisticated TL, with data alignment improving performance in many TL configurations.

3.1. MI Datasets

  • The two datasets covered seven subjects with 59-channel EEG and nine subjects with 22-channel EEG; the analyses used selected MI classes from each dataset.Dataset 1 included 100 trials per class; Dataset 2a used 72 trials per class.Source: §3.1 · pp. 10–11

3.2. Algorithms; 3.3. Experimental Settings and Results

  • The 27 algorithms varied TL across spatial filtering and classification, with EA- and PS-based alignment variants; comparisons used each subject as target and the remaining subjects as sources.Target labeled-sample count Nl ranged from 0 to 20, and randomized selections were repeated 30 times.Source: §3.2 · pp. 11–12, §3.3 · p. 12

3.5. The Effect of Data Alignment

  • EA and PS significantly improved accuracy when TL was used in spatial filtering or classification, especially on Dataset 1; alignment slightly reduced accuracy with CSP-LDA alone.The authors report no explanation for that reduction and note that alignment is unnecessary when no TL is performed.Source: §3.5 · p. 14, Table 3 · p. 17

3.4. The General Effect of TL

  • At small Nl, TL almost always outperformed CSP-LDA; as Nl increased, sophisticated approaches usually did, while some simple approaches fell behind (Figure 6).Across Nl values from 4 to 20, more sophisticated approaches generally achieved larger improvements over CSP-LDA.Source: §3.4 · p. 14, Figure 6 · p. 15, Table 2 · p. 16

3.6. The Effect of TL in Spatial Filtering

  • RCSP almost always outperformed corresponding CSP and CCSP versions, whereas simple spatial-filter TL did not consistently help across datasets.For example, CCSP-CLDA was worse than CSP-CLDA on Dataset 1 but better on Dataset 2a.Source: §3.6 · p. 18, Table 4 · p. 18

3.7. The Effect of TL in Feature Selection

  • CReliefF generally improved on ReliefF for selecting six filters from 20, but CSP's six leading eigenvectors generally performed best, supporting omission of feature selection.The comparison used cross-subject classification results across 24 algorithms.Source: §3.7 · p. 19, Table 5 · p. 20

3.8. The Effect of TL in the Classifier

  • wAR almost always outperformed CLDA, regardless of TL in other components; EA or PS reduced wAR's improvement over CLDA.The authors attribute the smaller margin to reduced source–target discrepancy after alignment.Source: §3.8 · pp. 19, 21, Table 6 · p. 21

4. Discussion

The discussion extends the proposed TL pipeline to deep-learning and online cross-subject classification, and to offline cross-session classification. Reported results support data alignment and multi-component TL in these tested settings, while cross-session performance varied across subjects.

4.1. Offline Cross-Subject Classification Using Deep Learning

  • EA improved classification performance for both deep-learning models, especially on Dataset 1, in offline unsupervised cross-subject classification with all target-subject samples unlabeled (Nl = 0).Source: §4.1 · p. 22

4.2. Online Cross-Subject Classification

  • In online cross-subject classification, the offline observations generally held: more TL components and data alignment benefited performance.EA and PS used only available labeled target samples, and OwAR replaced wAR to avoid using offline unlabeled target samples.Source: §4.2 · pp. 22–23
  • Offline cross-subject results were generally slightly better than online results, where unlabeled target samples were unavailable to EA, PS, and wAR.The offline–online comparison is reported across Tables 2 and 8.Source: §4.2 · p. 23

4.3. Offline Cross-Session Classification

  • Cross-session stability differed by subject: five subjects had stable performance as Nl increased, while the other four showed larger variations.For Dataset 2a, each subject's training session was the source and evaluation session the target.Source: §4.3 · p. 23
  • EA-CCSP-wAR achieved the best average cross-session performance, and almost all EA approaches outperformed their counterparts without EA.EA-RCSP-wAR was slightly worse than EA-CCSP-wAR but better than the other 16 approaches.Source: §4.3 · p. 23

5. Conclusions and Future Research

The paper argues for transfer learning across spatial filtering, feature engineering, and classification, with data alignment before spatial filtering. Experiments report improved classification across settings when transfer learning is used, especially when approaches are combined.

Conclusions

  • Data alignment before spatial filtering makes source and target data more consistent; the paper identifies it as important for both traditional machine learning and deep learning.Alignment and transfer learning in different components are described as complementary.Source: §5 · pp. 23, 27
  • Table 8 marks some online cross-subject gains over CSP-LDA as nonsignificant, while Table 9 reports all offline cross-session gains as significant at α = 0.05.The captions provide no individual accuracy values.Source: Table 8 · p. 25, Table 9 · p. 26
  • Transfer learning across BCI components generally improves classification over no transfer, and combining components can improve performance further.The reported pattern spans cross-subject and cross-session, online and offline classification.Source: §5 · p. 27
  • More sophisticated transfer-learning approaches generally outperform simpler ones.Source: §5 · p. 27

Future Research

  • The evaluation covers binary MI classification; future work would extend it to other BCI paradigms, regression, and multiclass classification.The paper also identifies feature-engineering transfer learning as an area needing more attention.Source: §5 · p. 27
  • Future work will explore more sophisticated transfer learning for feature engineering and integrating active learning with transfer learning in other BCI components.The paper notes that active learning has been integrated with classifier transfer learning in prior work.Source: §5 · p. 27
Loading 2007.03746v3…