PaperlayerView exact revision on arXiv

Source-linked AI summary

Manifold Embedded Knowledge Transfer for Brain-Computer Interfaces

Wen Zhang, Dongrui Wu

arXiv:1910.05878v2cs.HCcs.LG

TL;DR

Offline unsupervised cross-subject EEG classification must transfer from labeled source subjects to an unlabeled target despite subject variation and costly calibration. MEKT aligns covariance matrices on the Riemannian manifold, extracts tangent-space features, and adapts domains by reducing joint distribution shift while preserving geometry. Across four datasets and two BCI paradigms, MEKT outperformed state-of-the-art transfer methods, while DTE reduced computation by more than half for many source subjects with little accuracy sacrifice.

  • Problem

    Subject and task variations make cross-subject EEG classification difficult when source trials are labeled but target trials are unlabeled.

  • Method

    MEKT aligns covariance matrices on the Riemannian manifold, maps them into tangent-space features, and minimizes joint source-target distribution shift while preserving geometric structure.

  • Results

    MEKT achieved the best or second-best performance across all evaluated scenarios, while DTE saved over 50% computational cost when the number of source domains was large.

  • Takeaways & Limitations

    DTE can select beneficial source domains for large-scale multi-source transfer with little sacrifice of classification accuracy.

Abstract

from arXiv · show

Transfer learning makes use of data or knowledge in one problem to help solve a different, yet related, problem. It is particularly useful in brain-computer interfaces (BCIs), for coping with variations among different subjects and/or tasks. This paper considers offline unsupervised cross-subject electroencephalogram (EEG) classification, i.e., we have labeled EEG trials from one or more source subjects, but only unlabeled EEG trials from the target subject. We propose a novel manifold embedded knowledge transfer (MEKT) approach, which first aligns the covariance matrices of the EEG trials in the Riemannian manifold, extracts features in the tangent space, and then performs domain adaptation by minimizing the joint probability distribution shift between the source and the target domains, while preserving their geometric structures. MEKT can cope with one or multiple source domains, and can be computed efficiently. We also propose a domain transferability estimation (DTE) approach to identify the most beneficial source domains, in case there are a large number of source domains. Experiments on four EEG datasets from two different BCI paradigms demonstrated that MEKT outperformed several state-of-the-art transfer learning approaches, and DTE can reduce more than half of the computational cost when the number of source subjects is large, with little sacrifice of classification accuracy.

I. INTRODUCTION

EEG-based BCIs face subject-dependent neural responses and costly target-specific labeling, motivating transfer learning for cross-subject adaptation. The paper proposes MEKT to align EEG covariance information, embed it in tangent space, and adapt source knowledge to targets.

  • EEG-based BCIs use MI and ERP paradigms, whose discriminative information arises primarily from spatial and temporal patterns, respectively.
  • MEKT addresses the computational burden and Euclidean incompatibility of Riemannian approaches by combining manifold processing with tangent-space learning.
  • Subject-dependent neural responses and expensive labeled calibration make transfer learning valuable for adapting BCI models across users and tasks.
  • MEKT aligns EEG-trial covariance matrices on the Riemannian manifold, maps them to tangent-space features, and performs domain adaptation while preserving geometric structure.
  • Spatial Filters: CSP-based spatial filtering extracts features by maximizing variance differences between binary classes and using log-variances of spatially filtered trials.

B. Riemannian Geometry

Riemannian geometry represents EEG covariance matrices as points on a manifold and supplies distances, means, and tangent-space mappings for processing them. Congruent transforms preserve the relevant geometry, supporting alignment and classification.

  • Symmetric positive definite EEG covariance matrices form a differentiable Riemannian manifold.
  • Riemannian distance compares SPD matrices using the Frobenius norm of the matrix logarithm of their relative transform.
  • Euclidean, Riemannian, and Log-Euclidean means provide alternative centers for collections of SPD covariance matrices.
  • Tangent-space mapping converts an SPD matrix into a Euclidean vector around a reference matrix, using Mref = M^-1/2 for local homomorphism.
  • Congruent transforms preserve Riemannian distances, enabling covariance alignment without changing the geometry between transformed matrices.
  • Riemannian Alignment: Riemannian alignment uses a reference mean from resting or non-target covariance matrices and classifies aligned trials with minimum distance to mean.

E. Euclidean Alignment (EA)

Euclidean Alignment (EA) transforms EEG trials using the Euclidean mean to address alignment limitations, but its distributional scope and numerical stability depend on channel dimensionality. MEKT instead combines covariance alignment, tangent-space features, and shared-subspace mapping for labeled-source, unlabeled-target transfer.

  • Euclidean Alignment (EA): EA extends Riemannian alignment by transforming EEG trials in Euclidean space using the Euclidean mean of covariance matrices.The Euclidean mean is computed from all EEG-trial covariance matrices.
  • Euclidean Alignment (EA): EA considers only marginal probability distribution shift and works best when the number of EEG channels is small.With many channels, computing the inverse square root of the Euclidean mean may be numerically unstable.
  • Related transfer methods: Earlier transfer approaches include transfer component analysis, joint distribution adaptation, and joint geometrical and statistical alignment.These methods progressively address marginal, conditional, and geometric or class-scatter properties in domain adaptation.
  • MEKT setup: MEKT targets one or multiple source subjects with the same feature and label spaces as the target subject.The method assumes labeled source instances and unlabeled target feature matrices.
  • MEKT pipeline: MEKT aligns covariance centroids, extracts tangent-space features, and identifies shared-subspace projection matrices for source-target transfer.The shared subspace has dimensionality p much smaller than the tangent-space dimension d, and a classifier trained on projected labeled source data produces target pseudo-labels.

A. Covariance Matrix Centroid Alignment (CA)

Covariance Matrix Centroid Alignment (CA) aligns source and target covariance centroids so their marginal distributions become closer on the manifold. It also approximately whitens the aligned EEG trials, with computational variants based on different means.

  • Centroid alignment: CA serves as preprocessing that reduces marginal probability distribution shift and enables transfer from multiple source domains.The alignment operates on covariance matrices from source and target domains.
  • Centroid alignment: Choosing the Riemannian or Euclidean mean makes the corresponding domain centers equal the identity matrix after alignment.This brings source and target marginal distributions closer on the manifold or in arithmetic geometry, respectively.
  • Whitening effect: CA approximately whitens each aligned covariance matrix, with diagonal elements near 1 and off-diagonal elements near 0.The whitening result is established for Riemannian- and Euclidean-mean CA and approximately extends to Log-Euclidean CA.
  • Computational variants: Riemannian-mean CA is iterative and initialized by the Euclidean mean, whereas Log-Euclidean CA approximates it with reduced computational cost.The whitening effect is also evaluated experimentally in the paper.

B. Tangent Space Feature Extraction

After covariance alignment, MEKT maps EEG covariance matrices into tangent-space vectors and learns projections that reduce source-target joint distribution shift while preserving discriminability and target locality. The method also regularizes source-target projection transfer.

  • Tangent-space features: Covariance-aligned matrices are mapped into tangent-space feature vectors, then assembled into source and target feature matrices.The tangent-space feature dimension is d = c(c + 1)/2.
  • Tangent-space features: The proposed mapping uses subject-specific reference matrices rather than one common reference matrix for all subjects.This distinguishes the method from the original tangent-space mapping described earlier.
  • Distribution adaptation: Joint probability MMD measures source-target distribution shift directly through joint probabilities instead of treating marginal and conditional discrepancies separately.The measure uses source labels and predicted target labels represented with one-hot encoding.
  • Source discriminability: Source-domain discriminability is preserved during subspace mapping through within-class and between-class scatter terms.This retains class structure in the projected source features.
  • Target locality: A graph-based regularizer preserves target-domain locality by keeping nearby target samples close in the projected subspace.The similarity graph uses p-nearest neighbors and a normalized graph Laplacian.
  • Parameter transfer: Projection matrices are encouraged to remain similar across source and target domains and avoid extreme values.The shared feature space and covariance alignment motivate parameter transfer from source projection A to target projection B.

D. The Overall Loss Function of MEKT

MEKT integrates distribution alignment, source discriminability, target locality, and projection regularization into one objective. The optimization uses generalized eigen-decomposition and iterative pseudo-label refinement, while multiple source domains are assembled after separate alignment.

  • Overall objective: The overall MEKT formulation integrates the source-discriminability, target-locality, and parameter-regularization terms with joint distribution adaptation.The formulation combines the regularization components into a single optimization objective.
  • Overall objective: Trade-off parameters α, β, and ρ balance source discriminability, target locality, and parameter regularization, respectively.These parameters control the relative importance of the three corresponding regularizers.
  • Optimization: The objective is solved by generalized eigen-decomposition, with W formed from the p trailing eigenvectors.Because predicted target labels enter the objective, an expectation-maximization-like pseudo-label refinement procedure is used.
  • Multiple source domains: For multiple source domains, MEKT performs covariance alignment and tangent-space feature extraction separately before concatenating the source feature matrices.The assembled matrix contains features from each source domain.

E. Kernelization Analysis

MEKT can be kernelized in a reproducing kernel Hilbert space, replacing original samples with mapped representations and optimizing corresponding projection matrices.

  • E. Kernelization Analysis: Nonlinear MEKT is obtained through kernelization in a Reproducing Kernel Hilbert Space.This extends the framework beyond its primal formulation.
  • E. Kernelization Analysis: Kernelized MEKT maps samples through a kernel function and uses the Representer Theorem to express projections in terms of the combined source-target data.The projection matrices are optimized in coefficient form over the mapped data.
  • E. Kernelization Analysis: The kernel formulation constructs source and target kernel matrices, then replaces samples and domains in the MEKT derivation with their mapped counterparts.This yields the kernelized optimization problem.
  • E. Kernelization Analysis: The MEKT algorithm takes labeled source samples, unlabeled target feature matrices, iteration and weighting parameters, and a shared-subspace dimensionality as inputs.It outputs predicted target labels and the source-domain mean matrix.
  • E. Kernelization Analysis: MEKT constructs source and target tangent-space features before iteratively forming adaptation matrices, projections, and classifier inputs.The procedure trains on projected source features and updates target predictions and the adaptation matrix.

F. Domain Transferability Estimation (DTE)

DTE estimates which source domains are most transferable to an unlabeled target, helping select useful sources and reduce computation when many source domains exist.

  • F. Domain Transferability Estimation (DTE): DTE is motivated by large-source settings, where estimating transferability can identify important domains and reduce computational cost.It can also help find subjects with low task correlations that may cause negative transfer.
  • F. Domain Transferability Estimation (DTE): DTE assumes labeled source domains and an unlabeled target domain, then compares source discriminability with source-target domain difference.These quantities are defined using between-class and between-domain scatter matrices.
  • F. Domain Transferability Estimation (DTE): The method selects z∗ source subjects with the highest transferability scores.This provides a source-domain selection rule for multi-source transfer.
  • F. Domain Transferability Estimation (DTE): The experiments evaluate DTE-related transfer settings through unsupervised single-source-to-single-target and multi-source-to-single-target transfers.The code is available online.

A. Datasets

The study evaluates MEKT on two motor-imagery and two event-related-potential EEG datasets, using task-specific preprocessing and feature constructions.

  • A. Datasets: The experiments use two MI datasets and two ERP datasets, with dataset statistics summarized in Table I.The datasets span different EEG paradigms and subject counts.
  • A. Datasets: MI1 contains 59-channel EEG from seven subjects, while MI2 contains 22-channel EEG from nine subjects; both support two-class classification.Each dataset includes left- and right-hand motor-imagery trials.
  • A. Datasets: The RSVP ERP dataset contains 8-channel recordings from 11 subjects and classifies target versus non-target images in the 5 Hz experiment.Subject trial counts range from 368 to 565, with an approximately 1:9 target-to-non-target ratio.
  • A. Datasets: The ERN ERP dataset uses training data from 16 subjects, with 56-channel EEG downsampled to 200 Hz and an approximately 1:4 target-to-non-target ratio.The held-out test subjects were not used.
  • A. Datasets: For ERP data, augmented covariance matrices provide tangent-space features from temporal-information blocks, while Euclidean features use xDAWN, vectorization, and PCA.The ERP tangent-space input uses c^2 selected elements.
  • A. Datasets: The comparison includes MEKT-R, MEKT-E, and MEKT-L alongside seven state-of-the-art baselines spanning Euclidean, Riemannian, and tangent-space approaches.MEKT variants differ by their covariance reference matrix.
  • A. Datasets: MEKT experiments use T = 5, α = 0.01, β = 0.1, ρ = 20, and d = 10.These are the reported MEKT hyperparameter settings.

D. Experimental Settings

Experiments assess unsupervised single- and multi-source transfers using balanced classification accuracy, covariance alignment visualizations, statistical tests, and kernel comparisons.

  • D. Experimental Settings: Single-source transfers include z(z − 1) ordered source-target tasks, whereas multi-source transfers use all other subjects as sources for each of z targets.For MI1, this yields 42 STS and seven MTS tasks.
  • D. Experimental Settings: Balanced classification accuracy is the performance measure, with class-wise true positives divided by the number of samples in each class.The reported results include means and standard deviations for STS and MTS transfers.
  • D. Experimental Settings: Centroid alignment makes aligned covariance matrices approximately identity across Riemannian, Euclidean, and Log-Euclidean reference choices.The aligned EEG trials are therefore approximately whitened.
  • D. Experimental Settings: All MEKT-based approaches achieve the best or second-best performance across the reported STS and MTS scenarios.The Riemannian mean gives the best BCA in four of six tangent-space approaches and the best overall performance.
  • D. Experimental Settings: MEKT-R significantly outperforms all baselines in almost all STS transfers, while improvements are less significant with multiple source domains.The analysis uses normality checks, paired t-tests, and false-discovery-rate correction at α = 0.05.
  • D. Experimental Settings: MEKT-R is tested with primal, linear-kernel, and RBF-kernel forms; the primal version is best or second-best in every scenario, with small differences among variants.The RBF kernel width is 0.1.

G. Computational Cost

The experiments compare computational costs across transfer approaches and evaluate source-domain selection. MEKT variants offer different accuracy–cost trade-offs, while DTE substantially reduces computation with little accuracy loss when many source subjects are available.

  • Kernel comparison: The primal MEKT-R achieved the best or second-best performance across all kernel scenarios, although differences among kernels were very small.The comparison included primal, linear-kernel, and RBF-kernel MEKT-R.
  • Computational cost: MEKT-L achieved the best compromise between classification accuracy and computational cost, while MEKT-E and MEKT-L used much less computation than MEKT-R.These variants retained comparable classification performance with MEKT-R; RA-MDM, CA-JDA, and MEKT-R had similar computational costs.
  • MMD comparison: Joint probability MMD outperformed traditional MMD in six of eight tasks.The experiment replaced the joint probability MMD term with traditional MMD and repeated the evaluations.
  • Source-domain selection: DTE outperformed RAND and ROD in classification accuracy while using substantially less computation than selecting all source subjects.When the number of source subjects was large, DTE saved over 50% of computational cost, with BCAs generally only slightly below ALL.
Loading 1910.05878v2…