Source-linked AI summary
Multi-Feature Riemannian Hypergraph for Online Test-Time Adaptation of Motor Imagery Brain-Computer Interface
Siqi Li, Zhi Li, Tong Liu, Shuai Zhang, Yanfei Jia, Zhiqiang Yi, Jue Xie, Ni Ji
TL;DR
MI-BCI deployment is hindered by cross-day non-stationarity, online domain shifts, and reliance on calibration data. MRieHy combines Riemannian alignment with fused covariance and deep-feature hypergraphs for unlabeled online adaptation, achieving the highest average accuracy across three four-class datasets.
Problem
MI-BCI deployment requires calibration because neural signals shift across time, while streaming test-time domain shifts can degrade online control.
Method
MRieHy aligns multi-day covariance data and fuses Riemannian covariance and cosine-similarity deep-feature hypergraphs for unlabeled online test-time adaptation.
Results
MRieHy achieved the highest average accuracy on ECoG128 (64.1±0.9%), Stieger2021 (54.1±0.4%), and BCI Competition IV 2a (71.2±0.9%).
Takeaways & Limitations
MRieHy supports online MI-BCI adaptation without labeled test-day data and mitigates cross-day performance degradation within the evaluated datasets.
Takeaways & Limitations
Existing hypergraph transfer methods require labeled test-time data and use cosine-similarity k-nearest-neighbor hypergraphs rather than Riemannian geometry.
Abstract
from arXiv · showhide
In clinical motor imagery brain-computer interface (MI-BCI) decoding, cross-day transferability and online operation remain two critical challenges. Hypergraphs can improve transferability by capturing higher-order sample relationships, yet existing hypergraph-based methods for online emotion recognition neglect the cross-day benefits of Riemannian geometry widely adopted in EEG transfer learning. To bridge this gap, we propose the Multi-feature Riemannian Hypergraph (MRieHy), a framework tailored for online test-time adaptation in MI-BCI decoding that leverages Riemannian geometry to strengthen cross-day transferability. MRieHy first computes Riemannian means of covariance matrices from cross-day training data to align multi-day distributions. It then constructs a hypergraph over covariance matrices using Riemannian distance, complemented by a second hypergraph over deep features built with cosine similarity. The two hypergraphs are fused via adaptively learned combination weights, jointly optimized with the label projection matrices. During online testing, MRieHy maintains a first-in-first-out buffer of recent samples, performs Riemannian alignment on the buffered data, and decodes with the learned hypergraph. Extensive experiments on a private four-class ECoG dataset and two public four-class EEG datasets validate that MRieHy achieves notable performance gains over state-of-the-art baselines.
1 Introduction
MI-BCIs can support communication and rehabilitation by decoding imagined movements, but practical deployment is hindered by non-stationary neural signals, recurring supervised calibration, and online domain shifts. These constraints limit plug-and-play usability and can degrade online control performance even after calibration.
- Motivation: MI-BCIs decode imagined movement patterns to enable communication, environmental interaction, and neurorehabilitation for people with severe motor impairments.The passage identifies spinal cord injury, amyotrophic lateral sclerosis, and stroke-induced paralysis as relevant conditions.
- Deployment challenges: Non-stationary neural signals fluctuate with attention and fatigue, typically requiring a supervised calibration session before each use.This recurring calibration requirement is presented as a critical practical-deployment challenge.
- Deployment challenges: Recurring calibration burdens patients with limited cognitive resources and conflicts with the general public’s expectation of plug-and-play usability.Continuous online signal streaming also demands immediate feedback, while prior work reports performance degradation during online control after calibration due to domain shift.
2 Related Work
Related work establishes Riemannian geometry as a foundation for cross-day and cross-subject BCI generalization, while online test-time adaptation addresses unlabeled streaming data without calibration. Hypergraph methods capture higher-order and multimodal relationships, but existing online transfer approaches require labeled updates.
- Riemannian Geometry: Covariance matrices of spatiotemporal EEG signals lie on the SPD manifold, where Riemannian metrics provide distances and means with strong cross-day and cross-subject generalization.The manifold also supports sensor–source space equivalence and robustness of geometric means to artifacts.
- Riemannian Geometry: Riemannian BCI classifiers include minimum distance to mean, tangent-space linear discriminant analysis, and class-dispersion modeling for cross-day and cross-subject classification.These methods use Riemannian distance and Riemannian mean, with later improvements incorporating within-class dispersion.
- Online Test-Time Adaptation: Online test-time adaptation uses only unlabeled testing data during online inference to address training–testing domain gaps when same-domain calibration data is difficult to obtain.This setting is particularly suitable for BCI deployment because day-to-day instability must be addressed without calibration.
- Hypergraph-Based Transfer Learning: Hypergraphs model complex relationships beyond pairwise connections and support multimodal learning, including online cross-subject emotion recognition and fusion of EEG, ECG, and galvanic skin response signals.Existing online transfer methods add hyperedges for incoming unknown-subject samples but require labeled data to update the hypergraph.
3 Method
MRieHy adapts MI-BCI decoding across days by combining Riemannian-aligned covariance representations with deep features in two adaptively fused hypergraphs. Online, it aligns samples in a FIFO buffer and predicts using the learned hypergraph classifier.
- 3.1 Data Alignment and Feature Extraction: MRieHy aligns multi-day covariance data with Riemannian means before extracting covariance and deep features for hypergraph-based classification.Covariance alignment uses whitening based on Euclidean or Riemannian covariance means.
- 3.2 Multi-Feature Hypergraph Construction: The method constructs covariance and deep-feature hypergraphs using Riemannian-distance-based and cosine-similarity relationships, respectively.Covariance representations can also use tangent-space or Gaussian-kernel Riemannian similarities, while deep features are extracted before BaseNet’s final linear layer.
- 3.4 Online Test-Time Adaptation: During online testing, incoming samples are stored in a FIFO buffer, whose contents are aligned for domain adaptation before covariance and deep features are extracted for prediction.Alignment can be Riemannian or Euclidean and begins with the first sample.
- 3.2 Multi-Feature Hypergraph Construction: Each hypergraph connects every vertex to its k nearest neighbors, representing higher-order relationships through hyperedges and incidence matrices.Hyperedge weights are initialized uniformly as 1/ne, where ne is the total number of hyperedges.
- 3.3 Hypergraph Learning: MRieHy learns regularized projection matrices that map concatenated sample features to the label space while jointly optimizing combination weights across the two hypergraphs.The multi-feature objective aggregates individual hypergraph learning costs and regularizes the combination weights.
4 Experiments
MRieHy was evaluated for online cross-day MI-BCI adaptation on one four-class ECoG and two four-class EEG datasets, consistently achieving the strongest average accuracy among the compared methods. Ablations and sensitivity analyses attribute its performance to Riemannian similarity, multi-feature fusion, and robust behavior across training durations and hyperparameters.
- Online test-time adaptation: MRieHy achieved 64.1±0.9% accuracy on ECoG128, exceeding BaseNet+RieMDM by 1.4% and other adaptation methods by 1.9%–6.1%.EuHy and MEuHy reached 45.7±5.6% and 56.5±2.3%, respectively.
- Online test-time adaptation: MRieHy obtained the highest average accuracy on Stieger2021 at 54.1±0.4%, led CoTTA by 0.9%, and ranked first for four of seven subjects.The passage describes this subject-level pattern as consistent robustness.
- Online test-time adaptation: MRieHy reached 71.2±0.9% on BCI Competition IV 2a, surpassing BaseNet, BaseNet+RieMDM, and RieMDM by 2.6%–12.6%.It also outperformed other test-time adaptation methods by 6.6%–11.1%; EuHy and MEuHy achieved 25.9±0.7% and 43.5±1.6%.
- Ablation study: Riemannian similarity measures consistently outperformed Euclidean approaches across all three datasets, while combining covariance and deep features improved cross-day generalization.The ablation study examined alignment methods, feature combinations, and similarity strategies.
- Hypergraph analysis: TanDM hypergraphs contained more distinct days per hyperedge than cosine-similarity hypergraphs on ECoG128 and Stieger2021, with constant samples per hyperedge.This indicates stronger cross-day sample alignment from Riemannian distance-based similarity.
- Training duration and sensitivity: MRieHy accuracy rose sharply through three training days on ECoG128 and improved steadily with additional days on Stieger2021, while remaining stable across neighbor counts.Deep features benefited more from additional data, whereas the covariance component remained relatively stable; larger buffers produced only small gains before plateauing.
5 Conclusion
MRieHy is presented as an online test-time adaptation framework for MI-BCIs that combines Riemannian covariance and deep-feature hypergraphs to mitigate cross-day degradation without labeled test-day data. The paper notes that its buffer sampling lacks stability analysis and outlier rejection, identifying these as future-work needs.
- Conclusion: MRieHy combines a Riemannian-distance hypergraph over covariance matrices with a cosine-similarity hypergraph over deep features for online MI-BCI adaptation.The framework is designed for online test-time adaptation and addresses cross-day performance degradation without labeled test-day data.
- Conclusion: MRieHy mitigates cross-day performance degradation without labeled test-day data.This benefit is attributed to combining the covariance-matrix and deep-feature hypergraphs.
- Conclusion: The buffer-sampling strategy lacks stability analysis and outlier rejection mechanisms, motivating future work.These limitations are explicitly identified in the conclusion.
A Multi-feature Hypergraph Learning Details
The multi-feature hypergraph learns projection matrices that map covariance or deep features to labels while combining hypergraph-specific costs through regularized modality weights. Optimization alternates between independently solving each projection matrix and integrating modalities through the learned weights.
- Single-hypergraph learning: Hypergraph learning projects concatenated covariance or deep features into label space using a regularized matrix M.Covariance features produce Z ∈ R^(C·C)×n, while deep features produce Z ∈ R^d×n; labels are represented as one-hot vectors in Y.
- Single-hypergraph learning: The single-hypergraph cost combines a hypergraph Laplacian regularizer, empirical loss, and regularization on M.The Laplacian encourages strongly connected vertices to share similar labels, while the ℓ2,1 norm reduces overfitting and promotes row-wise sparsity.
- Multi-feature hypergraph learning: The multi-feature objective aggregates individual hypergraph learning costs and regularizes the combination weights ω across the two hypergraphs.Here, m = 2, and Γ(ω) is defined as the ℓ2 norm of the modality-weight vector.
- Multi-feature hypergraph learning: Optimization decomposes into m + 1 separable subproblems, solving each projection matrix Mh independently before optimizing ω to integrate the modalities.The projection-matrix updates use a method similar to the single-hypergraph objective, and the learned weights combine the modalities.
B Computational Cost
MRieHy requires about 63 ms per sample for offline training and 14 ms per sample for online inference on the reported GPU and CPU hardware.
- B Computational Cost: Offline training takes about 63 ms per sample, while online inference takes about 14 ms per sample.Experiments used an NVIDIA GeForce RTX 3090 Ti GPU and a 12th Gen Intel Core i9-12900K CPU.
C Ethics Statement
The experiments received IRB approval, all subjects provided consent, and electrode locations were determined exclusively by clinical considerations.
- Experiments contributing to this work were approved by an IRB, all subjects consented to participate, and electrode locations followed clinical considerations exclusively.
D Percentage of Distinct Days per Hyperedge for More Subjects
This section reports the percentage of distinct days represented per hyperedge in the covariance-based hypergraph component for more subjects.
- Percentage of Distinct Days per Hyperedge for More Subjects: The figure measures the percentage of distinct days per hyperedge.It focuses on how many distinct days are represented within each hyperedge.
- Percentage of Distinct Days per Hyperedge for More Subjects: The analysis concerns the covariance-based hypergraph component.The reported hyperedges are from the covariance-based component rather than another feature representation.
- Percentage of Distinct Days per Hyperedge for More Subjects: Figure D.1 presents this distinct-day percentage metric across hyperedges.The figure is labeled Figure D.1 and organizes the analysis around hyperedge-level day diversity.