Source-linked AI summary
MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data
Xabier de Zuazo, Vincenzo Verbeni, Eva Navas, Ibon Saratxaga, Mathieu Bourguignon, Nicola Molinaro
TL;DR
Speech BCIs need robust decoders despite limited subject-specific data. This study pre-trains a Conformer on 50 hours of single-subject listening data and fine-tunes with 5 minutes per subject across 18 participants. Transfer learning improves in-task performance and cross-task generalization between speech perception and production, including above-chance passive-listening decoding from production-trained models.
Problem
Speech BCI deployment is constrained because each individual typically provides minutes rather than hours of training data, while models are usually trained separately for each subject and task.
Method
A Conformer-based MEG model was pre-trained on 50 hours of single-subject listening data and fine-tuned on 5 minutes per subject across 18 participants and three speech tasks.
Results
Transfer learning improved in-task performance and cross-task generalization, with gains of 1-6% across metrics; production-trained models decoded passive listening above chance.
Takeaways & Limitations
The learned representations capture core neural processes underlying speech rather than relying solely on task- or subject-specific patterns, supporting MEG speech decoding with limited per-subject data.
Takeaways & Limitations
The study addresses speech detection rather than higher-level phoneme, word, or semantic representations, uses different languages for pre-training and fine-tuning, and relies on single-subject pre-training.
Abstract
from arXiv · showhide
Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.
1. Introduction
Speech BCIs need robust decoders, but subject-specific MEG data are typically limited to minutes. This study introduces MEG-to-MEG pre-training and cross-task transfer for speech detection across perception and production.
- Per-subject MEG speech-decoding data are typically limited to minutes, constraining practical BCI deployment.
- Transfer learning has not previously been demonstrated for MEG-based speech decoding despite progress in other neuroimaging modalities.
- The study pre-trains a Conformer-based model on 50 hours of single-subject listening data and fine-tunes it using 5 minutes per subject across 18 participants.
- Transfer learning improves in-task performance and cross-task generalization across speech perception and production, with gains of 1-6% across metrics.
- Production-trained models decode passive listening above chance, indicating representations shared across speech tasks rather than task-specific motor activity alone.
2. Related Work
Prior work has advanced MEG and EEG speech decoding and applied transfer learning in other neuroimaging modalities, but MEG speech transfer and perception–production cross-task decoding remain underexplored.
- MEG speech decoding has progressed from closed-vocabulary word classification to phone-level analysis and large-scale competitions.
- EEG studies have used self-supervised and contrastive pre-training to improve cross-subject generalization, unlike prior MEG speech-decoding work.
- Transfer learning had not been demonstrated for MEG-based speech decoding, leaving its value for limited per-subject clinical BCI data unresolved.
- Cross-task speech decoding between perception and production remains underexplored, including the roles of motor-related activity and shared speech representations.
3. Methods
The study evaluates MEGConformer transfer learning from a large single-subject listening dataset to limited-data, multi-subject speech tasks, comparing in-task and cross-task decoding against training from scratch.
- Datasets: The evaluation combines over 50 hours of single-subject English listening data with approximately 5 minutes per task from 18 native Spanish-speaking participants.
- Datasets: The contrasting datasets test whether single-subject pre-training improves decoding for new subjects performing different speech tasks with limited data.
- Model Architecture: MEGConformer is a compact Conformer encoder operating directly on 306-channel windowed raw MEG segments sampled at 250 Hz.
- Model Architecture: Fine-tuning adds validation-loss checkpoint selection, RollAugment temporal shifts, soft speech-fraction targets, and shorter early-stopping patience.
- Experimental Protocol: Models are evaluated in-task and across all six train-test pairings among Listen, Playback, and Production, comparing transfer learning with training from scratch.
- Experimental Protocol: Performance is assessed with F1-macro, balanced accuracy, and AUC-macro, using paired Wilcoxon tests with Holm-Bonferroni correction across subjects.
4. Results
Transfer learning improved MEG speech detection both within tasks and across perception and production tasks, with especially strong cross-task gains. Cross-task decoding was above chance even without transfer learning, and production-to-perception performance supported shared speech representations.
- In-Task Transfer Learning: +3.7% accuracy, +2.6% F1, and +7.3% AUC were achieved for listening with transfer learning.These gains were statistically significant (W = 17.0, p = 0.005).
- Cross-Task Decoding Baseline: 65.0%-73.4% cross-task accuracy was achieved without transfer learning, with all cross-task decodings significantly above chance.Perception-to-perception transfer was stronger than production-to-perception transfer.
- Cross-Task with Transfer Learning: +6.1% accuracy and +4.2% F1 improved listen-to-playback transfer, while playback-to-listen improved by +6.3% accuracy and +4.1% F1.Both directions also showed significant AUC gains, and the overall transfer benefit was highly significant (p < 0.001).
- Transfer Effect Size: 1.7-3.5% cross-task gains exceeded the 0.5-2.2% gains observed for in-task decoding.Cross-task decoding involving production showed greater variability across subjects.
- Cross-Task with Transfer Learning: 85.3% F1 for listen-to-production exceeded 80.1% F1 for production-to-listen, revealing directional asymmetry in cross-task transfer.Listening and playback transferred bidirectionally with similar F1 performance, whereas production-involving pairs showed clearer directional preferences.
- Subject-Level Effects: 15 of 18 perception subjects and 16 of 18 production subjects improved with transfer learning, although subject-level effects varied substantially.Subject 16 showed a marked negative production effect of -13.3% F1.
5. Discussion and Conclusion
Transfer learning improves MEG speech detection across subjects and tasks, with especially strong benefits for cross-task generalization. The results support shared speech representations while retaining limits from task scope, language differences, single-subject pre-training, and modest, variable gains.
- Discussion and Conclusion: Transfer learning enhanced MEG speech detection across new subjects and tasks despite only a few minutes of subject-specific data.The findings indicate that pre-training improved performance beyond task- or subject-specific training patterns.
- Discussion and Conclusion: Models trained on perception and production data showed asymmetric transfer, with production-to-perception decoding consistently weaker than the reverse direction.Perception tasks transferred bidirectionally with comparable performance, whereas production-to-perception transfer was weaker.
- Discussion and Conclusion: Figure 3 compares each subject’s in-task F1 improvement from transfer learning across the three tasks.Each bar represents transfer learning minus training from scratch; colors distinguish tasks.
- Discussion and Conclusion: Cross-task gains were larger than in-task gains, with statistically reliable improvements emerging primarily for generalization between speech perception and production.The authors frame cross-task improvement as the more important outcome for transfer learning.
- Discussion and Conclusion: The study is limited to speech detection, uses English pre-training and Spanish fine-tuning, relies on one pre-training subject, and finds modest, variable improvements.The authors characterize transfer learning as complementing rather than replacing subject-specific adaptation.