Source-linked AI summary
Autoreject: Automated artifact rejection for MEG and EEG data
Mainak Jas, Denis A. Engemann, Yousra Bekhti, Federico Raimondo, Alexandre Gramfort
TL;DR
M/EEG artifact preprocessing is difficult to scale because manual rejection is time-consuming and parameter tuning is challenging. Autoreject uses robust cross-validation to learn global or sensor-specific thresholds, then repairs or rejects contaminated trials. Across four open datasets with more than 200 subjects, validation found performance at least as good as diverse alternatives and often considerably better, while reducing expert inspection and preserving repairable data.
Problem
M/EEG preprocessing lacks a scalable, reproducible way to tune artifact-rejection parameters and avoid unnecessary loss of costly data.
Method
Autoreject learns peak-to-peak thresholds with robust cross-validation, extends them per sensor, and interpolates or rejects trials according to the number of bad sensors.
Results
Across four open datasets containing more than 200 subjects, autoreject performed at least as well as diverse alternatives and commonly used procedures, often considerably better.
Takeaways & Limitations
The automated, data-driven procedure reduces expert inspection, preserves repairable data, and establishes reproducible inclusion rules for M/EEG analysis.
Takeaways & Limitations
Autoreject assumes artifact independence across training and validation trials and detects bad sensors using only peak-to-peak thresholds.
Abstract
from arXiv · showhide
We present an automated algorithm for unified rejection and repair of bad trials in magnetoencephalography (MEG) and electroencephalography (EEG) signals. Our method capitalizes on cross-validation in conjunction with a robust evaluation metric to estimate the optimal peak-to-peak threshold -- a quantity commonly used for identifying bad trials in M/EEG. This approach is then extended to a more sophisticated algorithm which estimates this threshold for each sensor yielding trial-wise bad sensors. Depending on the number of bad sensors, the trial is then repaired by interpolation or by excluding it from subsequent analysis. All steps of the algorithm are fully automated thus lending itself to the name Autoreject. In order to assess the practical significance of the algorithm, we conducted extensive validation and comparison with state-of-the-art methods on four public datasets containing MEG and EEG recordings from more than 200 subjects. Comparison include purely qualitative efforts as well as quantitatively benchmarking against human supervised and semi-automated preprocessing pipelines. The algorithm allowed us to automate the preprocessing of MEG data from the Human Connectome Project (HCP) going up to the computation of the evoked responses. The automated nature of our method minimizes the burden of human inspection, hence supporting scalability and reliability demanded by data analysis in modern neuroscience.
1 Introduction
M/EEG preprocessing is a critical bottleneck for scalable and reproducible analysis because artifacts can corrupt results, while manual inspection is time-consuming and variable. The paper proposes automating manually tuned rejection decisions with cross-validation and sensor-specific repair.
- Large-scale M/EEG studies have increased the need for automated analysis pipelines that support scalability, reanalysis, and reproducibility.
- Artifact annotation and rejection remain a critical preprocessing bottleneck because M/EEG signals are easily corrupted by noise and artifacts.
- Rejecting contaminated data protects downstream analyses but can discard data that are costly to acquire, especially in clinical and infant studies.
- Manual inspection and peak-to-peak thresholds require human judgment, which can fluctuate, be difficult to reproduce, and depend on parameter tuning.
- Autoreject uses cross-validation with Bayesian optimization to automate manually chosen peak-to-peak thresholds and extends them to sensor-specific trial marking.
- Depending on how many sensors are bad, the method either interpolates selected sensors or rejects trials that cannot be reliably repaired.
2 Materials and methods
Autoreject uses cross-validation and robust error evaluation to select peak-to-peak thresholds, then extends this procedure to sensor-specific rejection and repair. The local method rejects trials with too many bad sensors and interpolates a limited number of the worst sensors in otherwise retained trials.
- Autoreject (global): Cross-validation selects a global peak-to-peak threshold by minimizing average mismatch between cleaned training data and robust validation estimates.The validation estimate uses the median to reduce sensitivity to high-amplitude artifacts.
- Autoreject (global): Low thresholds discard most trials and produce noisy averages, whereas high thresholds retain artifact-contaminated trials and overfit.The threshold therefore balances trial retention against artifact rejection.
- Autoreject (local): The local method estimates thresholds per sensor and marks each trial-sensor pair as bad when its peak-to-peak amplitude exceeds that sensor’s threshold.The indicator matrix records these sensor-level bad segments for subsequent trial-wise decisions.
- Autoreject (local): Trials exceeding κ bad sensors are rejected; otherwise, up to ρ of the worst sensors are interpolated, with ρ necessarily smaller than κ.EEG interpolation uses spherical splines, while MEG interpolation uses minimum-norm estimates with spherical harmonics.
- Data augmentation: Cross-validation fails for globally bad sensors because every trial and the validation median are corrupted, yielding thresholds higher than the optimum.Figure 3 contrasts regular and globally bad sensors and shows the estimated threshold marking all trials bad for the latter.
- Candidate thresholds using Bayesian optimization: Bayesian optimization adaptively proposes candidate thresholds using the observed cross-validation errors instead of relying on a fixed threshold grid.This avoids missing the optimum with sparse grids while reducing the computational cost of dense grids.
3 Experimental Validation Protocol
The study validates Autoreject against alternative rejection strategies using an evaluation metric based on cleaned evoked responses across four public EEG and MEG datasets. The protocol spans more than 200 subjects and includes HCP MEG data, for which clean average responses enable objective comparison.
- Evaluation metric: The evaluation compares evoked responses cleaned by each method against a ground-truth evoked response.The comparison uses X(method) and X(clean) as the respective cleaned and ground-truth responses.
- Evaluation metric: The infinity norm emphasizes maximum-amplitude discrepancies rather than averaging squared differences.This makes the metric sensitive to artifacts localized on one sensor or at a particular time instant.
- Competing methods: The benchmark includes no rejection, Sensor Noise Suppression, and FASTER among the rejection strategies evaluated against Autoreject.FASTER identifies outlier sensors using variance, correlation, Hurst exponent, kurtosis, and line-noise criteria, marking sensors when a criterion's z-score exceeds 3.
- Datasets: Validation uses four open datasets containing either EEG or MEG recordings from more than 200 subjects.The analyzed paradigms include working memory, perceptual, and motor tasks.
- Datasets: The datasets include MNE sample data, EEGBCI, EEG faces, and HCP MEG recordings.The HCP working-memory analysis uses data from 83 subjects, with two runs per subject; its MEG recordings were acquired with 248 magnetometers and 23 reference sensors.
- Datasets: HCP MEG provides an objectively evaluable benchmark because its official pipeline supplies a clean average response for comparison with reprocessed raw data.The study focuses on the HCP working-memory task and compares the resulting output with the official pipeline output.
4 Results
Autoreject learns data-dependent peak-to-peak thresholds, repairs bad sensors, and is evaluated qualitatively and quantitatively across MEG and EEG datasets. The results show subject- and sensor-specific thresholds, multimodal repair, and generally favorable comparisons with alternative rejection methods.
- Autoreject was evaluated qualitatively and quantitatively on four datasets against no rejection and three alternative artifact-rejection procedures.
- 4.1 Peak-to-peak thresholds: 17% of subjects would lose all trials in one condition under a constant 150µV threshold, demonstrating strong subject dependence.
- 4.1 Peak-to-peak thresholds: Thresholds varied across homogeneous MEG sensors, confirming that amplitude distributions differ at the sensor level.
- 4.2 Visual quality check: Across MNE, HCP, and EEG faces data, autoreject automatically detected and repaired bad sensors, including multiple sensors, in both MEG and EEG.
- 4.2 Visual quality check: Autoreject did not completely remove eyeblinks when some blinks were time-locked to the evoked response.
- 4.3 Quantification of performance and comparison with state-of-the-art: On HCP MEG data, autoreject improved data quality over no rejection, SNS, FASTER, and RANSAC in the reported subject-level comparisons.
- 4.3 Quantification of performance and comparison with state-of-the-art: On EEG faces data, autoreject offered good results across all subjects, including subjects for which RANSAC underperformed.
5 Discussion
The discussion positions autoreject as an adaptive, physics-informed approach that compares favorably with competing methods while clarifying important operating conditions and limitations.
- Method and conceptual comparison: Autoreject combines data-driven parameter tuning with deterministic, physics-driven interpolation to repair sensor-level artifacts.Its thresholds identify outlier segments for individual sensors, while interpolation uses sensor geometry and physical constraints.
- Method and conceptual comparison: Compared with SNS, autoreject accounts for sensor-location physics, whereas SNS’s statistical projection may fail or deteriorate signals.SNS can also be problematic when multiple sensors are corrupted simultaneously.
- Method and conceptual comparison: Autoreject’s adaptive threshold detection is advantageous because FASTER’s fixed z-score threshold performs less well for some subjects.FASTER uses between-sensor correlation, variance, and power-spectrum criteria with thresholds fixed to a z-score of 3.
- Operating conditions and limitations: Cross-validation assumes artifact-containing trials are independent; phase-locked artifacts across training and validation trials can interfere with threshold estimation.The paper also cautions that exceeding the maximum interpolatable sensors and relying only on peak-to-peak detection create additional constraints.
- Operating conditions and limitations: Autoreject is designed for artifacts local in time, while clustered bad sensors and physiological artifacts can exceed the reliability of its spatial interpolation.The paper recommends warning users when interpolation error is likely to be high and using ICA for prototypical spatial artifacts.
- Implications for downstream analysis: Autoreject can be followed by source-localization methods without changes to source-level analyses, unlike rank-reducing denoising methods that require special attention.The paper specifically notes compatibility with beamformers, minimum-norm estimates, dSPM, and sLORETA.
6 Conclusion
The conclusion presents autoreject as an automated, data-driven method for detecting and repairing bad single-trial M/EEG segments. Across open datasets, it performed at least as well as alternatives while reducing inspection costs and preserving data through repair.
- Conclusion: Autoreject automates data-driven detection and repair of bad segments in single-trial M/EEG data.The method combines outlier detection, physics-driven channel repair, and cross-validated parameter calibration.
- Conclusion: Across four open datasets containing more than 200 subjects, autoreject performed at least as well as diverse alternatives and commonly used procedures, often considerably better.The validation also consolidated the need for subject- and sensor-level parameter tuning.
- Conclusion: Autoreject balances accuracy with interpretability because its parameters have user-facing meanings and its decisions can be explained.Diagnostic visualization routines support integration into MEG/EEG pipelines, including clinical ones.
- Conclusion: By automating a largely manual task and repairing rather than removing data, autoreject reduces expert inspection costs and preserves costly acquisitions.The approach also removes biases associated with expert training, prior experience, and expectations.
Supplementary material
The supplementary material explains the purpose of the additional content and clarifies why the infinity norm is used for quantitative HCP comparisons.
- Supplementary material: The supplementary material provides additional information intended to answer reader questions about the main text.It includes an HCP results figure and a discussion of alternative norm choices.
- Supplementary material: Using the ℓ∞ norm rather than the ℓ2 norm highlights pathological cases in which even one sensor is corrupted.The ℓ2 norm averages across sensors and can conceal isolated sensors with large artifacts.
- Supplementary material: In the HCP supplementary scatter plots, each circle represents a subject and comparisons include autoreject against no rejection, SNS, FASTER, and RANSAC.The figure uses the same HCP data as Figure 6 while computing differences with the ℓ2 norm.