Source-linked AI summary
Machine-Learning-Based Diagnostics of EEG Pathology
Lukas Alexander Wilhelm Gemein, Robin Tibor Schirrmeister, Patryk Chrabąszcz, Daniel Wilson, Joschka Boedecker, Andreas Schulze-Bonhage, Frank Hutter, Tonio Ball
TL;DR
The paper asks whether comprehensive feature-based EEG pathology decoding can be fairly compared with end-to-end neural networks, given limited prior evidence. It evaluates both approaches on the TUH Abnormal EEG Corpus and finds similar performance, while identifying practical scope boundaries for current pipelines.
Problem
Prior EEG pathology studies typically used limited features, decoders, or comparisons, leaving systematic evidence about comprehensive feature-based versus end-to-end methods incomplete.
Method
The study compares a comprehensive feature-based framework with deep neural networks, including an EEG-optimized TCN, on approximately 3000 TUH Abnormal EEG recordings.
Results
Feature-based Riemannian-geometry decoding reached 85.87% accuracy and BD-TCN reached 86.16%, with no statistical evidence that either approach outperformed the others.
Takeaways & Limitations
Both approaches used similar informative EEG aspects, including slow-frequency temporal dynamics, and feature-based decoding can perform at a similar level to neural networks.
Takeaways & Limitations
The feature collection is incomplete, and incorporating clinical information such as medical history and ongoing medication remains an open challenge.
Abstract
from arXiv · showhide
Machine learning (ML) methods have the potential to automate clinical EEG analysis. They can be categorized into feature-based (with handcrafted features), and end-to-end approaches (with learned features). Previous studies on EEG pathology decoding have typically analyzed a limited number of features, decoders, or both. For a I) more elaborate feature-based EEG analysis, and II) in-depth comparisons of both approaches, here we first develop a comprehensive feature-based framework, and then compare this framework to state-of-the-art end-to-end methods. To this aim, we apply the proposed feature-based framework and deep neural networks including an EEG-optimized temporal convolutional network (TCN) to the task of pathological versus non-pathological EEG classification. For a robust comparison, we chose the Temple University Hospital (TUH) Abnormal EEG Corpus (v2.0.0), which contains approximately 3000 EEG recordings. The results demonstrate that the proposed feature-based decoding framework can achieve accuracies on the same level as state-of-the-art deep neural networks. We find accuracies across both approaches in an astonishingly narrow range from 81--86\%. Moreover, visualizations and analyses indicated that both approaches used similar aspects of the data, e.g., delta and theta band power at temporal electrode locations. We argue that the accuracies of current binary EEG pathology decoders could saturate near 90\% due to the imperfect inter-rater agreement of the clinical labels, and that such decoders are already clinically useful, such as in areas where clinical EEG experts are rare. We make the proposed feature-based framework available open source and thus offer a new tool for EEG machine learning research.
1 Introduction
The paper addresses the lack of systematic, fair comparisons between comprehensive feature-based and end-to-end EEG pathology decoding. It proposes comparing both approaches on the TUH Abnormal EEG Corpus using broader feature sets and established neural networks.
- Motivation: Clinical EEG assessment is time-consuming, expertise-intensive, and subject to limitations including variable training, experience, rating consistency, and moderate inter-rater agreement.These factors motivate automated clinical EEG diagnosis.
- Approaches: Machine-learning EEG analysis is broadly divided into feature-based methods using handcrafted features and end-to-end methods learning features from raw or minimally preprocessed data.Feature-based methods require a priori feature choices, whereas end-to-end networks jointly optimize feature extraction and classification.
- Research gap: Prior pathology-decoding studies used limited features, decoders, or both, leaving few systematic comparisons between comprehensive feature-based pipelines and end-to-end methods.Earlier comparisons often relied on simple feature-based baselines, potentially making deep-learning approaches appear superior.
- Study objective: The study compares large-feature-set decoding with deep neural networks on approximately 3000 TUH Abnormal EEG recordings.Feature-based models include RF, SVM, Riemannian geometry, and auto-sklearn; end-to-end models include several convolutional networks.
- Comparison design: The authors exclude results using pretrained models and additional training data because access to substantially more data would make direct comparison unfair.The paper notes that additional data can have a greater effect than more elaborate algorithms.
2 Material and Methods
The study uses the public TUH Abnormal EEG Corpus and evaluates feature-based and neural-network pipelines after common preprocessing. The methods combine broad handcrafted EEG features, covariance-based Riemannian representations, cropped decoding, and several ConvNet or TCN architectures.
- Data: The TUH Abnormal EEG Corpus contains 2993 recordings from 2329 unique patients, with separate development and final evaluation sets.Recordings span patients from 7 days to 96 years, and no patients overlap between development and final evaluation sets.
- Preprocessing: Both pipelines receive common preprocessing intended to normalize input distributions and stabilize deep-network learning and feature extraction without preselecting EEG features.The feature-based pipeline then applies additional extraction-specific processing.
- Neural networks: The neural-network pipeline evaluates BD-Deep4, BD-TCN, BD-Shallow, and BD-EEGNet architectures for pathology decoding.BD-Deep4 uses separated temporal and spatial convolution followed by convolution and max-pooling blocks, while BD-TCN uses optimized temporal-convolution blocks.
- Feature extraction: Feature extraction computes time, frequency, wavelet, Fourier, connectivity, patient-information, and Riemannian-geometry features from EEG crops.The implementations include Discrete Fourier, continuous and discrete wavelet, Hilbert-transform connectivity, and covariance features.
- Feature extraction: Time-resolved decoding treats feature vectors from individual 6-s crops as independent training examples.This increases the number of training examples but also raises memory consumption and learning times.
- Feature extraction: Covariance features are aggregated using Euclidean or geometric means into vectors of length E × (E + 1)/2 for each recording and aggregation type.These representations support the Riemannian-geometry-based decoding pipeline.
- Ensembling: The study also examines ensembles by combining model predictions according to correlations and non-overlapping cross-validation errors.Majority voting is treated as the special case in which individual model predictions receive equal weight.
3 Results
Feature-based models achieved strong pathology-decoding performance on TUH data, with Riemannian geometry reaching nearly 86% accuracy and feature-based models producing more false negatives than false positives.
- 81% cross-validation accuracy increased to 86% in final evaluation for Riemannian-geometry-based decoding.The increase could indicate underfitting of the training data during cross-validation.
- Greater than 84% accuracy was obtained with both traditional and automated feature-based approaches.
- All feature-based models classified more pathological examples as non-pathological than non-pathological examples as pathological.This corresponds to a higher ratio of false negatives than false positives.
- The feature-based baseline of 78.8% was increased by greater than 5% using RF and ASC and by more than 7% using RG.The baseline used cepstral coefficients with a CNN+MLP classifier.
- 85.87% accuracy was achieved by Riemannian-geometry-based classification, outperforming the other feature-based models.Using geometric rather than Euclidean means for covariance matrices yielded superior performance.
3.2 End-to-end decoding performance
End-to-end neural networks produced competitive pathology-decoding accuracies, with BD-TCN performing best among the investigated networks while statistical tests found no reliable superiority among models.
- 86.16% accuracy was the best overall end-to-end result, obtained by BD-TCN.This was extremely close to ChronoNet’s previously reported 86.57% accuracy.
- 84.57% and 84.13% accuracy were achieved by BD-Deep4 and BD-Shallow, respectively.
- Network performance differed between cross-validation and final evaluation by −0.6% to +0.4%.Feature-based approaches showed a wider range of −1.21% to +4.61%.
- BD-EEGNet showed the highest loss and misclassification rate, with learning curves indicating underfitting.The authors attribute this to its relatively small number of parameters.
- BD-TCN was competitive with ChronoNet and outperformed the other investigated networks.Its performance may reflect its design and optimization through neural architecture search.
- Across model pairs, p-values ranged from 0.37 to 1, providing no statistical evidence that one model performed better than the others.Equal classification decisions ranged from 241–266 of 276 possible.
3.4 Importance of learned and handcrafted features
Perturbation, feature-importance, covariance, and correlation analyses converged on frequency- and electrode-specific patterns, especially temporal delta/theta activity and occipital alpha activity.
- Increased activity at temporal electrodes T3 and T4 correlated with pathological predictions, especially in the delta and theta ranges.The corresponding occipital alpha effect was a decreased correlation at O1 and O2.
- RF identified T4 features in the delta and theta ranges as most informative, while occipital electrodes were most important in alpha.T3 was not considered as informative in the RF analysis, unlike in the perturbation analysis.
- Variance at temporal electrodes T3 and T4 was most indicative of pathology in the tangent-space covariance representation.Variance at O1 and O2 was indicative of normal activity, consistent with the perturbation analysis.
- Strong correlations appeared across feature domains, frequency bands, and electrode recording sites.The feature domains included CWT, DWT, FT, Riemannian, Connectivity, and Time.
- Time-resolved feature-based decoding did not improve cross-validation accuracy and was not investigated further.The authors note that improvements could be negligible for this recording-level classification task.
- Adding age and gender improved RF cross-validation accuracy by 0.15%, from 83.1% to 83.25%.The authors therefore did not pursue these meta-features further.
- Patient age is a dataset bias because pathologies appear more frequent at higher ages.The authors caution that models could use age as a proxy for pathology and that dataset shifts can reduce generalization.
3.7 Ensemble decoding performance
Model predictions were highly correlated but also contained substantial non-overlapping errors, motivating ensemble analysis; however, ensembling did not improve final evaluation over BD-TCN.
- Up to 0.96 Spearman correlation coexisted with up to 61% non-overlapping label errors across model predictions.
- BD-Deep4, RF, and RG had the highest non-overlapping cross-validation error ratio at 44.56%.Their 336 errors motivated selection for majority-vote ensembling.
- 86.23% cross-validation accuracy was achieved by the auto-sklearn ensemble.Majority voting achieved 85.51% and automatic ensemble selection achieved 85.14% in final evaluation.
- Neither ensemble improved final-evaluation performance over the best single model, BD-TCN at 86.16%.
4 Discussion
The study finds that comprehensive feature-based and end-to-end EEG pathology decoders achieve statistically similar performance, while drawing on similar temporal low-frequency information. The discussion emphasizes strong baselines, label noise, and incomplete feature coverage as important constraints and directions for evaluation.
- 4.1 Proposed feature-based pathology decoding concept: The feature-based pipeline tested 8633 features spanning 50 feature types and six domains, substantially exceeding the smaller feature sets commonly used in prior studies.
- 4.3 Feature-based pathology decoding accuracies: 85.87% was the best feature-based accuracy, achieved with Riemannian-geometry-based decoding and exceeding the previous 78.8% feature baseline by more than 7%.
- 4.5 Deep end-to-end versus feature-based decoding accuracies: 81–86% feature-based and 82–86% end-to-end accuracies formed narrow, highly similar ranges, with the EEG-optimized TCN reaching 86.16%.
- 4.5 Deep end-to-end versus feature-based decoding accuracies: No statistically significant accuracy differences were detected among the investigated feature-based models, neural networks, or their cross-approach comparisons.
- 4.5 Deep end-to-end versus feature-based decoding accuracies: Moderate inter-rater agreement introduced label noise, limiting theoretically achievable accuracy because evaluation used noisy labels without rater-independent ground truth.
- 4.7 Feature correlations: Strong feature correlations occurred across domains, locations, and frequency bands; T3 and T4 features correlated at approximately 0.9 in the 4–8 Hz band.
- 4.6 Feature visualizations: Temporal electrode features in the delta and theta ranges were informative for pathology decoding, and similar information was used by both pipelines.
5 Outlook and Conclusion
The outlook proposes addressing label-noise consequences through alternative decoding targets and anticipates automated EEG pipelines helping expand access to diagnostics and reduce neurological disease burden.
- 5 Outlook and Conclusion: The authors propose decoding patient gender and brain-age-related signals as alternatives for assessing pathology-related information despite label noise.These targets are intended to assess pipeline potential and use the gap between chronological and predicted brain age as an alternative pathology indication.
- 5 Outlook and Conclusion: A future pipeline incorporating proposed improvements could assist EEG interpretation and make diagnostics available to patients unable to attend specialized centers.
- 5 Outlook and Conclusion: The authors suggest such automated EEG diagnostics could help reduce the global burden of neurological diseases and disorders.
A Hyperparameters
The appendix reports hyperparameters for the investigated models to support reproducibility and lists configurations for feature-based and neural-network approaches.
- A Hyperparameters: The study reports all hyperparameters used for the investigated models to maximize reproducibility.
- A Hyperparameters: Feature-based approaches include RF, SVM, Riemannian SVM, covariance-matrix SVM, and ASC configurations.
- A Hyperparameters: Neural-network hyperparameters were optimized in previous experiments, while default values were used for BD-EEGNet.
B Performance differences to older experiments
The authors report marginal differences from older pathology-decoding experiments and attribute them to updates including duplicate removal between training and evaluation sets.
- B Performance differences to older experiments: 85.4% previously reported for BD-Deep4 became 84.57% in the present experiment.
- B Performance differences to older experiments: 84.5% previously reported for BD-Shallow became 84.13% in the present experiment.
- B Performance differences to older experiments: The authors associate these differences with dataset and experimental updates, including removing duplicates shared by training and final evaluation sets.
C Neural network parameters
The neural-network appendix compares architectures by parameter count and receptive-field size, reporting that greater model complexity was associated with higher accuracy in the analysis.
- C Neural network parameters: Models with more parameters achieved higher accuracies in the authors' analysis.
- C Neural network parameters: BD-TCN is the most complex investigated architecture and has the largest receptive field.
- C Neural network parameters: BD-EEGNet is significantly the simplest investigated model.
D Riemannian-geometry-based decoding
The analysis compares arithmetic and geometric covariance-matrix averaging, with and without tangent-space mapping, for Riemannian-geometry-based decoding. Geometric averaging and tangent-space mapping produced the best decoding performance.
- Using both tangent-space mapping and geometric-mean estimation yielded the best performance among the compared configurations.
- Geometric-mean aggregation yielded superior decoding accuracies to Euclidean-mean aggregation.
- Tangent-space mapping yielded substantial decoding improvements.
E Three model ensembles errors
The ensemble-error analysis evaluates combinations of three models by how their label errors overlap. BD-Deep4, RF, and RG were selected because exactly-one-model errors offered the greatest potential for reduction through majority voting.
- 44,56% was the highest ratio of cross-validation label errors made by exactly one model among the investigated three-model combinations.
- BD-Deep4, RF, and RG were chosen for ensembling based on their highest exactly-one-model error ratio.
- The error overview categorizes labels as errors made by none, exactly one, two, or all three models, with E1 errors representing potential majority-voting reductions.