Source-linked AI summary
Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls
Farhad Maleki, Katie Ovens, Rajiv Gupta, Caroline Reinhold, Alan Spatz, Reza Forghani
TL;DR
The paper examines methodological errors that produce deceptively strong internal evaluations despite poor model generalizability. Using empirical examples, it shows that data leakage, inappropriate evaluation choices, and batch effects can yield overoptimistic performance estimates that internal evaluation may not detect.
Problem
Machine-learning models often lack generalizability despite promising internal evaluation, hindering reliable clinical deployment.
Method
The study empirically examines independence violations, performance-indicator choices, and batch effects in machine-learning model development.
Results
The examined pitfalls produced superficially high internal performance, including high segmentation metrics despite obvious lung-segmentation flaws.
Takeaways & Limitations
Internal evaluation alone cannot detect these pitfalls, so awareness and avoidance are important for developing generalizable medical machine-learning models.
Takeaways & Limitations
Small sample sizes and unidentified covariates can limit evaluation reliability, and stratified splitting cannot address the latter.
Abstract
from arXiv · showhide
Purpose: Despite the potential of machine learning models, the lack of generalizability has hindered their widespread adoption in clinical practice. We investigate three methodological pitfalls: (1) violation of independence assumption, (2) model evaluation with an inappropriate performance indicator or baseline for comparison, and (3) batch effect. Materials and Methods: Using several retrospective datasets, we implement machine learning models with and without the pitfalls to quantitatively illustrate these pitfalls' effect on model generalizability. Results: Violation of independence assumption, more specifically, applying oversampling, feature selection, and data augmentation before splitting data into train, validation, and test sets, respectively, led to misleading and superficial gains in F1 scores of 71.2% in predicting local recurrence and 5.0% in predicting 3-year overall survival in head and neck cancer as well as 46.0% in distinguishing histopathological patterns in lung cancer. Further, randomly distributing data points for a subject across training, validation, and test sets led to a 21.8% superficial increase in F1 score. Also, we showed the importance of the choice of performance measures and baseline for comparison. In the presence of batch effect, a model built for pneumonia detection led to F1 score of 98.7%. However, when the same model was applied to a new dataset of normal patients, it only correctly classified 3.86% of the samples. Conclusions: These methodological pitfalls cannot be captured using internal model evaluation, and the inaccurate predictions made by such models may lead to wrong conclusions and interpretations. Therefore, understanding and avoiding these pitfalls is necessary for developing generalizable models.
Materials and Methods
The study uses multiple retrospective imaging datasets and modalities to examine methodological pitfalls affecting machine-learning generalizability.
- Multiple imaging modalities were used to show that the methodological pitfalls are not specific to one data modality.
- HNSCC CT dataset: The HNSCC dataset contains pre-treatment CT scans from 137 patients treated with radiotherapy.
- Digital Histopathology Dataset: The lung histopathology dataset includes 110 relatively balanced slides with solid or acinar predominant patterns selected from 143 whole-slide images.
- Digital Histopathology Dataset: Histopathology slides were downscaled, foreground-extracted, and partitioned into 22,000 tissue patches of 1024 by 1024 pixels.
- Chest X-ray datasets: The batch-effect analysis uses 8,851 normal adult chest X-rays from RSNA and 1,349 normal plus 3,883 pneumonia pediatric X-rays from Kermany et al.
Experiments
The experiments compare models built with and without independence violations, inappropriate evaluation choices, and batch effects across medical imaging tasks. These pitfalls produced superficial performance gains or poor external performance despite strong internal metrics.
- (1) Violation of independence assumption: Incorrect oversampling before splitting produced a statistically significant but superficial performance increase for local-recurrence prediction, whereas post-split oversampling performed poorly.The Wilcoxon Rank Sum test reported statistic=-12.22 with p-value <0.001.
- (1) Violation of independence assumption: 46.0% F1 improvement resulted when augmentation preceded splitting, while the correctly split model showed poor performance in lung histopathology classification.Model C augmented before splitting; model D augmented after splitting.
- (1) Violation of independence assumption: Randomly distributing patient-level data across splits produced a 21.8% superficial F1 increase compared with assigning each patient entirely to one split.The patient-level split preserved independence, while the random split allowed the same patient's data points into training and test sets.
- (1) Violation of independence assumption: Feature selection before splitting caused a statistically significant superficial boost in 3-year overall-survival prediction compared with feature selection after splitting.The Wilcoxon Rank Sum test reported statistic=-5.87 with p-value<0.001.
- (2) Performance indicators and baselines: A naive classifier labeling every sample as non-distant metastasis achieved 94% accuracy but zero recall, demonstrating why accuracy can misrepresent clinical utility in imbalanced data.The example shows that a high accuracy value can coexist with failure to diagnose any distant metastasis.
- (2) Performance indicators and baselines: A simple air-detection segmentation baseline achieved Dice 0.94 and IoU 0.88 despite medically unacceptable lung segmentation, showing that overlap metrics require an appropriate baseline and visual assessment.The prediction included air in the trachea and bowel gas, and models should outperform this computationally efficient but medically unreliable baseline.
- (3) Batch effect: 98.7% F1 on batch-effect data fell to 3.86% correct classification on normal pediatric X-rays, while attributions emphasized anatomy and body position rather than lung findings.The model achieved accuracy 0.997, precision 0.979, recall 0.995, and F1 0.987 on the batch-effect dataset.
Discussion
Methodological errors can produce deceptively strong internal results while undermining model generalizability. The paper emphasizes avoiding leakage, choosing clinically appropriate metrics, and recognizing limits from data scope and distribution shift.
- Oversampling before data splitting can place identical samples in training and test sets, producing superficially high performance.The correct approach produced poor results for the HNSCC dataset, consistent with its very small number of local recurrences.
- Applying data augmentation before splitting distributes highly correlated samples across sets, potentially inflating internal performance while harming external performance.Augmented images retain many characteristics of their originals, allowing near-duplicates to reappear during testing or validation.
- Distributing patches from one patient across training, validation, and test sets can artificially boost performance and reduce external generalizability.Patient-level separation is needed because patches may share characteristics unrelated to the study goal.
- Feature selection before splitting exposes test-set information and can select features that perform well on the test set but poorly on unseen data.The resulting superficial performance boost can degrade on unseen data and reduce generalizability.
- Performance indicators should reflect predictive utility and misclassification costs, because accuracy alone can mislead for rare or high-risk conditions.The discussion highlights recall for life-threatening conditions and precision when false positives could trigger invasive procedures.
- Segmentation metrics can appear strong despite visibly flawed outputs, supporting visual inspection and caution against pixel-level accuracy for small regions.The example reports IoU 0.88 and Dice 0.94 despite obvious segmentation flaws.
- The study does not cover sample size, other model architectures, data quality, bias, explainability, or responsible AI considerations.The authors identify these as additional challenges or scope boundaries requiring other literature or future empirical analysis.
- Internal evaluation can miss methodological pitfalls, yielding overoptimistic performance estimates and non-generalizable models.The authors argue that such errors may be difficult to detect from reported methodological details and can define erroneous state-of-the-art models.
Appendix A
The appendix describes radiomics feature extraction, stratified nested cross-validation, and feature-selection procedures for HNSCC analysis.
- Appendix A: Radiomics analysis extracted 1652 tumor features using the pyradiomics package.Features included shape, first-order statistics, and multiple texture-matrix categories.
- Appendix A: Data were stratified into train, validation, and test sets using 3-fold outer and 3-fold inner nested cross-validation.Oversampling was applied to training samples to address class imbalance.
- Appendix A: Feature selection removed constant variables, selected high-scoring features, and applied recursive elimination to reduce dimensionality.The workflow progressively narrowed the radiomic feature set before model construction.
Appendix B
The appendix presents a ResNet-50 image-classification pipeline using dropout, cross-entropy loss, Adam optimization, and image augmentation.
- Appendix B: A pretrained ResNet-50 was adapted into a binary classifier by replacing its classifier layer and adding dropout with probability 0.5.The architecture used ImageNet pretraining and a dropout layer between the backbone and classifier.
- Appendix B: Models were trained with cross-entropy loss and Adam optimization at a learning rate of 0.0001.The implementation used the specified optimizer settings throughout the experiments.
- Appendix B: Models were trained for 100 epochs, selecting the model with the lowest loss, while Albumentations supplied image augmentation.Experiments used Python 3.7, PyTorch 1.6, and a Titan RTX GPU.
- Appendix B: Figure 6 schematically illustrates the deep-learning pipeline for image classification.The figure provides a visual overview of the image-analysis workflow.
A. Dice score is always larger than or equal to IOU: a mathematical proof
This appendix proves mathematically that the Dice score is always greater than or equal to the intersection over union for arbitrary sets.
- A. Dice score is always larger than or equal to IOU: Dice(A,B) is shown to be greater than or equal to IoU(A,B) for arbitrary sets A and B.The proof begins by stating the target inequality and then manipulates the set-size expressions.
- A. Dice score is always larger than or equal to IOU: The derivation uses set cardinalities and multiplies the inequality by the positive quantity |A∩B|.Subsequent algebraic transformations preserve the inequality.
- A. Dice score is always larger than or equal to IOU: Adding matching terms to both sides leads to the final relationship between the Dice and IoU expressions.The concluding algebra uses the identity involving |A|, |B|, and |A∩B|.
B. Checklist for avoiding methodological pitfalls covered in this paper
The appendix references a guideline table for avoiding the methodological pitfalls examined in the paper.
- B. Checklist for avoiding methodological pitfalls covered in this paper: Table 4 presents guidelines for avoiding the methodological pitfalls covered in this paper.The supplied table passage identifies its purpose but does not provide the individual checklist items.
C. The pipeline for radiomic analysis
The radiomic-analysis pipeline separates patient data into training, validation, and test sets before extracting radiomic features.
- C. The pipeline for radiomic analysis: Patient images are manually contoured, assigned exclusively to one data split, and then processed for radiomic feature extraction.Figure 7 describes image acquisition, tumor-volume annotation, patient-level splitting, and subsequent feature extraction with the Pyradiomics Python package.