PaperlayerView exact revision on arXiv

Source-linked AI summary

How I failed machine learning in medical imaging -- shortcomings and recommendations

Gaël Varoquaux, Veronika Cheplygina

arXiv:2103.10292v2eess.IVcs.CVcs.LGstat.ML

TL;DR

Machine-learning research in medical imaging is extensive, but its growth does not inherently produce clinical progress. The paper reviews problems across datasets, evaluation, methods, and publication incentives, finding recurring biases and weak gains relative to evaluation noise. It recommends more transparent, critically assessed research practices.

  • Problem

    Despite extensive medical-imaging machine-learning research, benchmark performance may not bring practical clinical improvement.

    Source: §I · p. 1

  • Method

    The paper reviews prior research and its own analyses to examine failures across dataset choice, methods and evaluation, and publication practices, then discusses improvement steps.

    Source: §I · p. 1

  • Results

    The authors report that three of four studied challenges showed diminishing returns, alongside broader concerns about dataset choice, evaluation, and publication incentives.

    Source: A. Evaluation error is often larger… · p. 3, §VI · p. 6

  • Takeaways & Limitations

    The authors recommend steps to improve research practices and make medical-imaging machine learning more effective at addressing real-world medical challenges.

    Source: §I · p. 1

    Robust dataset construction or use of multiple data sources is recommended only when feasible, so existing datasets still require critical evaluation.

    Source: C. Let us build awareness of data… · p. 2

Abstract

from arXiv · show

Medical imaging is an important research field with many opportunities for improving patients' health. However, there are a number of challenges that are slowing down the progress of the field as a whole, such optimizing for publication. In this paper we reviewed several problems related to choosing datasets, methods, evaluation metrics, and publication strategies. With a review of literature and our own analysis, we show that at every step, potential biases can creep in. On a positive note, we also see that initiatives to counteract these problems are already being started. Finally we provide a broad range of recommendations on how to further these address problems in the future. For reproducibility, data and code for our analyses are available on \url{https://github.com/GaelVaroquaux/ml_med_imaging_failures}

I. INTRODUCTION

Machine learning offers clinical promise, but research volume and larger datasets do not by themselves ensure clinical progress. The paper reviews dataset and publication-lifecycle problems and recommends ways to improve research practice.

I. INTRODUCTION

  • The paper examines how research incentives can favor benchmark performance without practical clinical improvement, and reviews failures across the publishing lifecycle.Its analysis draws on previous research and its own analyses, then discusses steps intended to improve the situation.Source: §I · p. 1

II. IT’S NOT ALL ABOUT LARGER DATASETS

  • In Alzheimer’s imaging, larger cohorts did not improve accuracy on the clinically relevant progressive-versus-stable impairment task, though later studies improved on large, heterogeneous cohorts.The review reports this pattern across 478 studies compiled from six systematic reviews.Source: §II · p. 1, Figure 1 · p. 2

A. Datasets reflect an application only partly

  • Dataset bias arises when study data differ from the target population, and cross-source testing can reveal performance drops.Cohorts may omit patient and symptom diversity, while publications seldom report dataset demographics.Source: A. Datasets reflect an application… · p. 2
  • Imaging procedures can encode treatment clues, such as chest drains in pneumothorax images, while skin markings and labeling errors can also introduce spurious signals.Report-derived labels may incorrectly mark follow-up scans as negative when earlier findings are omitted.Source: A. Datasets reflect an application… · p. 2

B. Dataset availability distorts research

The section links dataset availability to which applications researchers study, recommends critical dataset assessment and documentation, and finds that evaluation noise often exceeds leaderboard gains.

B. Dataset availability distorts research

  • Lung imaging publications rose substantially in AI in 2016, while the corresponding lung-versus-breast topic prevalence in general oncology stayed relatively constant.The authors suspect Kaggle lung challenges around that time contributed to the disproportionate increase.Source: B. Dataset availability distorts… · p. 2, Figure 2 · p. 3

C. Let us build awareness of data limitations

  • Dataset choices shape trained models and conclusions; using multiple sources or robust datasets can help, while existing data can be checked for bias, hidden subgroups, and mislabeled instances.Relabeling a subset may be worthwhile when problems are found.Source: C. Let us build awareness of data… · pp. 2–3
  • The authors recommend documenting dataset characteristics and collection, detailing model limitations and training choices, and using cross-field meta-analyses to identify research opportunities.Contrasting clinically oriented and technical venues may reveal opportunities for machine-learning research.Source: C. Let us build awareness of data… · p. 3

A. Evaluation error is often larger than algorithmic improvements

  • Three of four medical-imaging challenges showed diminishing returns; in two, public–private differences exceeded top-10% gains, and lung-tumor segmentation showed a larger overfit than improvement.Only nerve segmentation showed substantial gains that were not overfits.Source: A. Evaluation error is often larger… · p. 3

B. Improper evaluation procedures and leakage

The section describes leakage risks in model evaluation and warns that common metrics may not reliably reflect objective rankings or practical clinical improvement.

B. Improper evaluation procedures and leakage

  • Leakage can make evaluation overoptimistic when feature selection precedes cross-validation or repeated measures from one person cross train–test splits.In the latter case, a model may recognize the patient rather than markers of the condition.Source: B. Improper evaluation procedures and… · p. 3
  • Overfitting by observer can persist despite cross-validating every method step when researchers adjust the method to improve observed cross-validation performance.The authors note this is harder to detect than other leakage and may help explain challenge overfitting.Source: B. Improper evaluation procedures and… · pp. 3–4

C. Metrics that do not reflect what we want

  • A review of 150 segmentation challenges found rankings sensitive to variants of the same metric, casting doubt on their objectivity.Image-similarity metrics can also be ineffective; scrambled images were shown to achieve high values [Rohlfing, 2011].Source: C. Metrics that do not reflect what we… · p. 4
  • Traditional metrics may miss calibration and practical outcomes, while robustness to out-of-distribution examples can matter for clinical use.The authors note that metric improvements do not necessarily translate into different clinical outcomes.Source: C. Metrics that do not reflect what we… · p. 4

D. Incorrectly chosen baselines

Poorly chosen baselines can mislead comparisons, while noisy experiments and weak statistical practice make it difficult to distinguish generalizable findings from chance.

D. Incorrectly chosen baselines

  • Recent baselines can be outdated, but omitting effective simple methods also misleads comparisons, as CNNs did not outperform SVMs for Alzheimer’s diagnosis.These are two distinct baseline failures: failing to reflect recent progress and overlooking simpler approaches.Source: D. Incorrectly chosen baselines · p. 4

E. Statistical significance not tested, or misunderstood

  • Experimental results vary with samples, random initialization, and hyperparameters, yet predictive-model benchmarking lacks widely adopted practices to separate noise from generalizable findings.Small samples can make measured prediction accuracy a distant estimate of real-life performance; leaderboard analysis suggests overfitting is less of an issue with test sets of at least several thousand.Source: E. Statistical significance not… · p. 4
  • Testing across multiple datasets can support statistical consensus, but such procedures require dozens of datasets and are seldom used; repeated reuse raises implicit multiple-comparison risk.The resulting risk is finding an algorithm that performs well by chance.Source: E. Statistical significance not… · p. 4
  • Only 6% of surveyed segmentation challenges analyzed chance, while 15% of 410 popular ACM papers used a statistical test.The passage reports these as separate surveys; the percentages describe different populations and should not be conflated.Source: E. Statistical significance not… · p. 4
  • Null-hypothesis tests neither establish no effect when results are nonsignificant nor prevent trivial effects from becoming significant with enough data; superiority testing is proposed as an alternative.The proposed test asks whether improvement exceeds a specified threshold.Source: E. Statistical significance not… · p. 4

F. Let us redefine evaluation

The section advocates stronger, clinically relevant evaluation and clearer reporting, with attention to test-set integrity, result variability, and meaningful improvement beyond benchmark wins.

F. Let us redefine evaluation

  • Benchmarks should prevent data leakage, document hyperparameter selection without test data, and use test sets large enough for statistical power.The passage recommends at least several hundred test samples, ideally thousands or more.Source: F. Let us redefine evaluation · p. 4
  • Evaluation should use diverse, preferably multi-institutional data, strong modern and historical or clinical baselines, and account for variability across choices and data sources.The data should represent patient diversity and disease heterogeneity, with explicit inclusion criteria; other cohorts can help establish external validity.Source: F. Let us redefine evaluation · pp. 4–5
  • Metrics should capture different aspects of the clinical problem and relate to clinical performance, with qualitative input from groups most affected by the application.The passage also recommends discussing results variability and statistical significance.Source: F. Let us redefine evaluation · p. 5
  • A tiny benchmark improvement is seldom useful; compare improvement with variation from sampling or random seeds, and analyze competing methods and failure cases rather than only selecting a winner.Blinded challenges with hidden test sets are described as mitigating the winner’s curse.Source: F. Let us redefine evaluation · p. 5

A. No incentive for clarity

  • Publication incentives for impressiveness can encourage unnecessary mathiness or suggestive language and leave out ablations or evaluation details, undermining reproducibility.The passage connects reproducibility both to reproducing exact results and to drawing the same conclusions.Source: A. No incentive for clarity · p. 5

B. Optimizing for publication

The paper links publication incentives to methodological and reporting problems, then recommends broader, more transparent publication practices and concludes that change requires shifts in research procedures, norms, and goals.

B. Optimizing for publication

  • Publication incentives can distort scientific aims: novelty pressure may encourage needless complexity, while state-of-the-art pressure can aggravate loopholes and HARKing.A comparison of 179 classifiers on 121 datasets found no statistically significant differences between the top methods [Fern´andez-Delgado et al., 2014]; unnecessary complexity can add technical debt [Sculley et al., 2015].Source: B. Optimizing for publication · p. 5
  • Positive results, including experimental flukes, are more likely to be published than negative findings; 97% of significance-testing papers among 410 most-downloaded ACM papers reported p-values below 0.05.The authors say it seems highly unlikely that only 3% of the initial working hypotheses were unconfirmed.Source: B. Optimizing for publication · p. 5

C. Let us improve our publication norms

  • Improve transparency by broadening metrics and evaluating methods beyond accuracy, including reproducibility, carbon footprint, costs, and relation to real-world patient outcomes.The authors also point to open datasets as a basis for collaborative work beyond the capacities of a single team.Source: C. Let us improve our publication norms · p. 5
  • Preregistration or registered reports review motivation and experimental setup before results are available; publication of negative results, replications, commentaries, and retrospectives can also help science progress.The passage notes that new data are seldom acquired in machine-learning studies, making preregistration challenging [Forde and Paganini, 2019, Cockburn et al., 2020].Source: C. Let us improve our publication norms · pp. 5–6

VI. CONCLUSIONS

  • The paper concludes that its strategies are not silver bullets and require changed procedures, norms, and goals to address the challenges it identifies across research.The stated aim is better patient health outcomes with less burden on the care system.Source: §VI · p. 6

S. Firestein. Failure: Why science is so successful. Oxford

The supplied passages consist of bibliography entries spanning medical-imaging machine learning, evaluation, reproducibility, and research publication practices.

S. Firestein. Failure: Why science is so successful. Oxford

  • Other entries address evaluation and statistical practice, including prediction evidence, testing, and overfitting.The cited titles include a review of evidence for prediction and a meta-analysis of overfitting.Source: S. Firestein. Failure: Why science is… · p. 8
  • The listed works also cover reproducibility, reporting, and machine-learning research culture.Titles include works on reproducibility taxonomies, reporting checklists, and troubling trends in machine-learning scholarship.Source: S. Firestein. Failure: Why science is… · pp. 7–8
  • The bibliography includes references on medical-imaging datasets, dataset bias, and hidden subgroups.These topics appear in titles concerning dataset preparation, dataset bias, and hidden stratification.Source: S. Firestein. Failure: Why science is… · pp. 7, 9

A. Literature popularity review methods

The review estimates the relative popularity of breast- and lung-cancer imaging studies by querying medical and AI research literature and normalizing publication counts within each category.

A. Literature popularity review methods

B. Details on Kaggle challenges studied

The analysis compares Kaggle challenge performance with leaderboard variation, extracting public–private differences and the gap between the best and top-10% algorithms. Table I describes the challenges’ test-image counts and private-leaderboard entries.

B. Details on Kaggle challenges studied

Loading 2103.10292v2…