PaperlayerView exact revision on arXiv

Source-linked AI summary

MEDFAIR: Benchmarking Fairness for Medical Imaging

Yongshuo Zong, Yongxin Yang, Timothy Hospedales

arXiv:2210.01725v2cs.LGcs.AIeess.IV

TL;DR

Medical-imaging fairness studies are difficult to compare because they differ in fairness metrics and experimental designs. MEDFAIR benchmarks algorithms across datasets and model-selection strategies, finding that selection strategy matters while state-of-the-art mitigation methods do not significantly outperform ERM in either in-distribution or out-of-distribution settings.

  • Problem

    Medical-imaging fairness studies use different metrics and experimental designs, making direct comparisons difficult, while fairness under dataset shift remains unclear.

    Source: §1 · p. 2

  • Method

    MEDFAIR benchmarks eleven algorithms across ten datasets, four sensitive attributes, and three model-selection strategies in in-distribution and out-of-distribution settings.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    State-of-the-art bias mitigation methods do not significantly outperform ERM in either in-distribution or out-of-distribution settings.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Fairness evaluation in medical applications should consider which definition fits the clinical context, since subgroup parity can trade off against utility.

    Source: §1 · p. 2, §2.2 · p. 4

    The benchmark is not exhaustive and focuses on image classification, leaving other modalities and tasks such as segmentation, regression, and detection underexplored.

    Source: §5 · p. 9

Abstract

from arXiv · show

A multitude of work has shown that machine learning-based medical diagnosis systems can be biased against certain subgroups of people. This has motivated a growing number of bias mitigation algorithms that aim to address fairness issues in machine learning. However, it is difficult to compare their effectiveness in medical imaging for two reasons. First, there is little consensus on the criteria to assess fairness. Second, existing bias mitigation algorithms are developed under different settings, e.g., datasets, model selection strategies, backbones, and fairness metrics, making a direct comparison and evaluation based on existing results impossible. In this work, we introduce MEDFAIR, a framework to benchmark the fairness of machine learning models for medical imaging. MEDFAIR covers eleven algorithms from various categories, nine datasets from different imaging modalities, and three model selection criteria. Through extensive experiments, we find that the under-studied issue of model selection criterion can have a significant impact on fairness outcomes; while in contrast, state-of-the-art bias mitigation algorithms do not significantly improve fairness outcomes over empirical risk minimization (ERM) in both in-distribution and out-of-distribution settings. We evaluate fairness from various perspectives and make recommendations for different medical application scenarios that require different ethical principles. Our framework provides a reproducible and easy-to-use entry point for the development and evaluation of future bias mitigation algorithms in deep learning. Code is available at https://github.com/ys-zong/MEDFAIR.

1 INTRODUCTION

MEDFAIR addresses the difficulty of assessing fairness in medical imaging amid differing fairness metrics and experimental designs. Its benchmark finds that model-selection strategy matters, while state-of-the-art bias mitigation methods do not significantly outperform ERM in either in-distribution or out-of-distribution settings.

1 INTRODUCTION

  • Medical-imaging models can show systematic performance bias across groups defined by race, gender, age, or health-insurance type.Such bias is socially and ethically detrimental in consequential healthcare settings.Source: §1 · p. 1
  • Existing medical-imaging studies are hard to compare because they use different fairness metrics, datasets, and model-selection strategies.It is also unclear whether fairness achieved in-distribution holds under dataset shift.Source: §1 · p. 2
  • Model-selection strategies can improve worst-case performance, so algorithms should be compared using the same criteria.Source: §1 · p. 3
  • State-of-the-art bias mitigation methods do not significantly outperform ERM in either in-distribution or out-of-distribution settings.The benchmark evaluates eleven algorithms across medical-imaging datasets and three model-selection strategies.Source: §1 · pp. 2–3
  • MEDFAIR is released as a reproducible codebase that researchers can run with one command and extend with additional algorithms, datasets, and strategies.Source: §1 · p. 3

2 FAIRNESS IN MEDICINE

The framework studies binary medical-image diagnosis across domains and evaluates fairness using both subgroup parity and worst-case subgroup utility. The authors distinguish in-distribution testing from deployment on an unseen domain and argue that the appropriate fairness definition depends on the clinical application.

2.1 PROBLEM FORMULATION

  • The framework evaluates binary diagnosis from images and considers one sensitive attribute at a time, even when a dataset contains several.Source: §2.1 · p. 3
  • In-distribution evaluation trains and tests on the same domain, while out-of-distribution evaluation tests a model selected in one domain on an unseen domain.The unseen domain shares the input space, diagnosis labels, and sensitive attributes but may have a different joint distribution.Source: §2.1 · p. 3

2.2 FAIRNESS DEFINITION IN MEDICINE

  • The authors argue that the fairness definition used for evaluation should depend on the clinical application.Source: §2.2 · p. 3
  • Group fairness measures parity in subgroup predictive performance, but parity can result from worsening advantaged-group performance and does not capture utility trade-offs alone.Source: §2.2 · pp. 3–4
  • Max-Min fairness treats a model as fairer when it improves the worst-case subgroup utility, which may suit some diagnosis applications.Source: §2.2 · p. 4

3 MEDFAIR

MEDFAIR provides a reproducible benchmark spanning medical-imaging datasets, bias-mitigation algorithms, model-selection strategies, and fairness evaluations. It evaluates methods under in-distribution and out-of-distribution settings using multiple metrics and statistical tests.

3 MEDFAIR

  • MEDFAIR benchmarks eleven algorithms across ten publicly available datasets, with up to three sensitive attributes per dataset and three model-selection criteria.The codebase and documentation support reproducing experiments and adding datasets or algorithms.Source: §3 · p. 4, §3.1 · p. 4

3.1 DATASETS

  • The ten datasets cover 2D and 3D imaging modalities, varied potential bias sources, sensitive attributes, dataset sizes, and paired settings for out-of-distribution evaluation.CheXpert and MIMIC-CXR provide a chest X-ray pair, while ADNI 1.5T and ADNI 3T provide a brain MRI pair.Source: §3.1 · pp. 4–5

3.2 ALGORITHMS

  • The benchmark re-implements eleven algorithms across subgroup rebalancing, domain-independence, adversarial training, disentanglement, and domain generalization.Examples include resampling, subgroup-specific classifiers with a shared encoder, and methods that reduce sensitive-attribute information in representations.Source: §3.2 · pp. 5–6

3.3 MODEL SELECTION

  • Three model-selection strategies compare overall validation performance, Pareto-optimal worst-case AUC, and distance to a subgroup-wise utopia point.Overall selection may favor majority-group performance, whereas Pareto selection chooses a front model with the best worst-case AUC.Source: §3.3 · p. 6

3.4 EVALUATION AND IMPLEMENTATION

  • Fairness is evaluated in-distribution and out-of-distribution using overall AUC, subgroup AUC gap, worst-case-group AUC, and additional classification metrics.Relative algorithm ranks are compared with Friedman and Nemenyi tests, using p < 0.05 for statistical significance; experiments use 2D or 3D ResNet-18 backbones and three random seeds.Source: §3.4 · pp. 6–7

4 RESULTS

Across in-distribution and out-of-distribution evaluations, no mitigation method significantly outperforms ERM, although model selection affects worst-case group performance. ERM-trained models also show subgroup performance gaps across datasets and sensitive attributes.

4.1 BIAS WIDELY EXISTS IN ML MODELS TRAINED IN DIFFERENT MODALITIES AND TASKS

  • ERM models show subgroup performance gaps across datasets and sensitive attributes, measured using AUC and underdiagnosis rates.The results extend systematic quantification of these gaps across modalities, diagnosis tasks, and sensitive attributes.Source: §4.1 · p. 7, Figure 3 · p. 8

4.2 MODEL SELECTION SIGNIFICANTLY INFLUENCES WORST-CASE GROUP PERFORMANCE

  • For worst-case AUC, Pareto selection has an average rank of around 1.5, significantly better than overall selection at around 2.5, without significantly worse overall AUC.The comparison uses ERM and the three model-selection strategies.Source: §4.2 · pp. 7–8, Figure 4 · p. 8

4.3 NO METHOD OUTPERFORMS ERM WITH STATISTICAL SIGNIFICANCE

  • No bias-mitigation method significantly outperforms ERM on any metric in either setting; some methods ranked above ERM in-distribution perform worse on unseen domains.SWAD ranks first for worst-case and overall AUC in both settings, but is still not significantly better than ERM.Source: Figure 5 · p. 8, §4.3 · p. 9

5 DISCUSSION

Bias reflects multiple interacting, measurable and unmeasurable confounders, so mitigation methods are not consistently effective across the benchmark. The discussion connects fairness trade-offs to application goals and notes that the evaluation is not exhaustive.

Source of bias

  • Bias can arise from interacting measured and unmeasured factors, including subgroup and disease-prevalence imbalance, imaging differences, spurious correlations, diagnostic difficulty, and labeler bias.Because these factors are difficult or impossible to disentangle, methods targeting one factor may not succeed across settings.Source: §5 · p. 9

Failure of the bias mitigation algorithms

  • Most mitigation algorithms are not consistently effective, potentially because methods targeting particular confounders may encounter other or mixed sources of bias.The discussion contrasts resampling for data imbalance with disentanglement for spurious correlations; it suggests SWAD's general robustness may benefit a broader range of confounders.Source: §5 · p. 9

Relation of domain generalization and fairness

  • Domain generalization and fairness differ in focus but share robustness to distribution changes; SWAD can improve all subgroup performance while enlarging subgroup gaps.The discussion suggests prioritizing Max-Min fairness for diagnosis and group fairness for zero-sum resource allocation, while noting that the choice depends on application scenarios.Source: §5 · p. 9

Are the evaluations enough for now?

  • The benchmark is not exhaustive: it omits some fairness methods and medical modalities, while segmentation, regression, and detection remain underexplored.The authors identify self-supervised learning and cardiology and digital pathology data as examples for future expansion.Source: §5 · p. 9

REPRODUCIBILITY STATEMENT

The supplementary material reviews reproducibility resources, related work, model-selection strategies, and in-processing bias mitigation methods. It motivates examining selection criteria and summarizes several mitigation categories.

REPRODUCIBILITY STATEMENT

  • The experiments use publicly available datasets and release code and documentation; running them required about 0.77 NVIDIA A100-SXM-80GB GPU years.The authors report preprocessing and hyperparameter details in appendices.Source: REPRODUCIBILITY STATEMENT · p. 10

A RELATED WORK

  • The related-work appendix reviews model selection, bias mitigation, fairness benchmarks, and domain-generalization methods and their relationship to fairness.Source: A RELATED WORK · p. 17

A.1 MODEL SELECTION STRATEGY IN FAIRNESS

  • Existing fairness studies use markedly different model-selection strategies, making direct comparison difficult and motivating an investigation of their influence.Table A1 summarizes the strategies adopted by recent work.Source: §A.1 · p. 17, Table A1 · p. 17

A.2 BIAS MITIGATION ALGORITHMS

  • MEDFAIR benchmarks in-processing methods, which aim to mitigate bias during model training, and selects representative methods from reviewed categories.The review describes preprocessing, post-processing, subgroup rebalancing, adversarial learning, and disentanglement approaches.Source: §A.2 · pp. 17–18
  • Reviewed methods include resampling for imbalanced data, adversarial training to reduce sensitive-attribute information or enforce fairness constraints, and disentanglement of sensitive and task-relevant representations.The review notes that data balancing has been found to improve worst-case group accuracy.Source: §A.2 · pp. 17–18

A.3 FAIRNESS BENCHMARKS

The appendix situates MEDFAIR among fairness benchmarks, describes interacting sources of bias, and relates fairness methods to domain generalization. It motivates broader benchmarking and examining fairness through domain generalization.

A.3 FAIRNESS BENCHMARKS

  • Unlike prior benchmarks on traditional models, tabular or synthetic data, or 3D shapes, MEDFAIR compares a wider range of algorithms across medical imaging modalities and investigates model selection.The authors identify Zhang et al. (2022) as a relevant chest-X-ray comparison whose conclusions may not generalize to other modalities or comprehensive method selections.Source: §A.3 · p. 18

A.4 SOURCE OF BIAS

  • Subgroup performance gaps may reflect data and class imbalance, label noise, spurious correlations, or inherent differences in diagnostic difficulty, and bias sources can correlate.The text notes that differing subgroup accuracy can accompany unequal sample counts; balanced data alone may not resolve difficulty differences, for which representative samples and improved imaging devices are also suggested.Source: §A.4 · p. 19

A.5 DOMAIN GENERALIZATION AND FAIRNESS

  • Fairness-promoting methods focus on known sub-populations, while domain generalization targets performance on unseen sub-populations; both seek robustness to distribution changes.The benchmark explores fairness from the domain-generalization perspective and includes GroupDRO, SAM, and SWAD among the selected methods.Source: §A.5 · pp. 19–20

B.1.1 DATASET

The appendix reports subgroup and class-label statistics and documents dataset processing, implementation, hyperparameter search, and benchmarked methods. Its procedures include modality-specific splits and backbones, Bayesian tuning, and validation-based early stopping.

B.1.1 DATASET

  • Tables A2–A5 summarize subgroup and class-label statistics, with percentages indicating prevalence and unhealthy-label rates.The datasets are publicly available, with access links provided in Table A6.Source: §B.1.1 · p. 21, Table A2 · p. 21, Table A3 · p. 21

B.1.2 DATA PREPROCESSING

  • Unless otherwise specified, data are split 80/10/10 for 2D datasets and 70/10/20 for 3D datasets, with dataset-specific task and preprocessing choices.Examples include excluding suspect labels for PAPILA, mapping HAM10000 diagnoses to benign or malignant, and treating age as a sensitive attribute for OL3I and OCT.Source: §B.1.2 · pp. 21–23

B.2.1 IMPLEMENTATION DETAILS

  • Experiments use ResNet-18 for 2D images and 3D ResNet-18 for 3D images, with ImageNet or Kinetics pretrained weights and modality-appropriate image processing.The study trained over 7,000 models using about 0.77 GPU year.Source: §B.2.1 · pp. 23–24

B.2.2 HYPER-PARAMETERS

  • Bayesian hyperparameter optimization searches 20 runs per method, dataset, and sensitive attribute; training uses SGD and early stopping when validation worst-case AUC fails to improve for five epochs.The authors report that the best learning rate is around 1e−4 for most methods, while SAM requires a higher rate depending on dataset.Source: §B.2.2 · p. 24

B.2.3 METHODS

  • Table A7 summarizes benchmarked methods by category and whether they accept or require sensitive attributes during training and testing.Source: §B.2.3 · p. 24

B.3 METRICS

The metrics assess predictive performance, calibration, and subgroup fairness, while minimax Pareto selection chooses among group-specific AUC trade-offs by prioritizing the worst-performing group.

B.3 METRICS

  • AUC measures binary-classification performance and is examined overall and by subgroup, with AUC gaps and worst-case AUC used to assess group and max-min fairness.The study uses “AUC” as shorthand for AUROC.Source: §B.3 · p. 24
  • ECE indicates group sufficiency, while BCE is the classification objective optimized by the models.The passage notes that high ECE may lead to a different optimal decision threshold.Source: §B.3 · p. 24
  • The evaluation reports overall and subgroup FPR and FNR, plus TPR at 80% TNR; its classification threshold follows the minimum F1 score.TPR at 80% TNR is the true-positive rate at a specified desirable true-negative rate.Source: §B.3 · p. 25
  • Equalized Odds averages Equality of Opportunity for negative and positive labels, capturing subgroup differences in both false-positive and true-positive rates.EqOpp0 and EqOpp1 are defined separately for y = 0 and y = 1.Source: §B.3 · p. 25

B.4 MINIMAX PARETO SELECTION

  • Minimax Pareto selection chooses the Pareto-front classifier with the highest worst-group AUC, balancing groups where improving one group’s AUC would hurt another’s.The example selects the front model with the best AUC for the disadvantaged group.Source: §B.4 · p. 26

C ADDITIONAL RESULTS

The additional-results section identifies tables covering ERM model-selection strategies, in- and out-of-distribution AUC results, and other evaluation metrics across datasets.

C ADDITIONAL RESULTS

D INCORPORATE NEW DATASETS AND ALGORITHMS IN MEDFAIR

MEDFAIR provides extension patterns for adding datasets and algorithms through base classes, while retaining reusable training and evaluation components.

D INCORPORATE NEW DATASETS AND ALGORITHMS IN MEDFAIR

D.1 ADDING NEW DATASETS

  • A new dataset is added by creating a file that inherits BaseDataset and supplies image transformation, labels, sensitive attributes, and an index.The example returns the image, label, sensitive attribute, and index.Source: §D.1 · p. 35

D.2 ADDING NEW ALGORITHM

  • A new algorithm can inherit BaseNet’s configuration and training, validation, and testing loops, then replace the training loop or loss as needed.The SAM example adds its method by re-implementing the training loop.Source: §D.2 · p. 35

E ANALYSIS OF SOURCE OF BIAS

The bias-source analysis finds that subgroup performance is not explained by observable data or class imbalance alone, and that methods targeting one factor may not outperform ERM.

E ANALYSIS OF SOURCE OF BIAS

  • In HAM10000, the worst-performing age-by-sex subgroup was not the smallest or most class-imbalanced, and resampling performed similarly to or worse than ERM.The resampling methods were also worse than SWAD; unrecorded skin type is offered as a possible additional bias source.Source: E ANALYSIS OF SOURCE OF BIAS · p. 36, Table A17 · p. 36
  • In MIMIC-CXR, the worst-performing subgroup was 60–80 non-White Female, but it was neither the smallest nor the most class-imbalanced subgroup.The analysis formed 20 subgroups from age, sex, and race.Source: E ANALYSIS OF SOURCE OF BIAS · p. 36, Table A18 · p. 37
  • MIMIC-CXR resampling slightly improved worst-case subgroup performance while decreasing overall performance, with label noise offered as a possible explanation.The authors suggest that upsampling may include more noisy labels from subgroups with more severe label noise.Source: E ANALYSIS OF SOURCE OF BIAS · pp. 36–37, Table A19 · p. 37
  • The authors conclude that medical imaging datasets contain correlated, multiple bias sources that can be difficult or impossible to disentangle, limiting methods focused on one factor.They note that SWAD, which lacks a specific bias-source assumption, is the most consistently high-ranked algorithm, and suggest general robustness may help across confounders.Source: E ANALYSIS OF SOURCE OF BIAS · p. 37

Related papers

Loading 2210.01725v2…