PaperlayerView exact revision on arXiv

Source-linked AI summary

FairCLIP: Harnessing Fairness in Vision-Language Learning

Yan Luo, Min Shi, Muhammad Osama Khan, Muhammad Muneeb Afzal, Hao Huang, Shuaihang Yuan, Yu Tian, Luo Song, Ava Kouhana, Tobias Elze, Yi Fang, Mengyu Wang

arXiv:2403.19949v2cs.CV

TL;DR

Fairness in medical vision-language models remains unclear, in part because suitable datasets are scarce. The authors introduce Harvard-FairVLMed and FairCLIP, reporting significant subgroup biases across models and improvements in performance and fairness over CLIP.

  • Problem

    Fairness in medical vision-language models remains unclear, while available datasets have limited demographic detail, potentially noisy labels, and narrow clinical text.

    Source: §1 · p. 2

  • Method

    The authors introduce Harvard-FairVLMed and use optimal transport in FairCLIP to align overall sample distributions with demographic-group distributions.

    Source: §1 · p. 2

  • Results

    All evaluated VL models showed significant biases, with Asian, Male, Non-Hispanic, and Spanish subgroups preferred for race, gender, ethnicity, and language, respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that FairCLIP balances performance and fairness while Harvard-FairVLMed supports fairness study of medical VL models.

    Source: §6 · p. 8

Abstract

from arXiv · show

Fairness is a critical concern in deep learning, especially in healthcare, where these models influence diagnoses and treatment decisions. Although fairness has been investigated in the vision-only domain, the fairness of medical vision-language (VL) models remains unexplored due to the scarcity of medical VL datasets for studying fairness. To bridge this research gap, we introduce the first fair vision-language medical dataset Harvard-FairVLMed that provides detailed demographic attributes, ground-truth labels, and clinical notes to facilitate an in-depth examination of fairness within VL foundation models. Using Harvard-FairVLMed, we conduct a comprehensive fairness analysis of two widely-used VL models (CLIP and BLIP2), pre-trained on both natural and medical domains, across four different protected attributes. Our results highlight significant biases in all VL models, with Asian, Male, Non-Hispanic, and Spanish being the preferred subgroups across the protected attributes of race, gender, ethnicity, and language, respectively. In order to alleviate these biases, we propose FairCLIP, an optimal-transport-based approach that achieves a favorable trade-off between performance and fairness by reducing the Sinkhorn distance between the overall sample distribution and the distributions corresponding to each demographic group. As the first VL dataset of its kind, Harvard-FairVLMed holds the potential to catalyze advancements in the development of machine learning models that are both ethically aware and clinically effective. Our dataset and code are available at https://ophai.hms.harvard.edu/datasets/harvard-fairvlmed10k.

1. Introduction

Fairness in medical vision-language models remains unclear, while existing datasets limit broad study through sparse demographics, potentially noisy labels, and narrow clinical text. The authors introduce Harvard-FairVLMed, analyze subgroup disparities, and propose FairCLIP to improve fairness and performance.

1. Introduction

  • Fairness in medical vision-language models remains unclear, and available datasets constrain study through limited demographics, potentially noisy labels, and unrepresentative clinical text.Existing public vision-language datasets primarily focus on chest X-rays and use labels automatically extracted from reports.Source: §1 · pp. 1–2
  • Harvard-FairVLMed provides 10,000 patients’ SLO images, clinical notes, ground-truth labels, and fine-grained demographic attributes for medical vision-language fairness research.Its notes include non-imaging information such as medication, test results, and family history.Source: §1 · p. 2
  • The authors motivate fairness work in glaucoma because undiagnosed disease is more pronounced among minority groups; one cited study reports Black individuals are 4.4 times more likely than White individuals to have undiagnosed and untreated glaucoma.Timely detection is important to avoid irreversible vision loss.Source: §1 · p. 2
  • FairCLIP uses optimal transport to align the overall sample distribution with distributions for demographic groups, and the authors report improved performance and fairness over CLIP.It optimizes Sinkhorn distance to address the identified disparities.Source: §1 · p. 2
  • The analysis finds significant biases across race, gender, ethnicity, and language, with Asian, Male, Non-Hispanic, and Spanish subgroups preferred, respectively.The authors examine CLIP and BLIP2 models pretrained on natural and medical domains.Source: §1 · p. 2

2. Related Work

Prior work addresses fairness in machine learning and develops vision-language models for medical tasks, but medical VL fairness remains underexplored. Existing datasets often lack the demographic, label, and textual detail needed for comprehensive fairness studies.

2. Related Work

  • Medical fairness methods target different stages: pre-processing changes data, in-processing incorporates fairness during training, and post-processing adjusts model outputs.More broadly, methods also seek to remove or neutralize sensitive features and promote fair representations.Source: §2 · p. 2
  • Medical vision-language research includes adapted CLIP models and systems for report generation, diagnosis, and image-based question answering.Examples include PLIP, which uses pathology images paired with language descriptions, and PMC-CLIP for biomedical applications.Source: §2 · p. 3
  • Fairness in medical vision-language models remains largely unexplored, despite the value of paired imaging and textual reports for examining diverse patient cases.The visual-textual interplay presents distinct fairness challenges.Source: §2 · p. 3
  • Chest X-ray datasets pair images with reports but may have noisy labels and limited demographic data, while radiology reports provide less patient-specific information than richer clinical notes.CheXpert labels are automatically extracted from reports; 16% of its Atelectasis labels are uncertain.Source: §2 · p. 3
  • Existing fundus and glaucoma fairness datasets lack the textual information needed for vision-language fairness studies.Harvard-FairVLMed is presented as a resource with ground-truth labels and broad identity attributes, including age, gender, race, ethnicity, language, and marital status.Source: §2 · p. 3

3. Dataset Analysis

Harvard-FairVLMed pairs SLO fundus images and ophthalmologist notes with demographic attributes and glaucoma diagnoses. The dataset is de-identified, and its 10,000 subjects are divided into training, validation, and test sets.

3. Dataset Analysis

  • The study reports adherence to the Declaration of Helsinki, institutional review board approval, and de-identification of all dataset records.These statements describe the study’s ethical and privacy safeguards.Source: §3 · p. 3

3.1. Data Collection and Quality Control

  • Harvard-FairVLMed contains SLO images, six demographic attributes, and de-identified ophthalmologist notes for glaucoma-care subjects seen at Massachusetts Eye and Ear from 2015 to 2022.Subjects are categorized as non-glaucoma or glaucoma using visual-field test criteria.Source: §3.1 · p. 4

3.2. Protected Information De-Identification

  • Clinical notes were de-identified using Microsoft Presidio, additional rules, and manual review by four medical experts.Experts checked every note and replaced sensitive information with placeholders when necessary.Source: §3.2 · p. 4

3.3. Data Characteristics

  • The dataset contains 10,000 samples split into 7,000 training, 1,000 validation, and 2,000 test samples; its mean age is 60.9 ± 16.2 years.The sample includes 819 Asian, 1,491 Black, and 7,690 White participants; 90.6% are Non-Hispanic and 92.5% prefer English.Source: §3.3 · p. 4

4. Methodology

The method starts from CLIP’s contrastive learning of paired image and clinical-note features, then introduces FairCLIP to reduce disparities between overall and demographic-group similarity distributions. FairCLIP estimates those distributions from batches and adds Sinkhorn distance to CLIP’s pre-training loss.

4.1. Background

  • CLIP learns from batches of paired images and clinical notes, maximizing cosine similarity for matched pairs and minimizing it for unmatched pairs.A symmetric cross-entropy loss over the similarity matrix optimizes the vision and text encoders.Source: §4.1 · p. 4

4.2. FairCLIP

  • FairCLIP minimizes the distance between overall and demographic-group distributions of image–text feature correlations to improve fairness during pre-training.The objective is defined over each attribute group, such as a racial group.Source: §4.2 · pp. 4–5
  • Clinical notes undergo de-identification and summarization before text encoding to address PHI and encoder length limits, including CLIP’s 77-token maximum.The schematic describes this preprocessing alongside distribution equalization.Source: Figure 2 · p. 5
  • FairCLIP approximates the overall and group-specific distributions with empirical distributions from batch B because the underlying distributions are computationally intractable.B_a contains the batch samples belonging to group a.Source: §4.2 · p. 5
  • FairCLIP uses Sinkhorn distance rather than KL divergence and adds the Sinkhorn loss to CLIP’s pre-training loss.The stated Sinkhorn formulation combines transport cost with relative-entropy regularization.Source: §4.2 · p. 5

5. Experiment & Analysis

CLIP and BLIP2 show demographic performance disparities, and medical pretraining generally improves the performance-fairness trade-off. FairCLIP improves fairness and performance metrics over CLIP, while ablation results vary by attribute and model choice.

5.1. Experimental Setup

  • Models were evaluated with linear probing and zero-shot transfer using AUC, ES-AUC, group-wise AUC, DPD, and DEOdds.The analysis uses official natural-domain checkpoints and medical-domain pretraining initialized from those checkpoints.Source: §5.1 · pp. 5–6, Table 2 · p. 6

5.2. VL Fairness Analysis

  • 73.47 versus 69.57 average ES-AUC marks ethnicity as the best and language as the worst trade-off; Asian, male, non-Hispanic, and Spanish are preferred subgroups.The averages are across four VL models; Black patients have the lowest racial-subgroup performance and female patients are diagnosed less well than male patients.Source: §5.2 · pp. 6–7
  • Medical pretraining improves the trade-off across attributes except language, with up to 8.38 ES-AUC gain on ethnicity and gains of up to 6.75 for White and 9.65 for Hispanic subgroups.The largest reported subgroup improvements are for White and Hispanic patients, respectively.Source: §5.2 · p. 6
  • Natural-pretrained CLIP exceeds BLIP2 (77.27 vs. 73.81 AUC), while medical pretraining brings BLIP2 close to CLIP (80.10 vs. 80.27 AUC).Among medical-pretrained models, CLIP has a better trade-off on race and gender, while BLIP2 does on ethnicity and language.Source: §5.2 · p. 7

5.3. CLIP vs. FairCLIP

  • FairCLIP significantly improves DPD, DEOdds, ES-AUC, and AUC over CLIP across demographic subgroups.The zero-shot comparison reports results across two architectures and four protected attributes.Source: §5.3 · p. 7, Table 3 · p. 7

5.4. Ablation Studies

  • Summarized notes improve ES-AUC except on language; MED42 performs best on race, GPT-4 on gender, and PMC-LLAMA on ethnicity and language.The study compares these summaries with the original clinical notes for BLIP2.Source: §5.4 · pp. 7–8, Figure 3 · p. 8
  • Multimodal BLIP2 features improve ES-AUC across attributes except language, most on race; PMC-CLIP encoders outperform natural-domain CLIP across all four attributes.These ablations compare vision-only with vision-plus-language features and medical with natural-domain vision encoders.Source: §5.4 · p. 8, Table 4 · p. 8, Figure 3 · p. 8
  • Adversarially trained CLIP does not consistently outperform standard CLIP, whereas FairCLIP consistently outperforms CLIP.The authors contrast adversarial loss with FairCLIP’s Sinkhorn loss, which encourages uniformity between overall and group-specific sample distributions.Source: §5.4 · p. 8, Figure 3 · p. 8

6. Conclusion

The paper introduces Harvard-FairVLMed for studying fairness in medical vision-language models, reports significant biases across models, and proposes FairCLIP to balance fairness and performance.

6. Conclusion

  • Harvard-FairVLMed is introduced as the first fair medical vision-language dataset for studying fairness in medical vision-language foundation models.The dataset and code are publicly available.Source: §6 · p. 8
  • The paper's fairness analysis on Harvard-FairVLMed reveals significant biases in all evaluated vision-language models.Source: §6 · p. 8
  • FairCLIP uses optimal transport to address these biases while balancing model performance and fairness.Source: §6 · p. 8

7. Experimental Setup

The experiments compare natural-checkpoint and medical-pretrained CLIP and BLIP2 variants, with FairCLIP integrating a Sinkhorn loss into CLIP. Evaluation uses fairness, performance, and combined metrics.

7.1. Pre-Training

  • Natural variants use official CLIP and BLIP2 checkpoints, while medical variants are pretrained on Harvard-FairVLMed; FairCLIP adds Sinkhorn loss to CLIP's original loss.The reported FairCLIP configuration uses batch size 32 per group and Sinkhorn-loss weight 1e-7.Source: §7.1 · p. 14

7.2. Metrics

  • Evaluation uses DPD and DEOdds for prediction fairness, alongside AUC, Equity-Scaled AUC (ES-AUC), and group-wise AUC for performance and group disparities.The passage characterizes AUC as a mainstream medical performance metric and group-wise AUC as a way to reveal between-group discrepancies.Source: §7.2 · p. 14
  • ES-AUC divides AUC by 1 plus the summed absolute gaps between overall and subgroup AUCs, providing a combined performance-and-equity measure.Higher ES-AUC indicates greater performance alongside improved model equity; groups are defined by race, gender, ethnicity, or preferred language.Source: §7.2 · p. 14

8. Results

End-to-end fine-tuning comparisons show FairCLIP's results vary across protected attributes and metrics, while supplementary analyses examine dataset composition and ablation settings.

8.1. CLIP vs. FairCLIP

  • FairCLIP improves gender DPD and DEOdds; racial results vary, with CLIP ViT-B/16 DPD 5.85 versus FairCLIP ViT-B/16 DPD 11.38.FairCLIP ViT-B/16 AUC is 81.88, and FairCLIP ViT-L/14 has higher racial group-wise AUC for Asian and Black groups.Source: §8.1 · p. 14, Table 5 · p. 15
  • FairCLIP ViT-L/14 achieves Hispanic-group ES-AUC 79.08 and “Others” language-group ES-AUC 74.44, with higher English and Spanish AUC and group-wise AUC than CLIP.Source: §8.1 · p. 14, Table 5 · p. 15

8.2. Dataset Analysis

  • Supplementary dataset analysis presents clinical-note word distributions and subject prevalence across race and gender.Figure 4 shows the word distribution and race- and gender-based prevalence plots.Source: §8.2 · p. 15, Figure 4 · p. 15

8.3. Ablation Studies

  • The FairCLIP ablation reports that a group batch size |Ba| = 32 achieves desired performance.Source: §8.3 · p. 15, Figure 5 · p. 15
  • Additional ablations cover note summarization, vision-only versus multimodal features, natural versus medical vision encoders, epsilon, marital status, and competing fairness algorithms.These are reported in Tables 6–8 and Figure 5.Source: §8.3 · p. 15, Figure 5 · p. 15, Table 6 · p. 16

Related papers

Loading 2403.19949v2…