PaperlayerView exact revision on arXiv

Source-linked AI summary

FRAME: separating sampling variation from representational cause in medical imaging fairness

Mahshad Lotfinia, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh

arXiv:2608.25981v1cs.CVcs.AIcs.LG

TL;DR

Medical imaging fairness research often interprets subgroup performance gaps as model bias because representations encode demographics, but the contribution of sampling variation and mechanisms remains uncertain. FRAME estimates the gap expected under exact fairness at observed subgroup sizes and tests whether representation interventions alter the remainder. Across encoders and published studies, the reference explains substantial portions of reported differences, while demographic decodability does not change the remainder and image-text pretraining improves worst-group performance.

  • Problem

    Medical imaging fairness studies commonly use subgroup performance differences as evidence of bias, although the roles of sampling variation and demographic encoding in those differences have not been quantified or tested by intervention.

    Source: Introduction · pp. 1–2

  • Method

    FRAME simulates a fair-model reference at observed subgroup counts, subtracts it from the reported difference, and tests candidate representation-space causes of the remainder.

    Source: Introduction · p. 2, Auditing published subgroup differences · p. 27

  • Results

    The reference accounts for a median 41% of reported race differences, 22% of age differences, 25% of rate differences, and 70% of AUROC differences; demographic decodability leaves the remainder unchanged, while image-text pretraining raises worst-group AUROC by about 0.05.

    Source: Introduction · p. 2, Discussion · p. 19

  • Takeaways & Limitations

    Applying FRAME before selecting an intervention can distinguish differences compatible with sampling variation at current cohort sizes from findings requiring a mechanistic explanation.

    Source: Discussion · pp. 19–20

    The mechanism behind the remainder is not identified because the interventions act on cached features rather than during pretraining, and the pooled race contrast is partly a site contrast.

    Source: Discussion · p. 20

Abstract

from arXiv · show

Subgroup performance differences are the standard evidence for fairness bias in medical imaging, and the usual response removes the demographic information that a model encodes. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME), a two-step framework for auditing such a claim. The first step derives a fair-model reference, the distribution of the difference under exact fairness at the observed subgroup sizes. In the second step, we test the remainder with two operators in representation space. One operator cannot change a within-group ranking by construction. Across 702,206 images and 36 encoders, the reference accounts for a median 41% of the reported race difference and 22% of the age difference. Injecting demographic decodability leaves the remainder unchanged, while entangling the group with the disease direction raises the race difference from 0.077 to 0.118. No intervention we tested changes the remainder more than a change of random seed does. Those interventions reduce a difference at the operating point and leave the within-group ranking difference at a median of 0.000. Applied to 89 differences in 9 published studies across 6 medical imaging modalities, the reference accounts for a median 25% of a rate difference and 70% of a difference in the area under the receiver operating characteristic curve. Image-text pretraining instead raises worst-group performance by about 0.05. Applying FRAME before choosing an intervention could distinguish differences that need a mechanistic explanation from differences compatible with sampling variation at the current cohort sizes.

Introduction

Medical imaging fairness work commonly treats subgroup performance differences as evidence of unfairness and responds by removing encoded demographic information. FRAME audits this reasoning by separating sampling variation at observed subgroup sizes from mechanisms that could affect the remaining difference.

Motivation

  • Medical imaging models show unequal performance across race, sex, insurance, and skin-tone groups, motivating model-level fairness interventions.Reported examples include underdiagnosis and AUROC gaps across diagnostic tasks.Source: Introduction · p. 1
  • The standard response assumes that demographic encoding causes subgroup differences and that removing this encoding will equalize performance.Mitigations range from subgroup thresholds and constrained losses to worst-group objectives, adversarial classifiers, and concept erasure.Source: Introduction · pp. 1–2

Open problem

  • Existing evidence leaves the causal mechanism untested, while task difficulty, confounding, correction scope, and small-sample statistics can also produce unequal performance.The maximum-minus-minimum statistic rises with sampling variation, and within-group AUROC changes only when score ordering changes within a subgroup.Source: Introduction · p. 2

FRAME

  • FRAME first simulates the difference expected under exact fairness at observed subgroup counts, then tests whether representation-space interventions change the remainder.One operator cannot change within-group rankings by construction; the framework also evaluates encoders across objectives, backbones, pretraining composition, and seeds.Source: Introduction · p. 2

Key findings

  • The fair-model reference accounts for a median 41% of reported race differences and 22% of age differences across the study’s evaluations.In published studies, it accounts for a median 25% of rate differences and 70% of AUROC differences.Source: Introduction · p. 2

Results

FRAME shows that subgroup differences can partly arise from sampling variation, while representation-space interventions distinguish demographic decodability from disease-direction entanglement. Across modalities and interventions, operating-point changes can reduce rate disparities without changing within-group rankings, whereas erasure methods trade disparity reduction against disease AUROC.

Step one of FRAME: the reference against the reported difference

  • 41% of the reported race difference and 22% of the age difference are accounted for by the fair-model reference.The observed race difference has median 0.082 against a reference of 0.034; age has 0.066 against 0.014.Source: Step one of FRAME: the reference… · p. 4
  • 79 of 130 race combinations and 116 of 130 age combinations significantly exceed their fair-model references.Across the full study, the corresponding counts are 129 of 243 for race and 226 of 252 for age.Source: Step one of FRAME: the reference… · p. 4

Step two of FRAME: candidate causes of the remainder

  • Demographic decodability does not predict the achievable difference, with linear-decoding correlation 0.036 and nonsignificant out-of-sample predictors.The cost of erasure and geometric overlap likewise show nonsignificant correlations of −0.142 and −0.027.Source: Step two of FRAME: candidate causes of… · pp. 4, 10
  • Injecting decodability raises linear race decodability from 0.866 to 0.942 while the race difference stays at 0.077, whereas entanglement raises it from 0.077 to 0.118.The decodability injection leaves the race difference unchanged; entanglement with the disease direction increases it.Source: Step two of FRAME: candidate causes of… · p. 10

The remainder under every intervention

  • Retraining changes the race difference by a median 0.012, compared with a median 0.005 reduction from nine mitigation methods.No trend across unfreezing levels is significant for any attribute or encoder.Source: The remainder under every intervention · p. 13

Operating point and ranking disparities under mitigation

  • Per-group operating-point shifts reduce median sensitivity and FPR differences by 0.075 and 0.069 while leaving the race AUROC difference at 0.000.Strictly increasing score maps cannot change within-group rankings, although calibration differences increase by 0.016.Source: Operating point and ranking… · p. 13

The remainder under every intervention

  • LEACE lowers the race AUROC difference by 0.012 while lowering disease AUROC by 0.030; iterative nullspace projection lowers them by 0.010 and 0.049.Other methods also trade disparity reductions against disease AUROC reductions.Source: Operating point and ranking… · p. 13

Acquisition view and site as competing explanations

Discussion

FRAME separates subgroup differences compatible with sampling variation from remainders that may require a mechanistic explanation. Across medical imaging studies, the reference often explains substantial reported differences, while tested interventions rarely change the remainder and image-text pretraining improves worst-group performance.

Discussion

  • A median 41% of race differences and 22% of age differences matched the fair-model reference in 125,992 chest radiographs.For race, 51 of 130 encoder–finding combinations did not exceed the reference.Source: Discussion · p. 18
  • Across 89 published differences, 40 of 53 thresholded-rate differences exceeded their reference, compared with 1 of 36 AUROC differences.The reference retained a median 75% of reported rate differences but represented a median 70% of AUROC differences.Source: Discussion · p. 19
  • Demographic decodability did not change the remaining subgroup difference, whereas entangling group with disease direction raised the difference from 0.077 to 0.118.Decodability varied substantially across encoders without predicting the achievable difference.Source: Discussion · pp. 18–19
  • Across nine mitigation methods, controlled pretraining, and four unfreezing levels, subgroup AUROC differences changed less than under a pretraining-seed change.Strictly increasing threshold and calibration maps leave within-group rank statistics unchanged; LEACE reduced race AUROC difference by a median 0.012 while lowering disease AUROC by 0.030.Source: Discussion · p. 19
  • Image-text pretraining and deeper finetuning each raised worst-group AUROC by about 0.05, exceeding the gain from tested fairness interventions.Image-text pretraining changed the subgroup difference by at most a median 0.015.Source: Discussion · pp. 18–19
  • The study cannot identify how entanglement arises in trained encoders because interventions operated on cached features rather than during pretraining.The pooled race contrast is also partly a site contrast because most records in the Other category come from sites without race fields.Source: Discussion · p. 20

Methods

FRAME estimates how much of a reported subgroup difference is compatible with sampling variation, then tests candidate mechanisms by intervening on cached representations.

Data and evaluation units

  • 702,206 images span chest radiography, dermatology, and retinal funduscopy, with sensitive attributes harmonized across sites.Subgroups with fewer than 20 evaluable images are dropped, and each attribute must have at least two evaluable subgroups.Source: Datasets · pp. 21–22

Difference statistic

  • The reported subgroup difference is the range between the maximum and minimum subgroup performance values.The framework evaluates AUROC, sensitivity, false-positive rate, and expected calibration error.Source: FRAME · p. 22

Fair-model reference

  • The fair-model reference simulates that range under exact fairness using the observed subgroup counts and the model’s overall performance.For AUROC, sampling distributions use subgroup positive and negative counts; the construction extends to measures with an available sampling distribution.Source: FRAME · p. 22
  • The reference is estimated from 2,000 simulations, and the remainder is the observed difference minus the reference.The reported interval uses the 2.5th and 97.5th percentiles of the simulated distribution.Source: FRAME · p. 23

Representation interventions

  • Two operators inject candidate causes into cached encoder features before refitting the prediction head.The decodability operator adds a group-dependent shift, while the entangling operator modifies features using a disease subspace.Source: FRAME · p. 23
  • The decodability operator preserves within-group rankings for any fixed linear head, whereas the entangling operator has no such invariance.Heads are refitted at every intervention strength, so only refitting can reorder scores within a group under the decodability operator.Source: FRAME · p. 23

Experimental design

The experiments test FRAME’s claims using matched controls: each difference is compared with its own fair-model reference, interventions are evaluated causally, and disease performance is held constant.

Controls

  • Every subgroup difference is compared against its own reference at the observed subgroup counts rather than against zero.This prevents the expected positive range under finite sampling from being treated as disparity by default.Source: Experimental design · p. 23
  • Candidate causes are tested by intervention as well as observation because cross-model association cannot establish that changing a candidate changes the difference.The design directly evaluates whether the remainder responds to representation-space manipulations.Source: Experimental design · p. 24
  • Interventions are compared at matched disease performance so reducing a subgroup difference by reducing overall performance is not counted as success.The comparison uses the best achievable difference subject to a disease-performance tolerance.Source: Experimental design · p. 24

Experimental scope

  • The full design runs on chest radiography, while dermatology and funduscopy provide scope checks under the identical protocol.A final experiment applies step one to 89 published subgroup differences from nine studies across six imaging modalities.Source: Experimental design · p. 24

Representation geometry and the synthetic model

The study measures sensitive-attribute information in frozen representations, manipulates its geometry relative to disease directions, and evaluates mitigation methods at matched disease performance.

Representation measurements

  • Four measures quantify how much frozen representations encode sensitive attributes, including linear and nonlinear decodability and geometric overlap.Decodability is evaluated on held-out test features, with chance at 0.5 for linear attribute prediction.Source: Representation geometry and the… · p. 24

Synthetic model

  • The synthetic model varies collinearity between group and disease directions while holding both group effects fixed.It uses 160-dimensional features with signal in 32 dimensions and recomputes discrimination and geometry measures at each collinearity level.Source: Representation geometry and the… · p. 24
  • The synthetic group-direction term is rank-neutral, whereas the group-specific variance term is not and projects onto disease according to their alignment.The alignment parameter ρ ranges from 0 to 1 across five seeds and 4,000 samples per setting.Source: Representation geometry and the… · p. 24

Encoder panel

  • The frozen encoder panel includes ten chest-radiography encoders, with native global embeddings extracted once and cached.The broader study uses released models and controlled pretraining recipes spanning image-image, disease-label, and image-text objectives.Source: Encoder panel and controlled pretraining · pp. 24–25

Mitigation evaluation

  • Nine mitigation methods cover data, objective, score, and feature-erasure interventions, and are evaluated for race and age across modalities and head types.The reported comparison selects the smallest subgroup difference among methods whose disease AUROC remains within 0.010 of the unmitigated model.Source: Mitigation methods, the achievable… · p. 26
  • Ridge models predict the matched-performance difference from erasure cost, geometric overlap, or linear and nonlinear decodability using five-fold cross-fitting.The reference distribution uses 1,000 target shuffles, and negative out-of-sample R2 indicates performance below the held-out mean.Source: Mitigation methods, the achievable… · p. 26

Finetuning and the acquisition controls

Additional experiments examine finetuning, acquisition-view variation, and published evidence, while using clustered resampling and multiplicity control for uncertainty and comparisons.

Finetuning

  • Four unfreezing levels compare a linear head, last transformer block, LoRA adapters, and full-backbone finetuning.Early stopping is used because the full level otherwise memorized the training split and underperformed the LoRA level on test AUROC.Source: Finetuning and the acquisition controls · pp. 26–27

Acquisition controls

  • Acquisition-view analyses recompute race and age differences with view-specific references and require at least 500 evaluable images per view.View and site are also analyzed as attributes in their own right.Source: Finetuning and the acquisition controls · p. 27

Published-study audit

  • The published-study audit includes 89 claims from nine studies across six modalities, selected using four criteria for recoverable subgroup differences and performance values.The authors state that the study set is not a systematic search.Source: Auditing published subgroup differences · p. 27
  • Published references use reported overall performance when available, otherwise recovering it from denominator-weighted subgroup values for rate statistics.Rate references use exact binomial draws, while AUROC references use the corresponding subgroup sampling procedure.Source: Auditing published subgroup differences · pp. 27–28

Statistical analysis

  • Patient-level cluster bootstrap, fixed simulation counts, and FDR correction govern uncertainty, paired comparisons, and exceedance testing.The fair-model reference uses 2,000 simulated fair models, while performance uncertainty uses 1,000 patient-level bootstrap resamples.Source: Statistical analysis · pp. 28–29

Data availability

The study uses existing, publicly released data sources and does not redistribute images or derived records. The published-difference audit uses only information printed in the audited articles.

Data availability

  • Existing, publicly released sources provide all study data, with one chest radiograph reproduced from NIH ChestX-ray14.No images or derived records are redistributed.Source: Data availability · p. 29
  • The audited-article analysis uses no data beyond what the published articles report.Source: Data availability · p. 29

Code availability

The analysis code is publicly available and documents the data-building, pretraining, analysis, and figure-generation workflow. Evaluated encoders were local, open-weight models run as frozen feature extractors without cloud services or network-served models.

Code availability

  • The analysis repository provides data-build, controlled-pretraining, analysis, and figure code, including fixed bootstrap, permutation, and simulation seeds.It does not redistribute model weights or underlying datasets.Source: Code availability · p. 29
  • All evaluated encoders were local, open-weight frozen feature extractors run entirely on-site without cloud services or third-party APIs.No closed model or network-served model was used.Source: Code availability · p. 29
  • The encoder inventory includes chest-radiograph, general-purpose, dermatology, and fundus models, plus a text encoder for image-text pretraining.Source: Code availability · p. 30
  • Analyses used a pinned Python and machine-learning environment and ran encoder training, finetuning, and feature extraction on NVIDIA L40S GPUs.Other stages ran on CPU.Source: Code availability · p. 30

Competing interests

The disclosure reports employment, editorial-board roles, lecture honoraria, and shareholdings among the authors.

Competing interests

  • Reported interests include company employment, editorial positions, lecture honoraria from several medical-technology companies, and shares in two companies.Source: Competing interests · p. 31

Supplementary Note 1: Additional checks on the chest radiograph protocol

Additional checks examine sensitive attributes, calibration, model-head capacity, encoder replication, entanglement, and a separate optimal-transport mitigation. These analyses identify substantial scope and interpretation boundaries for the protocol’s conclusions.

Sex and insurance

  • For sex, the observed difference has a median of 0.009 against a fair-model reference of 0.004 across 104 combinations.The observed difference exceeds the reference in 35 combinations.Source: Sex and insurance as sensitive… · p. 35
  • Insurance differences have a median of 0.062 across 78 combinations, but no fair-model reference or significance test was computed.The authors do not interpret these values as evidence of disparity because the Other category is strongly confounded with site.Source: Sex and insurance as sensitive… · p. 35

Calibration

  • Per-group Platt calibration leaves the race or age AUROC difference unchanged in 252 of 260 combinations or changes it by less than 1 × 10−6 in three more.Five larger changes occur in race comparisons with very small positive subgroups, where fitted logistic maps can become constant.Source: Non-monotone per-group calibration maps · p. 35

Disease-head capacity

  • Changing from a linear to a two-layer disease head changes median differences by −0.002 for race, +0.004 for age, +0.002 for insurance, and +0.001 for sex.The largest single change is 0.088.Source: The nonlinear disease head · p. 35

Encoder replication

  • Injecting demographic decodability raises decodability while leaving the race difference unchanged on both transformer and convolutional encoders.This reproduces the invariance across a second embedding width and backbone type.Source: The second injected encoder · p. 36

Entanglement

  • At k = 256, entanglement changes race differences little across findings, while the overall disease AUROC is unchanged on every finding for one encoder.Because disease AUROC is unchanged on DenseNet-121, the entanglement effect is not tested there and the main-text causal claim depends on RAD-DINO alone.Source: The second injected encoder · p. 36

Optimal-transport mitigation

  • Optimal-transport matching yields a median race difference of 0.078 versus 0.079 unmitigated across 13 findings.Only pleural effusion shows a significant reduction, falling by 0.006.Source: An optimal-transport fairness method · p. 36

Supplementary Note 2: Data and reporting caveats

The pooled race and insurance attributes include an “Other” category that combines recorded categories with patients from sites lacking the field. The fair-model reference uses image counts, although patient-level dependence makes those counts optimistic for sampling variability.

Attribute harmonization

Sampling model

Supplementary Note 3: Dermatology and funduscopy

Across dermatology and funduscopy, the fair-model reference often matches or exceeds observed subgroup differences, especially outside dermatology age comparisons. With these cohort sizes, mitigation tests generally cannot separate the observed differences from sampling variation, and cross-modality geometry prediction does not generalize.

Dermatology

Funduscopy

Dermatology and funduscopy

Mitigation tests

  • None of 126 mitigation tests across both modalities, attributes, and metrics is significant.At these cohort sizes, the analysis cannot distinguish most observed differences from sampling variation under exact fairness.Source: Dermatology and funduscopy against… · p. 37

Cross-modality prediction

  • Out-of-sample R2 is −3.40 for dermatology and −1.92 for funduscopy, neither significant (pFDR = 0.787 for both modalities).Within chest radiography, the leave-one-encoder-out predictor has out-of-sample R2 of 0.024 (pFDR = 0.021).Source: Transfer of the chest radiograph… · p. 37

Supplementary Note 4: The steps of FRAME

FRAME first estimates the subgroup difference expected under exact fairness at the observed counts, then tests whether candidate representation mechanisms change the remainder. The procedure can audit published results from summary counts alone in step one, while step two requires cached features.

Scope of application

  • Step 1 needs overall AUROC and subgroup positive and negative counts, whereas step 2 requires cached features and therefore applies only when the representation is available.Source: FRAME end to end · p. 38

Reference implementation

  • The closed-form reference draw is an approximation, but it agreed with slower binormal simulations on 12 randomly chosen units per run.Source: FRAME end to end · p. 38

Inputs

Step 1: fair-model reference

  • Step 1 simulates subgroup AUROCs with the same true AUROC for every subgroup and forms the fair-model reference from their maximum-minus-minimum differences.Source: FRAME end to end · pp. 37–38
  • The remainder is the reported difference minus the reference; if the exceedance test is nonsignificant, FRAME classifies it as consistent with exact fairness.Source: FRAME end to end · p. 38

Step 2: mechanisms

  • Step 2 injects demographic decodability and group–disease entanglement into cached features, then tests how each changes the remainder.The decodability direction is orthogonalized against the disease direction, and the operators are applied over graded strengths.Source: FRAME end to end · p. 38, Injection operators · pp. 38–39

Published-study audit

  • The published-study audit applies step one using each study’s reported values and subgroup counts, without using the authors’ models.Source: Table 7 · p. 42

Related papers

Loading 2608.25981v1…