Source-linked AI summary
Beyond Predictive Fairness: Quantifying Attribution Consistency Across Demographic Groups in Diabetic Retinopathy Screening
Kerol Djoumessi, Philipp Berens
TL;DR
Existing fairness metrics reveal subgroup performance disparities but offer limited insight into whether models use consistent visual evidence. This paper introduces the Jensen-Shannon divergence-based Explanation Consistency Score for diabetic retinopathy screening and finds high attribution consistency despite performance differences, with no significant association between them.
Problem
Existing subgroup performance metrics quantify predictive disparities but provide limited evidence about whether medical-imaging models rely on similar visual evidence across demographic groups.
Method
The study introduces the Jensen-Shannon divergence-based Explanation Consistency Score to quantify attribution similarity across demographic groups, including a severity-conditional formulation.
Results
Attribution consistency remained uniformly high despite predictive performance differences across ethnic groups, with no significant association between ECS and subgroup performance.
Takeaways & Limitations
Predictive fairness and explanation consistency capture complementary rather than equivalent aspects of model behavior in diabetic retinopathy screening.
Takeaways & Limitations
ECS was computed only on correctly classified samples, leaving subgroup-specific failure modes unexplored, and its group-level formulation may not capture individual-level explanation variability.
Abstract
from arXiv · showhide
Fairness in medical imaging is commonly evaluated through subgroup performance metrics, yet it remains unclear whether models rely on consistent visual evidence across demographic groups. This work introduces the Explanation Consistency Score (ECS), a fairness-aware metric based on Jensen-Shannon divergence that quantifies the similarity of attribution maps across subgroups. Using diabetic retinopathy screening as a case study, ECS is evaluated globally and within disease severity. Experiments reveal that while predictive performance differs across ethnic groups, explanation consistency remains relatively high and shows no significant association with performance disparities. These findings suggest that predictive fairness and explanation consistency capture complementary dimensions of model behavior, motivating fairness evaluations that extend beyond predictive performance.
1 Introduction
Predictive subgroup metrics may miss whether medical-imaging models rely on consistent visual evidence across demographic groups. This work introduces ECS, a Jensen–Shannon divergence-based measure of attribution similarity, with a conditional formulation accounting for disease severity in diabetic retinopathy detection.
- Motivation: Existing fairness studies primarily assess demographic disparities through predictive subgroup metrics, which may be insufficient for trustworthy medical-imaging deployment.Models can show similar subgroup performance while relying on different visual evidence, or different performance while relying on similar evidence.
- Motivation: Medical-imaging models can encode demographic information despite being trained for clinical tasks, including demographic attributes inferred directly from ophthalmic fundus images.
- Explainability: CAM-based explainability highlights retinal regions contributing to predictions and helps assess attention to clinically relevant structures and lesions.Attribution maps nevertheless have known limitations and sensitivity to model architecture.
- Contribution: The study introduces the Explanation Consistency Score (ECS), a Jensen–Shannon divergence-based metric quantifying attribution similarity across demographic groups in early diabetic retinopathy detection.A conditional ECS formulation is additionally proposed to account for potential confounding from differences in disease severity.
2 Methods
The methods use patient-level stratified EyePACS splits and a binary ResNet-50 screening task, then assess demographic attribution consistency with two attribution methods and an ECS based on Jensen–Shannon divergence. ECS is computed both across demographic groups and conditionally on disease severity, with conditional analyses restricted to grades 1 and 2.
- Dataset and task: 90,595 good-quality EyePACS images were split at the patient level into 70% training, 15% validation, and 15% test sets using iterative multi-label stratification.Stratification preserved ethnicity and diabetic retinopathy grade distributions.
- Dataset and task: The fairness analysis covered five largest ethnicity groups: Latin American, African Descent, Indian Origin, Caucasian, and Asian.Unspecified and underrepresented groups were retained for training but excluded from subgroup analyses.
- Dataset and task: A ResNet-50 distinguished healthy eyes (grade 0) from eyes with any diabetic retinopathy stage (grades 1–4) to benchmark demographic differences in explanations.The binary setting targets subtle lesions relevant to early disease detection.
- Attribution analysis: SmoothGradCAM++ and Score-CAM generated gradient-based and gradient-free attribution maps to test explanation-consistency robustness across attribution mechanisms.Each attribution map is converted into a spatial probability distribution.
- Explanation Consistency Score: ECS compares mean attribution distributions between demographic groups using symmetric, bounded Jensen–Shannon divergence, with scores ranging from 0 to 1.Larger ECS values indicate greater attribution similarity; scores are also computed conditionally on DR grade and averaged across grades.
3 Experiments and Results
Experiments found substantial demographic differences in predictive performance but much smaller variation in attribution consistency. High explanation consistency persisted across demographic groups after controlling for disease severity.
- Subgroup Predictive Performance: Indian-origin patients achieved the highest AUC (0.92 ± 0.001) and sensitivity (0.83 ± 0.003), while Caucasian patients had the lowest AUC (0.83±0.003) and sensitivity (0.54 ± 0.018).The corresponding sensitivity gap was 23%.
- Subgroup Predictive Performance: Specificity remained relatively stable across groups at 0.94–0.97, indicating that differences primarily involved detecting positive diabetic retinopathy cases.This contrasts with the larger subgroup variation in sensitivity.
- Global Explanation Consistency: Sensitivity varied by 23%, whereas ECS varied by only 0.03 for both ScoreCAM and SmoothGradCAM++, showing substantially lower attribution-consistency variability.ECS was computed from correctly classified test samples using ECS-SC and ECS-SG.
- Global Explanation Consistency: Groups with lower predictive performance, including Caucasian patients, remained highly consistent with other groups’ attribution patterns, while Indian-origin patients did not have the highest ECS.The findings indicate that predictive disparities were not necessarily accompanied by comparable attribution differences.
- Conditional Explanation Consistency: After conditioning on disease severity, ScoreCAM ECS ranged from 0.79 to 0.87 for grade 1 and 0.87 to 0.90 for grade 2.SmoothGradCAM++ ECS ranged from 0.77–0.82 for grade 1 and 0.81–0.86 for grade 2, preserving high explanation consistency.
4 Discussion and Conclusion
The study introduced ECS to quantify attribution similarity across demographic groups in diabetic retinopathy screening. Predictive performance varied across ethnicities while attribution consistency remained uniformly high, motivating complementary fairness evaluation and further investigation of limitations.
- Findings: ECS quantifies attribution-pattern similarity across demographic groups in diabetic retinopathy screening, while attribution consistency remained uniformly high despite variation in predictive performance across ethnicities.This pattern was observed for both ScoreCAM and SmoothGradCAM++.
- Findings: Subgroup performance was not determined solely by sample size: the Indian-origin subgroup had the highest AUC and sensitivity despite being among the smallest groups.The largest subgroup did not achieve the strongest performance.
- Limitations: ECS was computed only on correctly classified samples, leaving subgroup-specific failure modes unexplored.Future work should investigate attribution consistency in unsuccessful predictions.
- Limitations: As a group-level measure based on mean attribution distributions, ECS may not capture within-group variation, despite an additional within-group analysis.The passage also identifies Jensen–Shannon divergence as ECS’s similarity measure and notes alternatives such as cosine similarity, Wasserstein distance, and structural similarity.
- Limitations: The study used a single dataset, focused exclusively on ethnicity, and applied posthoc methods, motivating evaluation across modalities, datasets, protected attributes, and self-explainable models.The passage identifies these extensions as directions for future evaluation.