Source-linked AI summary
Explainable Diabetic Retinopathy Classification Using Vision Foundation Models
Abhishek Verma, Anila Krishna, Abhishek Gajanan Bankar, Juan Miguel Lopez Alcaraz
TL;DR
Diabetic retinopathy screening requires accurate predictions whose rationale can be assessed for clinical relevance. This study compares vision foundation-model backbones and adaptation strategies, validates performance across ODIR and APTOS, and evaluates Grad-CAM and HiResCAM against IDRiD lesion masks. Foundation models, particularly DINOv2, showed strong predictive performance, while LoRA provided a competitive parameter-efficient alternative.
Problem
Explainable diabetic retinopathy classification lacks standardized evidence that saliency maps are clinically meaningful and faithfully represent model behavior.
Method
The study compares ViT, CLIP, and DINOv2 under full fine-tuning, linear probing, and LoRA, with external validation and lesion-mask-based evaluation of Grad-CAM and HiResCAM.
Results
DINOv2-LoRA achieved the highest internal AUROC of 0.758, while ViT-Full achieved the highest external AUROC by a very small margin over DINOv2-Full.
Takeaways & Limitations
Foundation-model approaches have potential for diabetic retinopathy screening, and explanation evaluation should include quantitative assessment of correspondence with clinically relevant lesions.
Takeaways & Limitations
Benchmark datasets may not represent routine clinical variability, and the study focuses on binary detection with explainability analysis limited to available IDRiD masks and combined lesion classes.
Abstract
from arXiv · showhide
Diabetic retinopathy (DR) is a major cause of preventable blindness, creating a need for accurate and trustworthy automated screening. This study investigates an explainable DR classification framework using vision foundation models and multiple transfer learning strategies. Three backbones, DINOv2, CLIP, and Vision Transformer (ViT), were evaluated using full fine-tuning, linear probing, and Low-Rank Adaptation (LoRA). Models were trained and internally evaluated on the ODIR dataset and externally evaluated on APTOS to assess generalization. DINOv2-LoRA achieved the highest internal AUROC of 0.758, while DINOv2 full fine-tuning and ViT full fine-tuning achieved the highest external AUROC of 0.920. Calibration was further assessed using reliability analysis after isotonic regression. For explainability, Grad-CAM and HiResCAM were evaluated against expert-annotated lesion masks from the IDRiD dataset using Dice, Intersection over Union (IoU), and Pointing Game metrics. The results demonstrate that foundation models, particularly DINOv2, can provide strong predictive performance, while LoRA offers a parameter-efficient alternative to full fine-tuning. Quantitative evaluation of explanation maps further supports the assessment of whether model attention corresponds to clinically relevant retinal lesions.
1. INTRODUCTION
Diabetic retinopathy screening needs automated systems that are accurate, scalable, and clinically trustworthy. This study combines vision foundation models, transfer-learning comparisons, external validation, and quantitative evaluation of explanation maps against retinal lesions.
- Clinical motivation: Diabetic retinopathy causes major vision impairment, while manual fundus grading is time-consuming and requires trained ophthalmologists.AI is presented as a potential tool for fast, consistent, and scalable screening.
- Foundation-model adaptation: Foundation models offer transferable representations for medical imaging, while LoRA reduces adaptation costs by updating only a small subset of parameters.The study compares LoRA with full fine-tuning and linear probing.
- Explainability challenge: The paper addresses limited evidence on whether explanations are clinically meaningful and faithfully reflect model behavior.Existing explanation assessment is often qualitative, subjective, or vulnerable to visually plausible but misleading saliency maps.
- Quantitative explanation validation: Explanation maps are compared with expert-annotated IDRiD lesion masks using Dice, IoU, and Pointing Game accuracy.The evaluated lesions include microaneurysms, hemorrhages, and exudates.
- Study design: The framework compares ViT, CLIP, and DINOv2 across full fine-tuning, linear probing, and LoRA, with ODIR development and APTOS external validation.It jointly considers predictive performance, adaptation efficiency, calibration, and explanation localization.
2. RELATED WORK
Prior work established strong deep-learning and foundation-model performance for retinal disease classification, but interpretability and explanation reliability remain less systematically evaluated. This study links adaptation-strategy comparisons, external validation, and lesion-mask-based explanation assessment.
- Deep learning for diabetic retinopathy classification: Deep-learning systems have achieved performance comparable to expert ophthalmologists and shown potential for large-scale diabetic retinopathy screening.Transformer architectures provide an alternative to conventional CNNs through self-attention and long-range image relationships.
- Vision foundation models: Vision foundation models learn transferable representations through large-scale, contrastive, or self-supervised pretraining and can support downstream medical-imaging tasks.CLIP and DINOv2 are highlighted as influential examples.
- Parameter-efficient adaptation: LoRA reduces computational and memory requirements by updating a small subset of parameters while preserving much of full fine-tuning performance.This creates a parameter-efficiency trade-off for specialized medical tasks.
- Explainable AI for retinal analysis: XAI methods such as Grad-CAM and HiResCAM produce attribution maps intended to show image regions influencing predictions.This study evaluates their spatial correspondence with expert-annotated retinal lesions rather than relying only on visual inspection.
- Research gap: Quantitative localization does not establish causal fidelity, because high overlap with lesions may not mean those regions drove the model’s prediction.External validation is also needed to assess robustness beyond the development distribution.
3.1. Dataset
The experiment uses three public retinal-image datasets with distinct roles in model development, external testing, and explanation evaluation.
- Dataset roles: ODIR-5K supports training, validation, and internal testing with patient-level splitting to prevent same-patient eye images crossing partitions.APTOS 2019 is used for external testing; the passage is truncated before the full dataset-role description.
3.2. Models
The study compares three vision backbones from supervised, self-supervised, and vision-language pretraining paradigms, using three adaptation strategies for diabetic retinopathy classification.
- Three backbones were evaluated: supervised ViT-Small/16, self-supervised DINOv2 ViT-Small/14, and vision-language CLIP ViT-Base/32.
- Each backbone was tested with full fine-tuning, linear probing, and Low-Rank Adaptation (LoRA).
- DINOv2 uses self-supervised learning on large image collections without manual labels and was adapted for diabetic retinopathy diagnosis.
- CLIP uses contrastive learning to align image and text representations before adaptation to binary diabetic retinopathy classification.
- LoRA keeps pretrained parameters fixed while introducing trainable low-rank updates to reduce optimization requirements.
3.3. Training setup
All nine model configurations used a shared training protocol designed to support fair comparison across backbones and adaptation strategies.
- All nine configurations used weighted binary cross-entropy loss, AdamW optimization, cosine annealing, and a maximum of 10 training epochs.Early stopping used a patience of 4 epochs, with random seed 42.
3.4. Performance evaluation
Performance evaluation used standardized preprocessing and augmentation, AUROC as the primary metric, and calibration analysis for predicted-probability reliability across internal and external datasets.
- Training images received random flips, rotation, and color jitter, whereas validation and test images were only resized and normalized.
- ODIR supported model development and internal evaluation, APTOS external validation, and IDRiD explainability evaluation.The IDRiD segmentation subset contained 54 images with expert-annotated lesion masks.
- AUROC was the primary performance metric, while calibration assessed the reliability of predicted probabilities for clinical decision support.
- Isotonic regression was applied to probability outputs to reduce overconfident predictions without imposing a parametric calibration function.
- Reliability diagrams compared calibrated probabilities with observed outcomes for internal ODIR and external APTOS evaluations.
3.5. Explainability
The explainability framework compares Grad-CAM and HiResCAM spatial attribution maps with expert lesion masks using complementary localization metrics on confident positive predictions.
- Grad-CAM uses class-specific gradients from the topmost network layer to visualize regions contributing to predictions.
- HiResCAM produces higher-resolution activation maps using element-wise products of feature maps and gradients for precise localization.
- Grad-CAM and HiResCAM maps were selected because they can be directly compared with expert-annotated retinal lesion masks from IDRiD.
- Attention Rollout was explored qualitatively but excluded from the final quantitative comparison because only Grad-CAM and HiResCAM received systematic lesion-localization evaluation.
- Explanation maps were evaluated with Dice, Intersection over Union (IoU), and Pointing Game accuracy.Dice and IoU measure spatial overlap, while Pointing Game accuracy tests whether the most salient location lies within an annotated lesion.
- Quantitative evaluation used correctly classified DR-positive IDRiD images with prediction confidence greater than 0.80.
4. RESULTS
The evaluated foundation-model and adaptation configurations showed different strengths on internal ODIR and external APTOS evaluation. Predictive performance, calibration, and explanation localization were assessed using quantitative and visual analyses.
- Predictive performance: 0.758 internal AUROC was achieved by DINOv2-LoRA, the strongest internal result among evaluated configurations.
- Predictive performance: 0.9203 external AUROC was achieved by ViT-Full, closely followed by DINOv2-Full at 0.9202.
- Predictive performance: DINOv2-LoRA achieved 0.907 external AUROC while requiring substantially fewer trainable parameters than full fine-tuning.
- Calibration: Reliability diagrams compared DINOv2-LoRA and DINOv2-Full on ODIR and APTOS, assessing agreement between predicted probabilities and observed outcome frequencies.Isotonic regression was applied to validation-set predictions for probability calibration.
- Explainability validation: HiResCAM achieved higher Pointing Game accuracy than Grad-CAM, 66.7% versus 33.3%, despite lower Dice and IoU values.
5. DISCUSSION
The framework combines predictive performance, parameter-efficient adaptation, calibration, and lesion-based explanation analysis for diabetic retinopathy screening. The authors emphasize competitive LoRA performance while noting limits in dataset representativeness, task scope, explanation coverage, and clinical validation.
- Main findings: DINOv2-LoRA achieved the highest internal AUROC, while ViT-Full achieved the highest external AUROC by a very small margin over DINOv2-Full.
- Main findings: LoRA provided a parameter-efficient alternative to full fine-tuning while maintaining competitive predictive performance.
- Limitations: Benchmark datasets may not fully represent routine clinical variability across patient populations, imaging devices, and acquisition protocols.
- Limitations: The study focused on binary diabetic retinopathy detection rather than detailed disease severity grading.
- Limitations: Grad-CAM and HiResCAM provide localization-based explanations but cannot establish causal relationships between highlighted retinal features and model predictions.
- Explainability: Grad-CAM and HiResCAM were evaluated against expert-annotated IDRiD lesion masks to assess spatial correspondence between explanations and clinically relevant lesions.
- Conclusion: Further multi-center and prospective clinical validation is required before clinical deployment.
6. AUTHOR CONTRIBUTIONS
The listed contributors covered software, methodology, investigation, validation, data curation, analysis, visualization, writing, conceptualization, and supervision.
- Contributor roles spanned software, methodology, investigation, validation, data curation, formal analysis, visualization, writing, conceptualization, and supervision.