PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Diabetic Retinopathy Classification Using Vision Foundation Models

Abhishek Verma, Anila Krishna, Abhishek Gajanan Bankar, Juan Miguel Lopez Alcaraz

arXiv:2608.28207v1cs.CVcs.LG

TL;DR

Reliable diabetic retinopathy screening needs accurate, trustworthy automated classification, while evidence jointly assessing adaptation strategies, predictive performance, and lesion-grounded explanations remains limited. This study compares foundation-model adaptation and explanation methods, finding DINOv2-LoRA led internal AUROC while ViT-Full narrowly led external AUROC.

  • Problem

    Foundation-model prediction and explanation are often evaluated separately, and comparisons of adaptation strategies and lesion correspondence remain limited.

    Source: §2.5 · p. 4

  • Method

    The study compares ViT, CLIP, and DINOv2 adaptation strategies, tests generalization on APTOS, and evaluates Grad-CAM and HiResCAM against IDRiD lesion masks.

    Source: §2.5 · p. 4, §3.1 · p. 5, §3.5.2 · p. 6

  • Results

    0.9203 external AUROC for ViT-Full narrowly exceeded 0.9202 for DINOv2-Full, while DINOv2-LoRA achieved the highest internal AUROC.

    Source: §4.1 · p. 7, §5.4 · p. 9

  • Takeaways & Limitations

    LoRA offers a competitive parameter-efficient adaptation strategy, while the three explanation metrics provide complementary evidence of spatial correspondence with retinal lesions.

    Source: §4.1.1 · p. 8, §5.4 · p. 9

    Public benchmark datasets may not represent routine clinical variability, limiting how broadly these results reflect differences in patients, imaging devices, and acquisition protocols.

    Source: §5.2 · p. 8

Abstract

from arXiv · show

Diabetic retinopathy (DR) is a major cause of preventable blindness, creating a need for accurate and trustworthy automated screening. This study investigates an explainable DR classification framework using vision foundation models and multiple transfer learning strategies. Three backbones, DINOv2, CLIP, and Vision Transformer (ViT), were evaluated using full fine-tuning, linear probing, and Low-Rank Adaptation (LoRA). Models were trained and internally evaluated on the ODIR dataset and externally evaluated on APTOS to assess generalization. DINOv2-LoRA achieved the highest internal AUROC of 0.758, while DINOv2 full fine-tuning and ViT full fine-tuning achieved the highest external AUROC of 0.920. Calibration was further assessed using reliability analysis after isotonic regression. For explainability, Grad-CAM and HiResCAM were evaluated against expert-annotated lesion masks from the IDRiD dataset using Dice, Intersection over Union (IoU), and Pointing Game metrics. The results demonstrate that foundation models, particularly DINOv2, can provide strong predictive performance, while LoRA offers a parameter-efficient alternative to full fine-tuning. Quantitative evaluation of explanation maps further supports the assessment of whether model attention corresponds to clinically relevant retinal lesions.

1. INTRODUCTION

The introduction motivates scalable diabetic retinopathy screening and trustworthy interpretation, then proposes comparing vision foundation-model adaptation strategies and quantitatively evaluating lesion-localizing explanations.

1.1. Clinical importance of diabetic retinopathy diagnosis

  • DR is a leading cause of blindness, and fundus photography offers non-invasive screening, but manual grading is time-consuming and limits scalable screening.AI is presented as a way to enable fast, consistent screening.Source: §1.1 · p. 1

1.2. Clinical foundation models

  • Vision foundation models provide transferable representations for DR classification, while LoRA adapts them by updating fewer parameters than full fine-tuning.LoRA can reduce computational and memory requirements while maintaining competitive predictive performance.Source: §1.2 · p. 1

1.3. Explainable AI for trustworthy clinical decision support

  • Foundation-model decisions can remain opaque, but clinicians need to understand and trust prediction rationales before integrating AI into diagnostic workflows.Explainability is framed as a requirement for trustworthy, clinically deployable medical AI.Source: §1.3 · pp. 1–2

1.4. Challenges in evaluating explainability

  • Explanation quality lacks a universally accepted standard, while expert visual review is subjective and saliency maps may look plausible without reflecting model reasoning.IDRiD lesion masks enable objective localization assessment, but spatial agreement does not establish causal faithfulness.Source: §1.4 · p. 2

1.5. Contributions

  • The study compares ViT, CLIP, and DINOv2 using full fine-tuning, linear probing, and LoRA for binary diabetic retinopathy classification.It evaluates models on ODIR and externally validates on APTOS to assess generalization across datasets.Source: §1.5 · p. 2
  • Grad-CAM and HiResCAM explanations are evaluated against IDRiD expert lesion masks using Dice, IoU, and Pointing Game accuracy.The study also examines trade-offs among predictive performance, parameter-efficient adaptation, calibration, and explanation localization.Source: §1.5 · p. 2

2. RELATED WORK

Prior work established deep learning and vision foundation models as promising tools for retinal disease classification, but interpretability, quantitative explanation validation, and evaluation across adaptation strategies and external datasets remain important gaps.

2.1. Deep learning for diabetic retinopathy classification

  • CNNs achieved performance comparable to expert ophthalmologists in large-scale DR screening, while later work demonstrated effectiveness across populations and imaging conditions and the feasibility of autonomous screening.ViTs use self-attention to capture long-range image relationships and have shown competitive classification performance; RETFound results also suggest retinal pretraining can improve label efficiency and generalization,.Source: §2.1 · pp. 2–3

2.2. Vision foundation models in medical imaging

  • Vision foundation models learn transferable representations from large-scale pretraining, while CLIP uses contrastive learning and DINOv2 self-supervision; LoRA adapts models by updating few parameters to reduce computational requirements.Retinal foundation models have shown performance across disease-prediction tasks and robustness when transferred to new datasets and clinical settings.Source: §2.2 · pp. 3–4

2.3. Explainable AI for diabetic retinopathy analysis

  • Grad-CAM and other visualization methods highlight prediction-relevant image regions, but saliency maps may appear clinically plausible without reliably representing model behavior.This work focuses on Grad-CAM and HiResCAM and evaluates their correspondence with expert-annotated lesions rather than relying only on visual inspection.Source: §2.3 · p. 4

2.4. Quantitative evaluation of explainability

  • Explanation maps can be quantitatively compared with IDRiD pixel-level masks of microaneurysms, hemorrhages, and exudates using DSC, IoU, and Pointing Game accuracy.These localization metrics do not establish that highlighted regions caused the model’s prediction.Source: §2.4 · p. 4

2.5. Research gap

  • The study addresses the limited joint evaluation of predictive performance and explanation quality by comparing ViT, CLIP, and DINOv2 across adaptation strategies, validating externally on APTOS, and testing Grad-CAM and HiResCAM against IDRiD lesion masks.The authors also identify the need to assess whether LoRA maintains performance with fewer trainable parameters and whether predictions and explanations generalize across datasets and clinical settings.Source: §2.5 · p. 4

3. METHODS · 3.1. Dataset

The study used three public retinal-image datasets for model development, external testing, and explainability validation. Patient-level splitting of ODIR-5K kept each patient’s two eyes in the same partition.

3.1. Dataset

  • ODIR-5K was used for model training, validation, and internal testing.Images were split at the patient level so each patient’s left and right eyes remained in the same partition, preventing patient-level data leakage.Source: §3.1 · p. 5
  • APTOS 2019 served as an external test dataset and was not used to train the models.Its role was to assess generalization across datasets.Source: §3.1 · p. 5
  • IDRiD was used solely to validate explainability, using its expert-defined lesion segmentation masks.Source: §3.1 · p. 5
  • The three public-domain retinal image datasets had distinct functions in the experimental design.ODIR-5K supported model development, APTOS 2019 external testing, and IDRiD explainability validation.Source: §3.1 · p. 5

3.2. Models

The study compared supervised, self-supervised, and vision-language backbones for binary diabetic retinopathy classification, applying full fine-tuning, linear probing, and LoRA to each.

3.2.1. Vision Transformer (ViT)

  • ViT-Small/16 served as the supervised backbone and was evaluated with full fine-tuning, linear probing, and LoRA.Source: §3.2.1 · p. 5

3.2.2. DINOv2

  • DINOv2 (ViT-Small/14), a self-supervised model trained on images without manual labels, was adapted with full fine-tuning, linear probing, and LoRA [8, 20].Source: §3.2.2 · p. 5

3.2.3. CLIP

  • CLIP (ViT-Base/32) uses contrastive learning to align image and text representations; its visual encoder was adapted with full fine-tuning, linear probing, and LoRA.Source: §3.2.3 · p. 5

3.2.4. Low-Rank Adaptation

  • LoRA kept pretrained parameters fixed and added trainable low-rank updates, reducing optimized parameters while adapting each backbone to diabetic retinopathy classification.It was compared with full fine-tuning and linear probing to assess predictive performance versus parameter efficiency.Source: §3.2.4 · p. 5

3.3. Training setup

All nine configurations used a shared training protocol, and LoRA reduced trainable parameter counts relative to full fine-tuning while exceeding linear probing. The setup used weighted BCE, AdamW, cosine annealing, early stopping, and a fixed seed.

3.3. Training setup

  • 22,056,577, 385, and 221,569 trainable parameters were used for DINOv2 full fine-tuning, linear probing, and LoRA, respectively.LoRA reduced trainable parameters compared with full fine-tuning while providing greater adaptation capacity than linear probing.Source: §3.3 · p. 5
  • 87,456,769, 769, and 295,681 trainable parameters were used for CLIP full fine-tuning, linear probing, and LoRA, respectively.LoRA reduced trainable parameters compared with full fine-tuning while providing greater adaptation capacity than linear probing.Source: §3.3 · p. 5
  • 21,666,049, 385, and 221,569 trainable parameters were used for ViT full fine-tuning, linear probing, and LoRA, respectively.LoRA reduced trainable parameters compared with full fine-tuning while providing greater adaptation capacity than linear probing.Source: §3.3 · p. 5
  • All nine configurations used weighted binary cross-entropy, AdamW with cosine annealing, and a maximum of 10 training epochs.Early stopping used a patience of 4 epochs, and experiments used a fixed random seed of 42.Source: §3.3 · p. 5

3.4. Performance evaluation

The section defines AUROC as the primary performance measure and assesses probability calibration separately, using reliability diagrams for internal ODIR and external APTOS evaluation.

3.4. Performance evaluation

  • Model performance was primarily evaluated using AUROC, while calibration assessed the reliability of predicted probabilities for clinical decision support.Images were resized and normalized using pretrained-model statistics; training used random flips, rotation, and color jitter, unlike validation and test images.Source: §3.4 · pp. 5–6

3.4.1. Calibration analysis

  • Isotonic regression calibrated model probabilities, and reliability diagrams compared predictions with observed outcomes on ODIR and APTOS independently of discrimination performance.Calibration was interpreted alongside AUROC to distinguish probability reliability from discrimination performance.Source: §3.4.1 · p. 6

3.5. Explainability

The study evaluated Grad-CAM and HiResCAM explanations against expert-annotated IDRiD lesion masks using spatial-overlap and localization metrics. Attention Rollout was explored preliminarily but excluded from the final quantitative comparison.

3.5.1. Grad-CAM

  • Grad-CAM uses class-specific gradients from a network’s topmost layer to create heatmaps that visualize regions important to a diabetic retinopathy prediction.Source: §3.5.1 · p. 6

3.5.2. HiResCAM

  • HiResCAM computes pixel-wise importance from feature-map and gradient products to create high-resolution activation maps for assessing alignment with disease symptoms [21].Both methods produce spatial attribution maps that can be compared with IDRiD expert-annotated lesion masks.Source: §3.5.2 · p. 6

3.5.3. Attention Rollout (Preliminary Evaluation)

  • Attention Rollout was explored qualitatively in preliminary experiments but excluded from quantitative comparison; only Grad-CAM and HiResCAM received systematic lesion-localization evaluation.Source: §3.5.3 · p. 6

3.5.4. Quantitative explainability evaluation

  • Dice and IoU measured map–lesion overlap, while Pointing Game accuracy tested whether the most salient location fell within an annotated lesion.Evaluation used correctly classified DR-positive IDRiD images with prediction confidence greater than 0.80.Source: §3.5.4 · p. 6

4. RESULTS

DINOv2-LoRA had the strongest internal discrimination, while ViT-Full and DINOv2-Full led externally. The study also assessed probability calibration and the spatial correspondence of explanation maps with annotated lesions.

4.1. Predictive performance and calibration

  • DINOv2-LoRA led internal AUROC (0.758), while ViT-Full led external AUROC (0.9203), narrowly ahead of DINOv2-Full (0.9202).The ROC curves show the same internal and external discrimination rankings.Source: §4.1 · pp. 6–7, Table 2 · p. 7, Figure 2 · p. 7
  • DINOv2-LoRA achieved external AUROC (0.907) while requiring substantially fewer trainable parameters than full fine-tuning.The study applied isotonic regression to validation-set predictions and used reliability diagrams to assess agreement between calibrated probabilities and observed outcome frequencies on ODIR and APTOS.Source: §4.1 · p. 7, Figure 3 · p. 8

4.1.1. Comparison with existing approaches

  • The study’s main contribution was a controlled comparison of supervised, self-supervised, and vision-language foundation models under identical adaptation and external-validation settings.Published RETFound results were AUROC 0.943 on APTOS-2019 and 0.822 on IDRiD, but direct numerical comparison is cautious because datasets, preprocessing, class definitions, and evaluation protocols differ.Source: §4.1.1 · pp. 7–8
  • Grad-CAM achieved mean Dice (0.143) and IoU (0.077), while HiResCAM achieved 0.129 and 0.071, respectively, and higher Pointing Game accuracy (66.7% vs. 33.3%).Both methods highlighted clinically relevant lesions; their modest Dice and IoU values reflect classification explanations rather than dedicated lesion-segmentation algorithms, and the three metrics provide complementary evidence of spatial correspondence.Source: §4.1.1 · p. 8

5. DISCUSSION

The discussion reports competitive internal and external performance from vision foundation models, alongside calibration and lesion-localization evaluation, but stresses that clinical deployment requires further validation. It identifies dataset and explainability limitations and outlines broader validation and modeling priorities.

5.1. Clinical Significance

  • DINOv2-LoRA achieved the highest internal AUROC, while ViT-Full achieved the highest external AUROC by a very small margin over DINOv2-Full.LoRA offered a parameter-efficient alternative with competitive predictive performance; calibration assessed predicted-probability reliability, and Grad-CAM and HiResCAM showed spatial correspondence with expert-annotated lesions.Source: §5.1 · p. 8

5.2. Limitations

  • The study’s public benchmark datasets may not reflect routine clinical variability, and its evaluation covered binary detection and combined lesion masks rather than severity grading or separate lesion classes.Explainability analysis was limited to IDRiD images with expert masks; Grad-CAM and HiResCAM localize regions but do not reveal the full decision process or establish causality. Figure 4 shows three correctly classified DR-positive images with confidence greater than 0.80; quantitative metrics used a combined lesion mask and excluded the optic disc.Source: §5.2 · p. 8, Figure 4 · p. 9

5.3. Future Work

  • Future work should test larger multi-center datasets, additional retinal foundation models and multimodal approaches [27], more attribution and causal methods, lesion-level validation, severity classification, and prospective clinical evaluation.These studies would assess robustness across populations, imaging devices, and acquisition settings.Source: §5.3 · p. 9

5.4. Conclusion

  • The study combines predictive performance, external generalization, calibration, and lesion-mask explainability evaluation, but multi-center and prospective clinical validation remains necessary before deployment.The framework compared full fine-tuning, linear probing, and LoRA.Source: §5.4 · p. 9

Related papers

Loading 2608.28207v1…