PaperlayerView exact revision on arXiv

Source-linked AI summary

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, Ling Shao

arXiv:2008.09772v3cs.CV

TL;DR

The paper addresses limited and inconsistent fine-grained annotations for interpretable diabetic retinopathy diagnosis. It constructs FGADR and benchmarks segmentation, joint grading, and ocular multi-disease transfer learning, with lesion-supervised grading and inductive transfer methods improving reported outcomes.

  • Problem

    Existing DR diagnosis research is constrained by limited training data and inconsistent annotations, especially the scarcity of pixel-level lesion labels needed for interpretable analysis.

  • Method

    The paper constructs FGADR and evaluates lesion segmentation, joint classification and segmentation for DR grading, and inductive transfer learning for ocular multi-disease identification.

  • Results

    Joint classification and segmentation methods improve DR grading, while the proposed transfer method improves ocular multi-disease identification on ODIR-5K.

  • Takeaways & Limitations

    FGADR provides a benchmark for studying fine-grained DR segmentation, grading, and transfer learning across ocular diseases.

Abstract

from arXiv · show

People with diabetes are at risk of developing an eye disease called diabetic retinopathy (DR). This disease occurs when high blood glucose levels cause damage to blood vessels in the retina. Computer-aided DR diagnosis is a promising tool for early detection of DR and severity grading, due to the great success of deep learning. However, most current DR diagnosis systems do not achieve satisfactory performance or interpretability for ophthalmologists, due to the lack of training data with consistent and fine-grained annotations. To address this problem, we construct a large fine-grained annotated DR dataset containing 2,842 images (FGADR). This dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency. The proposed dataset will enable extensive studies on DR diagnosis. We set up three benchmark tasks for evaluation: 1. DR lesion segmentation; 2. DR grading by joint classification and segmentation; 3. Transfer learning for ocular multi-disease identification. Moreover, a novel inductive transfer learning method is introduced for the third task. Extensive experiments using different state-of-the-art methods are conducted on our FGADR dataset, which can serve as baselines for future research.

I. INTRODUCTION

The paper motivates fine-grained diabetic retinopathy analysis and introduces FGADR as a benchmark for lesion segmentation, grading, and transfer learning.

  • Motivation: Diabetic retinopathy grading is clinically important but time-consuming and challenging for ophthalmologists, motivating automated diagnosis systems.The disease can damage retinal blood vessels and lead to blindness.
  • Motivation: DR severity is graded from 0 to 4 using lesion appearances and complications, making lesion identification relevant to grading.The described lesions include microaneurysms, hemorrhages, exudates, intraretinal microvascular abnormalities, and neovascularization.
  • Research gap: Existing DR datasets are limited by small amounts of training data and inconsistent, mostly image-level annotations.Pixel-level lesion annotations are identified as beneficial for segmentation and training diagnosis models.
  • Dataset and contributions: FGADR contains 1,842 fundus images with pixel-level lesion and image-level grading annotations, plus 1,000 images with grading labels.The dataset supports exploration of segmentation, classification, transfer learning, supervised learning, and semi-supervised learning.
  • Dataset and contributions: Three benchmark tasks evaluate lesion segmentation, joint classification and segmentation for DR grading, and transfer learning for ocular multi-disease identification.The third task uses a novel inductive transfer learning method with multi-scale transfer connections and domain-specific adversarial adaptation.

II. DATASETS

The paper contrasts commonly used DR datasets with FGADR’s fine-grained annotations and uses ODIR-5K for multi-disease transfer learning.

  • Dataset comparison: Most existing DR datasets provide image-level grading labels with few pixel-level lesion annotations, limiting interpretability of predictions.The paper presents FGADR as a response to this annotation gap.
  • Public DR datasets: One described dataset has 35,126 training images and 53,576 testing images with grading labels, varied lighting, weak annotation quality, and image artifacts.The labels use a 0-to-4 DR severity scale.
  • Public DR datasets: Another described dataset contains 3,662 training images and 1,928 testing images with grading labels only and noise in images and labels.
  • ODIR-5K: ODIR-5K is a structured dataset of 5,000 patients with multilabel annotations for eight ocular disease categories.The paper uses it to study transfer learning from DR to ocular multi-disease identification.

4) Messidor [28]:

The referenced datasets illustrate varied retinal annotations and grading information, while FGADR provides fine-grained lesion labels and additional image-level lesion annotations.

  • Messidor provides four-level diabetic retinopathy grading and macular-edema risk labels for 1,200 fundus images.
  • FGADR pixel-level examples cover six lesions: microaneurysms, hemorrhages, soft and hard exudates, IRMA, and neovascularization.
  • The referenced lesion dataset contains 516 images, but only 81 have pixel-level binary lesion masks.
  • FGADR additionally labels laser marks and proliferative membranes at image level because their global-like appearance complicates pixel-wise annotation.

1) Dataset Construction and Labeling:

FGADR construction combines privacy-preserving image selection with extensive ophthalmologist annotation and reports lesion and grade distributions across its two sets.

  • FGADR excludes duplicate retinal structures by selecting one high-quality image per patient and favors severe images containing more lesions.
  • The Grade-set uses labels from six ophthalmologists, with final grades determined by voting to support grading evaluation.
  • Seg-set annotation took over 10 months and used three ophthalmologists plus quality control to improve labeling accuracy.
  • Microaneurysms, hemorrhages, and hard exudates are most common, whereas IRMA, neovascularization, laser marks, and proliferative membranes are rare.
  • Seg-set contains 1,842 images and Grade-set contains 1,000, with different distributions across grades 0–4.
  • Microaneurysms usually appear in early grades, lesion counts generally increase with severity, and neovascularization, laser marks, and proliferative membranes help distinguish grades 3 and 4.

III. BENCHMARK SETTINGS FOR DR LESION SEGMENTATION, GRADING, AND TRANSFER LEARNING

The benchmark evaluates FGADR through lesion segmentation, joint lesion-aware grading, and transfer learning for other ocular diseases.

  • The benchmark defines three tasks: lesion segmentation, DR grading by joint classification and segmentation, and transfer learning for multi-disease identification.
  • Task 1: Lesion Segmentation: Task 1 evaluates six binary lesion-segmentation subtasks on Seg-set using two-fold cross validation with equal training and testing splits.
  • Task 2: DR Grading: Task 2 evaluates grading on 1,000 Grade-set test images and combines classification with lesion segmentation for diagnosis.
  • Task 2: DR Grading: Task 2 uses joint classification-segmentation methods as baselines and additionally co-trains classifiers with image-level laser-mark and proliferative-membrane labels.

C. Task 3: Inductive Transfer Learning for Ocular Multi-Disease Identification

The paper proposes inductive transfer from DR lesion segmentation to multi-disease classification, using shared retinal lesion knowledge while adapting source and target representations.

  • Task 3 tests whether FGADR’s fine-grained lesion annotations improve identification of other eye diseases on ODIR-5K.
  • Inductive transfer is appropriate because the source and target are fundus-image domains with different tasks: lesion segmentation and multi-disease classification.
  • The method uses a lesion-segmentation source module, a multi-label classification target module, and multi-scale transfer connections.
  • Separate source and target batch normalization adapts representations across domain shifts and enhances transferred features.
  • The overall objective combines source loss, weighted target loss, and adversarial adaptation loss, with λ and γ controlling their weights.
  • A Dense U-Net segments source-domain lesions, while target-domain features incorporate corresponding multi-scale segmentation features through transfer connections.
  • The classification module transfers descriptive segmentation representations while using weighted binary cross-entropy for its target objective.

D. Evaluation Metrics

Task 1 evaluates lesion segmentation using Dice, AUC-ROC, AUC-PR, and MAE, with sigmoid outputs compared against pixel-level ground truth.

  • Four metrics evaluate lesion segmentation: Dice, AUC-ROC, AUC-PR, and MAE.The final prediction map is compared with pixel-level segmentation ground truth.
  • Dice measures region overlap between the predicted segmentation map and ground truth.
  • AUC-ROC compares sensitivity with 1 − specificity, measuring true-positive versus false-positive discrimination.Higher AUC-ROC indicates greater distinction between true positives and true negatives.

3) AUC-PR:

AUC-PR focuses on positive-class performance and is more suitable than ROC for imbalanced data, while grading and multi-disease tasks use agreement and classification metrics.

  • 3) AUC-PR:: Precision-recall curves compare positive predictive value with true-positive rate while ignoring true negatives.
  • 3) AUC-PR:: For imbalanced data, precision-recall analysis is more suitable than ROC because it focuses on the minority positive class.
  • Task 2 evaluates five-grade DR classification using accuracy, a confusion matrix, and quadratic weighted kappa.
  • Quadratic weighted kappa is computed from observed and expected rating distributions using a confusion matrix and quadratic weights.
  • Task 3 evaluates multi-disease classification with Cohen’s kappa, F-1 score, and AUC-ROC.

7) F-1 Score:

F-1 score balances precision and recall, making it useful when class distributions are uneven; the segmentation evaluation also includes established neural architectures.

  • 7) F-1 Score:: F-1 score balances precision and recall in a single comparison indicator.
  • 7) F-1 Score:: F-1 is used for uneven class distributions because precision and recall alone may give misleading results.
  • Lesion segmentation experiments compare classic semantic segmentation methods from Non-U-Net and U-Net framework categories.
  • Dense U-Net integrates a densely connected convolutional network into U-Net to strengthen feature use and improve segmentation performance.

2) Grading:

The benchmark evaluates DR grading with classification, ensemble, lesion-informed, and transfer-learning baselines, alongside segmentation results and ablations of the proposed transfer method.

  • 2) Grading:: Task 2 rates DR severity from 0 to 4 as a five-grade classification problem using individual, ensemble, and lesion-informed baselines.
  • 2) Grading:: Individual grading baselines use VGG-16, ResNet-50, Inception v3, and DenseNet-121 backbones.
  • 2) Grading:: Ensemble baselines average predictions from multiple models, including three-model and eight-model solutions.
  • 2) Grading:: Lesion-informed methods use visual attention or predicted lesion masks to support DR grading, with DenseNet-121 used for comparison in two methods.
  • 2) Grading:: Transfer-learning ablations test multiscale transfer connections and adversarial domain-specific adaptation against a target-domain classification baseline.

B. Results of Task: DR Lesion Segmentation

Lesion segmentation comparisons favor U-Net-based models, while lesion masks also improve downstream DR grading outcomes. These results support evaluating segmentation alongside grading.

  • Segmentation model comparison: Dense U-Net and U-Net++ achieve the best segmentation results across lesions except hard exudates.Hard exudates remain relatively easy to segment, with no method showing dominant performance.
  • Segmentation model comparison: Multi-class U-Net slightly improves AUC of PR over standard U-Net while significantly reducing computational cost.
  • Segmentation model comparison: U-Net frameworks consistently outperform non-U-Net frameworks, consistent with their upsampling and skip connections for propagating context to higher-resolution layers.
  • Downstream grading: Lesion segmentation predictions notably improve DR grading, increasing grade-1 and grade-3 classification accuracies by 12.8% and 15.25%, respectively.The misclassification rate from grade-2 to grade-0 decreases by 6.89%.

D. Results of Task 3: Ocular Multi-Disease Identification

The ocular multi-disease task evaluates transfer from fine-grained DR data using multi-scale connections and domain-specific adversarial adaptation. These components improve identification performance and produce more precise lesion-related regions.

  • Transfer learning results: On ODIR-5K, multi-scale transfer connections increase Kappa by 2.87%, while domain-specific adversarial adaptation adds a further 5.05% increase.The evaluation uses 7,000 images for training and validation with five-fold cross-validation.
  • Transfer learning results: Separate batch-normalization layers in the domain-specific discriminator increase Kappa by 1.96% compared with normal adversarial adaptation.
  • Transfer learning results: Transfer learning from the fine-grained annotated DR domain consistently improves identification results across all ocular diseases.The source-domain lesion segmentation ability is integrated into the target-domain network to obtain more precise lesion regions related to disease.
  • Method contribution: The paper introduces DSAA as an inductive transfer learning method for improving ocular multi-disease identification.
Loading 2008.09772v3…