Lesion detection and Grading of Diabetic Retinopathy via Two-stages Deep Convolutional Neural Networks

Yehui Yang; Tao Li; Wensi Li; Haishan Wu; Wei Fan; Wensheng Zhang

Source-linked AI summary

Lesion detection and Grading of Diabetic Retinopathy via Two-stages Deep Convolutional Neural Networks

Yehui Yang, Tao Li, Wensi Li, Haishan Wu, Wei Fan, Wensheng Zhang

arXiv:1705.00771v1May 2, 2017cs.CV

TL;DR

Automated DR analysis needs to identify clinically relevant lesions while grading disease severity, rather than producing only a black-box grade. The paper proposes a two-stage DCNN with local lesion detection, global grading, and an imbalanced lesion-weighting map; experiments report lesion-detection performance comparable to trained observers and improved grading capability.

Problem
Existing DR grading methods may provide severity grades without indicating lesion location or type, while tiny lesions can be overwhelmed during end-to-end grading despite their importance.
Method
A two-stage DCNN uses a local network for lesion detection, a global network for severity grading, and a weighted lesion map to emphasize lesion patches.
Results
A lesion-patch classifier achieved an AUC of 0.9687 and was reported to outperform trained human observers at a proper operating point.
Takeaways & Limitations
The complementary local and global networks provide lesion and image-level information for DR analysis, while lesion weighting improves the global grading network’s capability.
Takeaways & Limitations
The study’s annotated training samples need improved quantity and quality, and lesion detection covers only microaneurysms, hemorrhages, and exudates.

Abstract

from arXiv · show

We propose an automatic diabetic retinopathy (DR) analysis algorithm based on two-stages deep convolutional neural networks (DCNN). Compared to existing DCNN-based DR detection methods, the proposed algorithm have the following advantages: (1) Our method can point out the location and type of lesions in the fundus images, as well as giving the severity grades of DR. Moreover, since retina lesions and DR severity appear with different scales in fundus images, the integration of both local and global networks learn more complete and specific features for DR analysis. (2) By introducing imbalanced weighting map, more attentions will be given to lesion patches for DR grading, which significantly improve the performance of the proposed algorithm. In this study, we label 12,206 lesion patches and re-annotate the DR grades of 23,595 fundus images from Kaggle competition dataset. Under the guidance of clinical ophthalmologists, the experimental results show that our local lesion detection net achieve comparable performance with trained human observers, and the proposed imbalanced weighted scheme also be proved to significantly improve the capability of our DCNN-based DR grading algorithm.

I. INTRODUCTION

The paper addresses the need for automated DR analysis that both grades disease severity and identifies lesion location and type. Its two-stage DCNN combines local lesion detection with global grading, using lesion-focused weighting to improve grading.

Motivation: DR severity spans five stages, while early disease signs include lesions such as microaneurysms, hemorrhages, and exudates.The stages are normal, mild, moderate, severe, and proliferative.
Research gap: Existing DCNN grading methods output DR grades but do not indicate lesion location or type, limiting the detailed information available to clinicians.The paper characterizes this as a black-box limitation for treatment.
Research gap: Tiny lesions can be overwhelmed in end-to-end image processing even though they are critical for DR grading under the international standard.Examples include microaneurysms and some small hemorrhages.
Proposed approach: The proposed two-stage DCNN uses a local network for lesion detection and a global network for holistic DR grading.The design targets complementary local and global image features.
Proposed approach: A weighted lesion map assigns greater attention to patches containing more severe lesions during global-network training.This replaces direct end-to-end grading with lesion-informed weighting.
Contributions: The paper presents the algorithm as an advance over existing DCNN-based DR detection methods by combining lesion detection, severity grading, and imbalanced attention.These are stated as the paper’s contributions and advantages.

II. RELATED WORK

Prior work established DCNNs as powerful tools for computer vision and automated DR grading, but many methods treated grading as a black-box task. The paper positions its local-global approach as a way to incorporate lesion-level information into DR analysis.

DCNN background: DCNNs comprise convolution, subsampling, and fully connected layers and have been applied across major computer-vision tasks.Examples include image classification, object detection, segmentation, action recognition, and visual saliency detection.
DR grading: Automated DR grading research has used architectures such as Inception-v3, AlexNet, and GoogleNet, including methods developed for Kaggle competition datasets.The cited work covers referable DR detection and severity grading.
Limitations of prior work: Earlier DCNN DR methods generally produced grades without providing lesion information for screening.Related lesion-detection work separately targeted hemorrhages, microaneurysms, or transformed retinal representations.
Positioning of this work: The proposed method extracts local and global fundus-image features, with lesion information from the local network improving global DR grading.The paper presents the local and global networks as complementary components.

III. METHODS

The proposed method uses a two-stage DCNN pipeline that divides fundus images into overlapping patches, detects lesions locally, and grades DR globally.

The workflow preprocesses fundus photographs, divides them into patches, generates a weighted lesion map, and grades DR severity with a global network.
A. Image Division Via Overlap Grids: Overlapping grids preserve patch locations and reduce misclassification when small lesions fall near conventional grid edges.The grid size is h × h, with sliding stride h−ov for overlap ov.
B. Local Network: The local network classifies patches as normal, microaneurysm, hemorrhage, or exudate to detect major NPDR indicators.
B. Local Network: The architecture combines convolutional, max-pooling, and fully connected layers with ReLU, batch normalization, and dropout.Patch labels are produced by soft-max regression, and the detailed architecture appears in Table I.

C. Weighed Lesion Map

The weighted lesion map combines local patch labels and confidence scores to emphasize lesion-bearing regions before global DR grading.

The local network produces a label map and a probabilistic map for all patches in each fundus image.The label map records predicted patch labels, while the probabilistic map retains each patch’s largest softmax output probability.
The weighting matrix is formed as LP = (L + 1) ⊙P, expanded to patch size, and tiled according to the patches’ relative image locations.Values in overlapping regions are averaged between adjacent expanded matrices.
The weighted lesion map I∗= MI ⊙I assigns higher weights to patches with more severe lesions and higher predicted probabilities.The all-one matrix prevents completely removing information from patches labeled normal.
Feeding the weighted lesion map to the global network directs more attention toward patches with severe lesions during DR grading.
The weighting scheme outperforms traditional end-to-end grading networks under the same implementation setup.

D. Global Network

The global network grades DR severity according to the International Clinical Diabetic Retinopathy scale, focusing on four classes from normal through severe NPDR.

The global network grades fundus-image severity according to the International Clinical Diabetic Retinopathy scale.
Class 0 represents images with no visible lesions or abnormalities.
Class 1 represents mild NPDR characterized only by microaneurysms.
Class 2 represents moderate NPDR with extensive microaneurysms, haemorrhages, and hard exudates.
Class 3 represents severe NPDR involving venous abnormalities, large blot haemorrhages, cotton wool spots, and related vascular changes.

IV. EXPERIMENTS, RESULTS AND DISCUSSIONS

The study re-annotates a Kaggle subset and lesion patches, then preprocesses fundus images and applies augmentation before evaluation. Clinical ophthalmologists participate in checking lesion and DR-grade annotations.

Datasets: 22,795 randomly selected NPDR fundus images were re-annotated, with 21,995 for training and 800 for testing.Each NPDR grade contributes 200 testing images.
Datasets: 12,206 lesion patches and over 140,000 randomly cropped normal patches were prepared for lesion detection.Licensed ophthalmologists and trained graduate students annotated lesions and re-annotated fundus-image DR grades.
Data Preparation: Preprocessing improves contrast and extracts a circular region of interest before lesion detection and grading.The preprocessing output is described as making some underlying lesions easier to exploit than in raw images.
Data Preparation: Lesion-detection images are resized to 800×800, while global-network inputs are resized to 256×256 to reduce computational complexity.The relative image ratio is preserved by padding before resizing raw images.
Reference Standard and Annotation: Training annotations combine Kaggle labels, trained-observer annotations, and clinical-ophthalmologist review.Testing references require agreement among all annotators after discrepancy patches are further analyzed.

C. Evaluation design and Metrics

The evaluation measures lesion recognition and DR grading with class-wise and binary classification metrics. Experiments use recall, precision, accuracy, sensitivity, specificity, ROC curves, AUC, and kappa.

Evaluation Design: Lesion recognition is evaluated using per-lesion recall and precision, plus binary lesion-versus-normal ROC and AUC analysis.The design also compares the local network with baseline algorithms.
Multi-class Metrics: Recall is the proportion of correctly predicted samples among all testing samples belonging to class i.Precision instead uses the number of samples correctly predicted for class i over all samples predicted as class i.
Binary and Grading Metrics: Accuracy is the total number of correct predictions divided by the number of testing samples.It is used for lesion-versus-normal patches and referable-DR-versus-normal fundus images.
Binary Metrics: Sensitivity and specificity are defined from true positives, false negatives, true negatives, and false positives.Sensitivity uses TP and FN, while specificity uses TN and FP.

D. The Identification of Lesions

The local network identifies lesion types from fundus-image patches and is compared with random forests and SVMs. Its binary lesion detector is also assessed against trained human observers using ROC analysis.

Lesion Recognition: The proposed local network significantly outperforms random forests and SVM under the same training-image conditions for lesion recognition.Recall and precision are recorded separately for each lesion class.
Lesion Recognition: MA and hemorrhage recall and precision are relatively worse than exudate in the proposed algorithm.The paper attributes this to small, inconspicuous MAs, MA–hemorrhage similarity, and less strict training annotations.
Confusion Analysis: Figure 5 records true lesion classes by predicted classes, with correct predictions shown in red.Rows and columns use class indices 0 through 3.
Human Comparison: The ROC comparison includes high-sensitivity and high-specificity operating points for the algorithm and binary lesion-detection results from two trained human observers.The algorithm's ROC curve is shown in red, while observer results are shown as green and blue dots.
Binary Lesion Detection: AUC = 0.9687 for binary lesion-versus-normal patch detection.The reported operating points are sensitivity 0.863, specificity 0.973 and sensitivity 0.959, specificity 0.898.

E. Grading The Severity of Fundus Images

The grading experiments focus on four NPDR stages and compare weighted with non-weighted lesion-map schemes. Weighting improves both tested grading networks, while referable-DR detection reaches a reported AUC of 0.9590.

Task Definition: The NPDR grading task covers four stages: normal, mild, moderate, and severe.These stages are numbered 0 to 3 in the paper's grading setup.
Weighted Grading: Both the proposed global network and AlexNet achieve higher kappa and accuracy with the weighted lesion map than without weighting.The comparison uses the same implementation setup for weighted and non-weighted schemes.
Referable DR Detection: AUC = 0.9590 for referable DR detection with the weighted scheme.The weighted ROC curve is shown in red and the non-weighted curve in blue.

V. DISCUSSION

The proposed two-stage pipeline combines local lesion detection with global DR grading, while an imbalanced weighting scheme improves grading by emphasizing lesion information. The authors report reasonable lesion-detection and NPDR-grading results but identify annotation and lesion-coverage limitations.

The local network detects lesions, while the global network grades DR using holistic image features.
The global network with a weighting lesion map outperforms the unweighted approach, supporting the imbalanced weighting scheme for DR grading.The weighting matrix supplies lesion information, while joint image and weighting features can smooth some local-network grading mistakes.
The complementary local and global networks provide more abundant information for ophthalmologists and patients in DR analysis.
The quantity and quality of carefully annotated training samples need improvement.The authors note that more professional observers could contribute to annotation, although resources limited the study.
The study detects only microaneurysms, hemorrhages, and exudates, leaving other important abnormalities unaddressed.Examples include venous beading in NPDR and neovascularization in PDR.
The pipeline and imbalanced weighting framework are architecture-independent because the local and global networks can use any architecture with better performance.

VI. CONCLUSION

The paper concludes that a two-stage DCNN can detect abnormal lesions and DR severity grades in fundus images. It reports useful information for clinical examination while identifying data quality, lesion coverage, and diabetic macular edema as future challenges.

The proposed two-stage DCNN detects abnormal lesions and DR severity grades in fundus images.
The study reports that its experimental results demonstrate the algorithm’s effectiveness and provide valuable information for clinical ophthalmologists.
Future work should collect more high-quality annotated fundus data and address more lesion types.
Diabetic macular edema remains an important open issue for future investigation.