Source-linked AI summary
Deep Learning Approach to Diabetic Retinopathy Detection
Borys Tymchenko, Philip Marchenko, Dmitry Spodarets
TL;DR
Early diabetic retinopathy staging from fundus photographs is difficult because clinicians may disagree and the disease can lack early warning signs. The paper proposes multistage transfer learning with an ensemble of CNNs for single-image stage detection. The method achieved a quadratic weighted kappa of 0.925466 and ranked 54 of 2943 methods on the APTOS2019 test dataset.
Problem
Early diabetic retinopathy staging from fundus images is difficult because disease may lack warning signs and clinicians can disagree, creating unstable ground truth.
Method
The paper uses multistage transfer learning and an ensemble of CNN architectures to detect diabetic retinopathy stage from a single fundus photograph.
Results
0.925466 quadratic weighted kappa and rank 54 of 2943 methods were achieved on the 13000-image APTOS2019 test dataset.
Takeaways & Limitations
The method could be used as a diabetic retinopathy stages screening method in automatic solutions.
Takeaways & Limitations
Kappa is volatile because its value depends on rating proportions and the number of categories and may be low despite high agreement.
Abstract
from arXiv · showhide
Diabetic retinopathy is one of the most threatening complications of diabetes that leads to permanent blindness if left untreated. One of the essential challenges is early detection, which is very important for treatment success. Unfortunately, the exact identification of the diabetic retinopathy stage is notoriously tricky and requires expert human interpretation of fundus images. Simplification of the detection step is crucial and can help millions of people. Convolutional neural networks (CNN) have been successfully applied in many adjacent subjects, and for diagnosis of diabetic retinopathy itself. However, the high cost of big labeled datasets, as well as inconsistency between different doctors, impede the performance of these methods. In this paper, we propose an automatic deep-learning-based method for stage detection of diabetic retinopathy by single photography of the human fundus. Additionally, we propose the multistage approach to transfer learning, which makes use of similar datasets with different labeling. The presented method can be used as a screening method for early detection of diabetic retinopathy with sensitivity and specificity of 0.99 and is ranked 54 of 2943 competing methods (quadratic weighted kappa score of 0.925466) on APTOS 2019 Blindness Detection Dataset (13000 images).
1 INTRODUCTION
Diabetic retinopathy damages retinal blood vessels and can cause blindness, while early staging from fundus images is difficult and subject to clinician disagreement. The paper proposes automatic CNN-based detection using transfer learning for screening.
- Diabetic retinopathy damages retinal blood vessels, distorts vision, and can cause blindness.
- DR progresses through four stages, each with distinct characteristics that may be overlooked during diagnosis.
- 56% of new cases could be reduced with proper and timely treatment and eye monitoring, but early disease may have no warning signs.
- Clinicians may disagree when evaluating fundus images, producing incorrect diagnoses and unstable ground truth for automated solutions.
- CNNs have been applied to diabetic retinopathy diagnosis and related classification and object-detection tasks.
- The proposed method detects DR stage from a single fundus photograph using transfer learning and is intended as a screening method.
2 RELATED WORK
Earlier diabetic retinopathy work used classical image processing, SVMs, and CNNs, while later studies applied ImageNet-pretrained CNN architectures to five-class classification.
- Classical approaches extracted features from color fundus images and used SVMs for diabetic retinopathy classification.
- CNN-based methods used augmentation to identify retinal features such as microaneurysms, exudates, and hemorrhages automatically.
- ImageNet-pretrained InceptionNet V3, ResNet50, Xception, DenseNet, and VGG models were evaluated for five-class classification.
3 PROBLEM STATEMENT
The study combines multiple fundus-image datasets for five-class DR evaluation and uses quadratic weighted Cohen’s kappa as its main metric, while noting important interpretive limitations.
- 3.1 Datasets: EyePACs provided 35126 fundus photographs for CNN pretraining, while IDRiD and MESSIDOR supplied additional smaller datasets.
- 3.1 Datasets: The five DR classes range from no diabetic retinopathy through mild, moderate, severe, and proliferative disease.
- 3.1 Datasets: APTOS2019 contains 18590 photographs divided into 3662 training, 1928 validation, and 13000 testing images.
- 3.1 Datasets: The datasets were treated as having similar class distributions, without undersampling or oversampling.
- 3.2 Evaluation metric: Quadratic weighted Cohen’s kappa measures agreement between human and predicted ratings on a scale from -1 to 1.
- 3.2 Evaluation metric: Kappa interpretation is sensitive to rating proportions and the number of categories, and may be low despite high agreement.
- 3.2 Evaluation metric: Validation and test labels were unavailable, so Kaggle submissions provided their kappa values; additional metrics were calculated on 736 holdout images.
4 METHOD
The DR detection task can be formulated as classification, regression, or ordinal regression because disease stages occur sequentially.
- DR stage detection can be treated as classification, regression, or ordinal regression because the stages are sequential.
4.1 Preprocessing
The preprocessing pipeline crops fundus images and resizes them before model training. It also addresses spurious correlations between disease stage and image meta-features through augmentation.
- Images are cropped and then resized before training and validation.
- APTOS2019 contains spurious correlations between disease stage and resolution, crop type, zoom level, and brightness.
- Augmentation exposes models to greater data variance to reduce overfitting to meta-features and content–meta-feature correlations.
4.2 Data augmentation
The data-augmentation strategy applies diverse online image transformations before CNN input, aiming to expose models to variance and reduce reliance on spurious image characteristics.
- At least one Albumentations transformation is applied online before each training image enters the CNN.Transformations include geometric distortions, flips, rotations, color changes, noise, blur, sharpening, gamma adjustment, and cutout.
- The augmentation strategy targets spurious correlations between image meta-features and diabetic-retinopathy diagnosis.
4.3 Network architecture
The model uses an ImageNet-initialized CNN feature extractor with three task-specific heads for classification, regression, and ordinal regression. A frozen linear model later combines their outputs.
- The architecture consists of a CNN feature extractor and smaller task-specific decoders, or heads.
- ImageNet-pretrained CNNs initialize the encoder, while three decoders perform multi-task diabetic-retinopathy detection.
- The three heads are classification, regression, and ordinal regression.
- Classification outputs a one-hot stage vector, regression outputs a value in [0,4.5) rounded to a stage, and ordinal regression predicts all categories below the target.
- A linear regression model combines the outputs of the three heads and remains frozen until post-training.
4.4 Training process
Training proceeds through staged pretraining, main training, post-training combination, and regularization. The final system ensembles cross-validated models with test-time augmentation and trimmed-mean aggregation.
- Training stages: The method uses different training settings and datasets across multiple stages.
- Pretraining: The largest 2015 dataset pretrains CNNs because labeling schemes differ across datasets, while transfer learning relies on consistent natural DR features.
- Pretraining: Different acquisition equipment motivates incorporating dataset variation to reduce sensitivity to instrument noise and improve generalization.
- Pretraining: Pretraining uses ImageNet-initialized encoder weights, randomly initialized heads, 20 epochs, minibatch-SGD, and cosine-annealing learning rates.
- Pretraining: The three heads minimize cross-entropy, binary cross-entropy, and mean absolute error losses during pretraining.
- Main training: Subsequent stages reuse encoder weights but reinitialize heads after pretraining because this consistently improved metrics in the experiments.
- Main training: Main training combines 2019, IDRID, and MESSIDOR data, using 5-fold cross-validation and holdout evaluation.
- Main training: Main-stage losses are focal loss for classification, binary focal loss for ordinal regression, and mean-squared error for regression.
5 RESULTS
Evaluation used local and Kaggle testing to compare ensembles with and without test-time augmentation (TTA). TTA slightly reduced local performance but improved the 13,000-image testing result and maintained a strong competition rank.
- The test stage comprised local testing and Kaggle validation and testing, with ensembling selected as the best local method.
- Ensembling with TTA performed slightly worse on the local dataset of 736 images than without TTA.
- Ensembling with TTA performed better on the 13,000-image testing dataset, where it generalized better to unseen images.
- 0.818462/0.924746 validation/test QWK was achieved without TTA, compared with 0.826567/0.925466 using TTA.
- Binary classification was also evaluated to assess model quality as a screening method.
- The TTA ensemble ranked 58th on validation and 54th on testing among 2,943 methods.
6 INTERPRETATION
The interpretation section addresses model interpretability using SHAP visualizations. These visualizations identify features contributing to disease-stage predictions and indicate the saliency and direction of regional contributions.
- Validation performance alone was considered insufficient for real-life medical use, motivating interpretation of model predictions.
- SHAP provides locally accurate additive feature attributions for visualizing features that contribute to disease-stage assessment.
- SHAP was used to assess whether the model learned and used appropriate features during training and inference.
- Visualizing salient features can help physicians focus on regions of interest in uncertain cases.
- In Figure 8, red regions increase the output for a class, blue regions decrease it, and intensity indicates regional saliency.
7 CONCLUSION
The paper proposes multistage transfer learning and automatic diabetic-retinopathy stage detection from a single fundus photograph. It combines pretrained CNN ensembles with fine-tuning to improve generalization and reduce variance.
- The method uses an ensemble of three CNN architectures—EfficientNet-B4, EfficientNet-B5, and SE-ResNeXt50—with transfer learning.
- The authors report high and stable experimental results despite an unstable metric.
- The ensemble is intended to increase generalization and reduce variance through large-dataset pretraining and target-dataset fine-tuning.
- Future work includes SHAP calculation for the whole ensemble, improved hyperparameter optimization, related eye-ailment tasks, and meta-learning.