Source-linked AI summary
An Ensemble Deep Learning Based Approach for Red Lesion Detection in Fundus Images
José Ignacio Orlando, Elena Prokofyeva, Mariana del Fresno, Matthew B. Blaschko
TL;DR
Red-lesion detection supports earlier diabetic-retinopathy assessment, but manual screening is laborious and deep-learning methods are limited by lesion-level annotation costs. The paper combines CNN features with hand-crafted descriptors and classifies candidates using a Random Forest. This hybrid strategy improved lesion-level and image-level performance across benchmark datasets and achieved higher reported AUC values than competing approaches.
Problem
Manual red-lesion screening is laborious, while deep-learning approaches remain scarce because lesion-level annotations are expensive.
Method
The method extracts CNN features from candidate patches, augments them with hand-crafted descriptors, and uses a Random Forest to refine lesion candidates.
Results
The hybrid approach improved lesion-level performance over either feature source alone and achieved higher AUC values than other approaches for DR screening and need-for-referral detection.
Takeaways & Limitations
Combining deep learned features with domain knowledge provides statistically significant improvements for red-lesion detection and related screening evaluations.
Takeaways & Limitations
Candidate detection produces false positives from vessels, scars, and capture-device dirt, requiring a refinement stage.
Abstract
from arXiv · showhide
Diabetic retinopathy is one of the leading causes of preventable blindness in the world. Its earliest sign are red lesions, a general term that groups both microaneurysms and hemorrhages. In daily clinical practice, these lesions are manually detected by physicians using fundus photographs. However, this task is tedious and time consuming, and requires an intensive effort due to the small size of the lesions and their lack of contrast. Computer-assisted diagnosis of DR based on red lesion detection is being actively explored due to its improvement effects both in clinicians consistency and accuracy. Several methods for detecting red lesions have been proposed in the literature, most of them based on characterizing lesion candidates using hand crafted features, and classifying them into true or false positive detections. Deep learning based approaches, by contrast, are scarce in this domain due to the high expense of annotating the lesions manually. In this paper we propose a novel method for red lesion detection based on combining both deep learned and domain knowledge. Features learned by a CNN are augmented by incorporating hand crafted features. Such ensemble vector of descriptors is used afterwards to identify true lesion candidates using a Random Forest classifier. We empirically observed that combining both sources of information significantly improve results with respect to using each approach separately. Furthermore, our method reported the highest performance on a per-lesion basis on DIARETDB1 and e-ophtha, and for screening and need for referral on MESSIDOR compared to a second human expert. Results highlight the fact that integrating manually engineered approaches with deep learned features is relevant to improve results when the networks are trained from lesion-level annotated data. An open source implementation of our system is publicly available online.
1. Introduction
Diabetic retinopathy can be clinically silent while its early red-lesion signs are small and difficult to detect manually. Existing detection commonly relies on hand-crafted features, motivating a hybrid approach that combines CNN-learned descriptors with domain knowledge.
- Diabetic retinopathy is a leading cause of preventable blindness, and its prevalence is expected to rise from 2.8% to 4.4% between 2000 and 2030.
- Red lesions comprise microaneurysms and hemorrhages, with microaneurysms appearing early and some later producing blood leakages.
- DR may be asymptomatic initially, so regular fundus examinations are recommended despite the intensive effort required for manual screening.
- Automated screening can reduce analysis time, cost, effort, and intra-expert variability, but red-lesion detection typically uses candidate generation followed by hand-crafted-feature classification.
- The proposed method combines CNN-learned and manually engineered features, then uses a Random Forest to distinguish true lesions from false candidates.
- The combined strategy improved performance over either feature source alone and outperformed existing approaches across lesion-level and image-level evaluations.
2. Methods
The method detects red-lesion candidates morphologically, then refines them with CNN-derived and hand-crafted descriptors in a Random Forest classifier.
- 2.1. Candidate detection: Morphological operations generate potential red-lesion candidates across multiple scales before classification.The green channel is expanded at the field-of-view border, enhanced, filtered, morphologically processed, thresholded, and combined across scales.
- 2.1. Candidate detection: Candidate detection uses a wider simulated field of view and resolution-scaled parameters to reduce border artifacts and support different image resolutions.A 3 30X-pixel expansion is simulated from the green band, with relevant parameters expressed in terms of FOV width X.
- 2.2. CNN-based features: The CNN is trained on centered candidate patches resized to 32 × 32 pixels, capturing lesion appearance, shape, borders, and context.Patch size is twice the candidate’s major axis, or 32 × 32 pixels for smaller candidates; larger patches are downsized.
- 2.2. CNN-based features: The CNN contains four convolutional layers and one 128-unit fully connected layer, producing a 128-dimensional deep-feature vector.The architecture was developed from LeNet, while deeper VGG-S and Inception-V3 architectures were avoided because of their larger parameter counts.
- 2.3. Hand-crafted feature extraction: A 63-dimensional hand-crafted feature vector adds intensity- and shape-based descriptors to the CNN representation.Shape descriptors encode domain expectations such as circularity, small area and perimeter, and similar major and minor axes.
- 2.4. Random Forest classifier: A Random Forest classifies candidates using the hybrid feature vector and is suited to small, noisy, high-dimensional, imbalanced training data.The forest estimates class probabilities for non-lesion and true-lesion candidates by averaging predictions across trees.
3. Experimental setup
The experiments use three public fundus-image datasets, with lesion-level evaluation on DIARETDB1 and e-ophtha and image-level DR evaluation on MESSIDOR. Candidate detection, CNN and Random Forest parameters, and FROC/ROC-based metrics are specified for evaluation.
- Datasets: DIARETDB1, e-ophtha, and MESSIDOR provide the publicly available datasets used in the experiments.DIARETDB1 and e-ophtha include lesion annotations, while MESSIDOR provides image-level DR grades.
- Datasets: DIARETDB1 contains 89 images split into 28 training and 61 test images, with expert annotations for microaneurysms and hemorrhages.
- Experimental parameters: Candidate detection uses scales L = {3, 6, 9, . . . , 60}, K = 120 candidates per scale, and a minimum candidate area of px = 5 pixels.The values were adjusted on the DIARETDB1 training set, while the maximum scale was adapted for other datasets using a scaling factor.
- Experimental parameters: CNN parameters were selected on held-out validation data using N = 128 and p = 0.99, while the Random Forest tree count was selected by minimizing out-of-bag error.
- Evaluation: Per-lesion performance is evaluated with FROC curves and CPM, whereas per-image performance uses ROC curves and sensitivity at specificity Sp = 50%.FROC reports lesion sensitivity against average false-positive detections per image; CPM averages sensitivity at reference FPI values.
4. Results
The hybrid representation generally improves red-lesion detection over CNN-only and hand-crafted features, with statistically significant gains in lesion-level experiments and higher screening AUCs. Qualitative results also show detections that may correspond to lesions omitted from manual ground truth labels.
- Per-lesion evaluation: The hybrid representation significantly improves per-lesion sensitivity over CNN probabilities, hand-crafted features, and the Seoud et al. method.Reported hypothesis-test values are p < 2 × 10^-18 versus CNN features and Seoud et al., and p < 4 × 10^-17 versus hand-crafted features.
- Per-lesion evaluation: 0.2885 sensitivity at FPI = 1 is obtained for microaneurysm detection with the combined approach, versus 0.202 for CNN and 0.2 for hand-crafted features.
- Qualitative analysis: In qualitative DIARETDB1 results, several method-only detections appear to be subtle microaneurysms or hemorrhages omitted from manual ground-truth labeling.The figure distinguishes ground-truth detections, method-only detections, and manually annotated lesions missed by the method.
- Per-image evaluation: On MESSIDOR DR screening, CNN features achieve AUC = 0.7912 versus AUC = 0.7325 for hand-crafted features, while the combination is reported as substantially higher.
- Per-image evaluation: The combined approach achieves significantly higher AUC than CNN and hand-crafted features for DR screening on e-ophtha, reaching AUC = 0.9031.CNN and hand-crafted features obtain AUC = 0.8374 and AUC = 0.8812, respectively; the CNN comparison has p < 5 × 10^-3.
- Feature analysis: The first CNN-layer visualization presents 32 learned filters for training on DIARETDB1 alone versus DIARETDB1 combined with the ROC training set.
5. Discussion
Combining deep learned and hand crafted features improved red lesion detection over either approach alone across lesion-level and image-level evaluations. The hybrid approach also compared favorably with published methods, while t-SNE analyses examined feature complementarity and candidate separation.
- The combined approach consistently improved CPM and per-lesion sensitivity over separate feature approaches on e-ophtha and DIARETDB1.
- The combined approach achieved higher AUC values than other approaches for DR screening and need-for-referral detection on a per-image basis.
- The method slightly exceeded Quellec et al.’s MESSIDOR screening result of AUC=0.893 without additional clinical information.
- t-SNE visualizations compare deep learned, hand crafted, and combined feature mappings for lesion candidates and false detections.
- Candidate extraction increased maximum achievable per-lesion sensitivity, enabling classifier training with more false-positive lesions.
- The CNN and hand crafted approaches differed by lesion type, with individual methods less precise for microaneurysms than for other hemorrhages.
6. Conclusions
The paper concludes that a hybrid descriptor combining CNN-based and hand crafted features improves red lesion detection and screening performance. The authors report state-of-the-art benchmark results and release implementation resources publicly.
- The proposed detector combines CNN-based and hand crafted features in a hybrid vector classified by a Random Forest.
- Benchmark experiments showed statistically significant improvements from combining both feature sources compared with using either separately.
- The system achieved higher AUC values than existing approaches for diabetic-retinopathy screening and need-for-referral detection.
- The authors position the method as a robust alternative for deep-learning approaches when lesion-level fundus-image labels are expensive.
- An open-source implementation and detection masks are publicly available.