Source-linked AI summary
Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection
Zhe Wang, Yanxin Yin, Jianping Shi, Wei Fang, Hongsheng Li, Xiaogang Wang
TL;DR
Diabetic retinopathy detection needs scalable methods that localize lesions without expensive location annotations. Zoom-in-Net uses image-level supervision to classify whole retinal images and attention-selected high-resolution patches, achieving state-of-the-art performance on EyePACS and Messidor while providing lesion localization.
Problem
Expensive lesion-location annotations limit dataset size, motivating simultaneous diabetic-retinopathy classification and localization from weak image-level supervision.
Method
Zoom-in-Net uses a CNN with global classification, gated attention maps, and high-resolution suspicious patches to mimic clinicians’ zoom-in examination process.
Results
Zoom-in-Net achieves state-of-the-art performance on EyePACS and Messidor, with attention localization reaching a recall of 0.82.
Takeaways & Limitations
Image-level supervision can support both diabetic-retinopathy classification and useful localization of suspicious regions and potential lesions.
Abstract
from arXiv · showhide
We propose a convolution neural network based algorithm for simultaneously diagnosing diabetic retinopathy and highlighting suspicious regions. Our contributions are two folds: 1) a network termed Zoom-in-Net which mimics the zoom-in process of a clinician to examine the retinal images. Trained with only image-level supervisions, Zoomin-Net can generate attention maps which highlight suspicious regions, and predicts the disease level accurately based on both the whole image and its high resolution suspicious patches. 2) Only four bounding boxes generated from the automatically learned attention maps are enough to cover 80% of the lesions labeled by an experienced ophthalmologist, which shows good localization ability of the attention maps. By clustering features at high response locations on the attention maps, we discover meaningful clusters which contain potential lesions in diabetic retinopathy. Experiments show that our algorithm outperform the state-of-the-art methods on two datasets, EyePACS and Messidor.
1 Introduction
Zoom-in-Net addresses the need for weakly supervised simultaneous diabetic retinopathy classification and lesion localization, using image-level labels to produce clinically interpretable attention maps. It improves detection while identifying suspicious regions and potential lesions on EyePACS and Messidor.
- Motivation: Strong lesion-location annotations are expensive, limiting medical-imaging datasets and motivating weakly supervised classification with localization.The paper frames intuitive localization as useful for physicians and patients while seeking to use larger image-level-labeled datasets.
- Contribution: Zoom-in-Net is a CNN-based weakly supervised framework that learns attention maps from image-level supervision for simultaneous classification and localization.Its attention mechanism highlights pixels that contribute more to the image-level decision.
- Contribution: The framework mimics clinicians by skimming retinal images for suspicious regions and zooming in on high-resolution patches before making a decision.Predictions use both the whole image and suspicious local patches.
- Results: Zoom-in-Net outperforms state-of-the-art methods and general physicians on EyePACS and Messidor datasets.The paper evaluates the framework on both datasets to validate classification effectiveness.
- Results: The attention localization reaches a recall of 0.82 on images labeled by an experienced ophthalmologist, while high-response clusters contain potential diabetic-retinopathy lesions.Reported clusters include microaneurysms, blot/frame haemorrhages, and hard/soft exudates.
2 Architecture of Zoom-in-Net
Zoom-in-Net combines global image classification with attention-guided analysis of high-resolution suspicious patches. M-Net produces image features, A-Net gates disease-level attention, and C-Net uses selected patches to refine predictions.
- Architecture: Zoom-in-Net contains M-Net for image classification, A-Net for attention maps, and C-Net for high-resolution suspicious patches.The architecture targets five disease levels from no diabetic retinopathy through proliferative diabetic retinopathy.
- Main Network (M-Net): M-Net processes an image with an Inception-ResNet CNN and produces disease-level probabilities from intermediate feature maps.Its intermediate feature maps are also supplied to A-Net.
- Main Network (M-Net): Features from both eyes can be concatenated end-to-end because more than 95% of eye pairs differ in scores by at most 1.This exploits the correlation between left- and right-eye assessments.
- Attention Network (A-Net): A-Net generates separate disease-level attention maps by combining score maps with spatial-softmax attention gates.Spatial softmax makes attention values compete and concentrate on informative regions.
- Crop Network (C-Net): The model selects high-response regions greedily, masks previously selected areas, and crops corresponding high-resolution patches for C-Net.C-Net max-pools patch features, concatenates the result with M-Net features, and classifies the combined representation.
3 Attention Localization Evaluation and Understanding
Zoom-in-Net’s attention maps localize diabetic-retinopathy lesions using only image-level supervision, while feature clustering reveals visually meaningful lesion groups.
- Localization Evaluation: 306 lesions across 182 EyePACS images were annotated with ophthalmologist-drawn bounding boxes for localization evaluation.
- Localization Evaluation: 0.76 box recall and 0.83 person recall were achieved at an IoM threshold of 0.3 using the four sampled boxes.Person recall counts an image as correct when at least one lesion box is retrieved, so it exceeds box recall.
- Attention Visualization: The four selected regions are derived from gated attention maps at levels 1–4, while the level 0 map is ignored because it contains no information.
- Attention Visualization: Clustering feature-map locations with high attention responses produced groups corresponding to microaneurysms, hemorrhages, and hard or soft exudates.The clustering method does not require a predefined number of clusters.
4 Quantitative Evaluation
Zoom-in-Net was evaluated on EyePACS and Messidor under dataset-specific protocols, achieving strong diabetic-retinopathy classification performance and competitive results against established methods and experts.
- Datasets and Protocols: EyePACS contains 35k/11k/43k train/validation/test images with disease grades from 0 to 4, evaluated using quadratic weighted kappa.
- EyePACS Evaluation: 0.832/0.825 validation/test kappa was achieved by M-Net alone, while adding A-Net improved both scores by 0.5%.The A-Net adds attention maps but no additional information for classification at this stage.
- EyePACS Evaluation: 0.857/0.849 kappa was achieved by the complete Zoom-in-Net, compared with 0.86/0.849 for the first-ranked Min-pooling entry.An ensemble of three models reached 0.865/0.854.
- Messidor Evaluation: Messidor evaluation used binary referable/nonreferable and normal/abnormal tasks because its grades range from 0 to 3, unlike EyePACS grades 0 to 4.
- Messidor Evaluation: Zoom-in-Net achieved the highest reported AUC for both Messidor binary tasks, with sensitivities of 0.978 and 0.960 at specificity 0.5.The two sensitivities correspond respectively to normal and referral classification.
5 Conclusions
Zoom-in-Net achieves state-of-the-art performance on two datasets while learning from image-level supervision. It generates attention maps that highlight suspicious regions, whose high-response areas can support classification.
- Zoom-in-Net achieves state-of-the-art performance on two datasets.
- Image-level supervision enables Zoom-in-Net to generate attention maps highlighting suspicious regions.
- High-response regions in gated attention maps correspond to potential diabetic retinopathy lesions and can further boost classification performance.