PaperlayerView exact revision on arXiv

Source-linked AI summary

CANet: Cross-disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading

Xiaomeng Li, Xiaowei Hu, Lequan Yu, Lei Zhu, Chi-Wing Fu, Pheng-Ann Heng

arXiv:1911.01376v1eess.IVcs.CV

TL;DR

DR and DME require grading for tailored early treatment, but prior methods often ignore their relationship and costly lesion-location annotations. CANet jointly grades both diseases with disease-specific and disease-dependent attention using image-level supervision, outperforming other methods on Messidor and achieving the best result on IDRiD.

  • Problem

    Prior DR and DME grading methods often ignore the relationship between the diseases, while lesion-location annotations are costly despite their use in grading.

  • Method

    CANet jointly grades DR and DME by combining disease-specific attention with disease-dependent attention to learn individual and cross-disease features from image-level supervision.

  • Results

    CANet outperforms other methods on both diseases in Messidor and achieves the best results on the IDRiD challenge dataset.

  • Takeaways & Limitations

    The study supports jointly modeling DR and DME relationships for image-level grading across two public benchmark datasets.

  • Takeaways & Limitations

    Image-level supervision makes accurate localization of abnormal signs challenging, and lesion annotations could benefit grading but are not available in public datasets with both disease labels.

Abstract

from arXiv · show

Diabetic retinopathy (DR) and diabetic macular edema (DME) are the leading causes of permanent blindness in the working-age population. Automatic grading of DR and DME helps ophthalmologists design tailored treatments to patients, thus is of vital importance in the clinical practice. However, prior works either grade DR or DME, and ignore the correlation between DR and its complication, i.e., DME. Moreover, the location information, e.g., macula and soft hard exhaust annotations, are widely used as a prior for grading. Such annotations are costly to obtain, hence it is desirable to develop automatic grading methods with only image-level supervision. In this paper, we present a novel cross-disease attention network (CANet) to jointly grade DR and DME by exploring the internal relationship between the diseases with only image-level supervision. Our key contributions include the disease-specific attention module to selectively learn useful features for individual diseases, and the disease-dependent attention module to further capture the internal relationship between the two diseases. We integrate these two attention modules in a deep network to produce disease-specific and disease-dependent features, and to maximize the overall performance jointly for grading DR and DME. We evaluate our network on two public benchmark datasets, i.e., ISBI 2018 IDRiD challenge dataset and Messidor dataset. Our method achieves the best result on the ISBI 2018 IDRiD challenge dataset and outperforms other methods on the Messidor dataset. Our code is publicly available at https://github.com/xmengli999/CANet.

I. INTRODUCTION

The paper introduces CANet to jointly grade DR and DME by modeling their internal relationship using image-level supervision. It combines disease-specific and disease-dependent attention to improve grading for both diseases.

  • Motivation: DR and DME grading matters clinically because early severity assessment supports tailored treatment.DR and DME are associated with retinal abnormalities and macular-exudate distance, respectively.
  • Research gap: Prior methods often graded DR and DME independently, ignoring their internal relationship and the fact that DME is a DR complication.Some joint methods treated the task as multilabel classification without modeling the diseases’ implicit relationship.
  • Method: CANet uses disease-specific attention to select useful features for each disease and disease-dependent attention to capture relationships between them.The disease-dependent module learns attention weights in which larger weights indicate higher complication risk.
  • Contributions: CANet jointly models DR and its complication DME, which the authors identify as the first such joint modeling work for fundus images.The method is designed to produce disease-specific and disease-dependent features for both grading tasks.
  • Results: Experiments on IDRiD and Messidor show that CANet outperforms other methods for grading both diseases and achieves the best performance on IDRiD.The reported evaluation uses public benchmark datasets.

II. RELATED WORK

Related work spans handcrafted and deep-learning methods for DR and DME grading, including image-level classification and lesion-location approaches. Existing joint approaches and medical-imaging multitask methods motivate CANet’s explicit modeling of disease relationships.

  • Diabetic Retinopathy Grading: Early DR grading used handcrafted vessel, optic-disc, and lesion-abnormality features with machine-learning classifiers.Examples include SVM, k-nearest neighbor, and Gaussian mixture models.
  • Diabetic Retinopathy Grading: Deep-learning DR methods either use lesion location information or train image-level classifiers to predict severity directly.Attention maps and suspicious high-resolution patches are also used in image-level approaches.
  • Diabetic Macular Edema Grading: DME grading commonly relies on the distance between exudates and the macula, with earlier systems using handcrafted exudate-position features.The cited example detects exudate candidates relative to the macular region before classification.
  • Joint Grading: Medical-imaging multitask methods share information across tasks, but CANet explicitly explores relationships between two diseases rather than only combining classification and regression tasks.The related methods use shared CNN features with task-specific layers or decoders.

III. METHODOLOGY

CANet uses disease-specific attention to isolate features relevant to each disease, then disease-dependent attention to model relationships between DR and DME for joint grading.

  • Cross-disease Attention Network: CANet uses two disease-specific and two disease-dependent attention modules to learn individual-disease and cross-disease features.The network takes shared convolutional features, refines them through both attention types, and predicts DR and DME scores.
  • Cross-disease Attention Network: ResNet50 supplies high-level feature maps that feed the disease-specific attention modules before average pooling and classification.The selected feature maps come from the deepest convolutional layer before ResNet's average-pooling and fully connected layers.
  • Disease-specific Attention Module: Disease-specific attention combines channel-wise and spatial-wise attention to emphasize disease-relevant features and suppress irrelevant ones.Channel attention is generated from pooled spatial information, while spatial attention is learned after pooling across channels and concatenation.
  • Disease-specific Attention Module: The channel-attention weights are broadcast across spatial positions, while spatial-attention weights are broadcast across channels during feature refinement.These broadcast operations enable selective feature enhancement along both channel and spatial dimensions.
  • Disease-specific Attention Module: The attention modules use fully connected and convolutional operations, with ReLU and sigmoid activations and channel reduction through r = 0.5.Table I documents the module structure; the channel-reduction ratio is used to save network parameters.

C. Disease-dependent Attention Module

The disease-dependent attention module transfers informative features from one disease branch to the other, motivated by the reported internal relationship between DR and DME.

  • Disease-dependent Attention Module: The design is motivated by two reported relationships: more exudates may indicate greater macular risk, while exudates closer to the macula may indicate more pathological DR signs.The authors use these observations to motivate cross-disease feature interaction.
  • Disease-dependent Attention Module: The disease-dependent module is intended to capture DR–DME correlation and improve grading performance for both diseases.The paper describes this as the purpose of the cross-branch feature aggregation.
  • Disease-dependent Attention Module: The module uses disease-specific features from both branches to produce disease-dependent features for DR and DME.Average pooling and fully connected operations convert the disease-specific feature maps into branch inputs for cross-disease processing.
  • Disease-dependent Attention Module: For DME grading, attention weights learned from DR features select useful cross-disease information before it is added to DME-specific features.The selected DR features and DME-specific features are combined element-wise to generate refined DME features.

D. Network Architecture

CANet combines disease-specific and disease-dependent feature-learning losses with final DR and DME grading losses in a weighted objective.

  • Loss Design: The network uses separate cross-entropy losses for DR-specific and DME-specific feature learning, alongside losses for final DR and DME grading.The objective therefore supervises both disease-specific representation learning and task outputs.
  • Loss Design: DR grading uses binary cross-entropy on Messidor and 5-class cross-entropy on IDRiD, while DME grading uses 3-class cross-entropy on both datasets.The loss definitions match the dataset-specific grading-label configurations.
  • Classification: DR classification applies a fully connected layer to DR-specific features Gi, with output dimensions 2 for Messidor and 5 for IDRiD.The reported fully connected kernel is 1024 × 2 for Messidor and 1024 × 5 for IDRiD.
  • Loss Design: The loss weight λ controls the balance between disease-specific feature learning and refined features containing disease-dependent information.At λ = 0.0, optimization uses refined DR and DME features; increasing λ emphasizes disease-specific learning.

E. Training and Testing Strategies

Training uses augmented 350 × 350 fundus images cropped to 224 × 224 patches and Adam optimization, while testing uses refined-feature predictions without post-processing or ensembles.

  • Training: Training images are normalized and resized to 350 × 350, then randomly scaled and cropped into 224 × 224 patches.Random horizontal and vertical flips provide additional augmentation.
  • Training: The network is optimized with Adam from an initial learning rate of 0.0003, cosine-annealed per batch, for 1000 epochs with batch size 40.These settings define the reported training schedule.
  • Testing: Inference uses only prediction scores from refined DR and DME features containing disease-dependent information.The class with the maximum prediction value is selected independently for each disease.
  • Testing: Testing applies neither post-processing operations nor model ensemble techniques.The reported predictions therefore come directly from the network's refined-feature outputs.

IV. EXPERIMENTS AND RESULTS

The evaluation uses the Messidor and IDRiD datasets, which provide image-level DR and DME severity annotations for joint grading.

  • Datasets: The experiments evaluate CANet on Messidor and the ISBI 2018 IDRiD challenge dataset.These are described as public datasets with annotations for both DR and DME severity grading.
  • Messidor Dataset: Messidor contains 1,200 posterior-pole fundus images acquired from three ophthalmologic departments.Medical experts provide DR and DME grading annotations for each image.
  • Messidor Dataset: Messidor uses binary DR classification by grouping stages 0–1 as referable and stages 2–3 as non-referable.The binary setting follows prior DR screening work and is evaluated with 10-fold cross-validation.
  • Messidor Dataset: DME severity in Messidor is graded from 0 to 2 using the shortest distance between hard exudates and the macula.Grade 0 indicates no visible hard exudate; grades 1 and 2 correspond to distances above or at most 1 papilla diameter, respectively.
  • IDRiD Dataset: IDRiD provides 516 images, including 413 training and 103 test images, with five-class DR and three-class DME grading labels.Its DME annotation uses the same hard-exudate-to-macula distance criterion.

B. Evaluation Metrics

The study measures joint correctness with Joint Accuracy and evaluates disease-specific performance using standard classification metrics, while comparing CANet against individual and shared-feature baselines.

  • Evaluation Metrics: Joint Accuracy counts an image as correct only when both its DR and DME predictions match the ground truth.The metric is computed as the number of jointly correct instances divided by the total number of images.
  • Evaluation Metrics: Messidor reports accuracy, AUC, precision, recall, and F1-score separately for DR and DME.IDRiD comparisons follow the challenge metric Joint Ac.
  • Compare with Baselines: The Messidor baseline comparison contrasts separate ResNet50 networks for each disease with one shared ResNet50 and two disease-specific classifiers.These are labeled Individual training and Joint training, respectively.
  • Compare with Baselines: 96.3% AUC for DR and 92.4% AUC for DME are achieved by CANet, compared with 94.2% and 90.5% for Joint training.The comparison uses the same ResNet50 backbone and training strategies.
  • Compare with Baselines: 85.1% Joint Ac is obtained by CANet under matched model complexity, versus 82.8% and 82.5% for two complex joint-training baselines.CANet has 29.03 M parameters, while the matched baselines are expanded to a similar parameter count.
  • Attention Ablation: Adding disease-specific attention raises Joint Ac from 82.0% to 84.1% on Messidor.DR accuracy rises from 89.1% to 91.7%, and DME accuracy rises from 90.4% to 91.0%.

3) Analyze the Weight λ in the Loss Function:

The experiments examine loss weighting, backbone choice, and comparisons with multitask and image-level-supervision methods on Messidor.

  • Analyze the Weight λ in the Loss Function: Different λ values produce little variance in the reported results.λ controls the relative emphasis on disease-specific feature supervision versus refined disease-specific and disease-dependent features.
  • Analysis on Architectures: ResNet50 is selected as the backbone after comparison with ResNet34 and DenseNet161 on the Joint training framework.The selection is based on better results in the backbone analysis.
  • Comparisons with Multitask Methods: CANet outperforms the compared multitask learning methods on Joint Accuracy.Compared with, CANet improves DR by 1.4% AUC and 2.3% accuracy, and DME by 1.8% AUC and 0.8% accuracy.
  • Comparisons on the Messidor Dataset: The Messidor comparisons restrict competing methods to those using only image-level supervision.This excludes methods relying on lesion-location or other additional supervision for fair comparison.
  • Comparisons on the Messidor Dataset: CANet exceeds clinical experts by 2.3% and 4.3% on DR AUC and gains 1.5% accuracy and 0.6% AUC over Zoom-in-net.CANet also improves DME accuracy by 2.4% over other reported results.

F. Results on the IDRiD Challenge Leaderboard

On IDRiD, CANet achieves the strongest reported joint grading result using only Sub-challenge 2 image-level data, while attention ablations show benefits from both disease-specific and disease-dependent modules.

  • Results on the IDRiD Challenge Leaderboard: 65.1% Joint Accuracy is achieved by CANet on the IDRiD challenge leaderboard, a relative 2.0% improvement over LzyUNCC.The model is trained using only Sub-challenge 2 data with image-level supervision.
  • Visual Results: Figure 5 presents ground truth and predicted severity scores for DR grades 0–4 and DME grades 0–2.Blue marks the predicted grade and green boxes mark the ground truth.
  • Results on the IDRiD Challenge Leaderboard: The leaderboard comparison is qualified because other entries may use ensembles or supervision from additional IDRiD sub-challenges.CANet uses only Sub-challenge 2 data.
  • Attention Module Analysis: Both disease-dependent attention directions further improve joint grading beyond the disease-specific-only model on IDRiD.The two directions model dependence from DR to DME and from DME to DR.
  • Attention Module Analysis: DME has a stronger influence on DR grading in the IDRiD attention analysis, consistent with the Messidor results.The passage reports this as an observation from the ablation results.

V. DISCUSSION

CANet jointly grades diabetic retinopathy and diabetic macular edema by modeling both disease-specific features and their internal relationship. Experiments show strong performance, while the discussion identifies challenges in lesion localization and scaling the architecture to multiple diseases.

  • Limitations: Image-level supervision makes accurate localization of abnormal signs challenging, although lesion masks or bounding boxes could benefit grading.The discussion notes that no public dataset combines both disease-grading labels and lesion segmentation masks.
  • Limitations: Extending CANet to five correlated diseases would require 20 disease-dependent attention modules, creating high computational cost and design challenges.
  • Future Work: Future work includes better modeling of DR–DME relationships, multi-disease interactions, graph convolutional networks, and joint training with lesion annotations.
  • Method: CANet uses disease-specific and disease-dependent attention modules to jointly model DR, DME, and their internal relationship.The network leverages both feature types simultaneously for joint grading.
  • Results: The method demonstrated superiority on both DR and DME grading tasks on the Messidor dataset.
  • Results: CANet achieved the best results on the IDRiD challenge dataset.
Loading 1911.01376v1…