Source-linked AI summary
Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs
Yuxing Tang, Xiaosong Wang, Adam P. Harrison, Le Lu, Jing Xiao, Ronald M. Summers
TL;DR
The paper addresses joint thoracic-disease classification and weakly supervised localization when only image-level labels and limited severity information are available. It proposes AGCL, which orders samples by mined severity and iteratively uses confident seed heatmaps for attention-guided refinement. On ChestXray14, AGCL improves classification and localization over the benchmark, with reported gains in AUC, recall, and precision.
Problem
Joint thoracic-disease classification and weakly supervised localization must operate with image-level labels and disease-severity information available for only a subset of images.
Method
AGCL mines severity attributes to order samples from severe to mild and iteratively regresses confident seed heatmaps alongside multi-label classification.
Results
AGCL achieves the best localization results, with precision 0.44 and recall 73% while proposing an average of 3.5 boxes per image.
Takeaways & Limitations
The experiments validate performance improvements from the overall framework and its curriculum-learning and attention-guided components on ChestXray14.
Abstract
from arXiv · showhide
In this work, we exploit the task of joint classification and weakly supervised localization of thoracic diseases from chest radiographs, with only image-level disease labels coupled with disease severity-level (DSL) information of a subset. A convolutional neural network (CNN) based attention-guided curriculum learning (AGCL) framework is presented, which leverages the severity-level attributes mined from radiology reports. Images in order of difficulty (grouped by different severity-levels) are fed to CNN to boost the learning gradually. In addition, highly confident samples (measured by classification probabilities) and their corresponding class-conditional heatmaps (generated by the CNN) are extracted and further fed into the AGCL framework to guide the learning of more distinctive convolutional features in the next iteration. A two-path network architecture is designed to regress the heatmaps from selected seed samples in addition to the original classification task. The joint learning scheme can improve the classification and localization performance along with more seed samples for the next iteration. We demonstrate the effectiveness of this iterative refinement framework via extensive experimental evaluations on the publicly available ChestXray14 dataset. AGCL achieves over 5.7\% (averaged over 14 diseases) increase in classification AUC and 7%/11% increases in Recall/Precision for the localization task compared to the state of the art.
1 Introduction
Chest radiograph interpretation is valuable but difficult, motivating computer-aided methods that can classify and localize multiple thoracic diseases using weak supervision and disease-severity information.
- Chest X-rays are fast, painless, cost-effective, and minimally irradiating while displaying diverse diagnostic information.
- Interpreting varied thoracic abnormalities remains challenging even for human observers, creating demand for computer-aided detection methods.
- ChestX-ray14 contains 112,120 frontal-view images with 14 thoracic disease labels mined from radiology reports.
- Disease severity levels correlate with different visual appearances and can enrich NLP-mined disease labels.
- AGCL jointly performs thoracic disease classification and weakly supervised localization using image-level labels and severity information for only a subset, without bounding-box training.
- The framework progressively trains on severe, moderate, and mild samples while iteratively using confident-image heatmaps to guide learning.
- Experiments on ChestXray14 report improved classification and localization accuracy from the framework and its attention-guided seed-learning process.
2 Method
AGCL combines severity-based curriculum learning with iterative heatmap guidance, using selected confident seeds to improve disease-specific convolutional features for classification and localization.
- The baseline initializes an ImageNet-pretrained CNN and fine-tunes it across all 14 ChestX-ray14 disease categories for multi-label classification and localization.
- Disease Severity-Level Based Curriculum Learning: Curriculum learning feeds easier concepts or samples before harder ones, using prior information to control training data order.
- Disease Severity-Level Based Curriculum Learning: Radiology-report severity descriptions are mined with dependency graphs and grouped into mild, moderate, and severe clusters to build the curriculum.
- Disease Severity-Level Based Curriculum Learning: Class-specific CAM heatmaps combine fully connected-layer weights with last-convolutional-layer activations at spatial coordinates.
- Attention Guided Iterative Refinement: Separate binary disease models generate more discriminative and spatially accurate disease-specific heatmaps because severity levels can differ across diseases.
- Attention Guided Iterative Refinement: Seeds include severe or moderate samples and correctly classified images whose disease-specific probability exceeds a threshold.
- Attention Guided Iterative Refinement: An added heatmap-regression path shares convolutional blocks and matches current heatmaps to prior seed attention maps using channel-wise smooth L1 loss.
- Attention Guided Iterative Refinement: The objective combines sigmoid cross-entropy classification loss with weighted heatmap-regression loss, and retraining repeatedly harvests additional confident seeds.
3 Experiments
Experiments on ChestXray14 evaluate AGCL for multi-label disease classification and weakly supervised localization. The framework improves classification through iterative attention-guided refinement and achieves the strongest reported localization precision and recall.
- Experimental setup: The ChestXray14 evaluation uses 112,120 frontal-view X-rays from 30,805 patients, with 14 disease labels and 984 radiologist-annotated bounding boxes from 880 images.Images are resized from 1024 × 1024 to 512 × 512 pixels, and ResNet-50 is used as the CNN backbone.
- Disease classification: Classification is assessed with per-disease AUC, comparing the benchmark, baseline, attention-guided learning, and AGCL after one or two refinement steps.AGCL-2 represents two refinement steps; higher AUC indicates a better classifier.
- Disease classification: AGCL-2 outperforms AGCL-1, while AGCL-3 produces similar results to AGCL-2, suggesting that iterative refinement converges after two steps.Attention-guided learning also consistently outperforms the baseline, and AGCL further improves upon it using curriculum learning.
- Weakly supervised localization: Localization evaluates heatmap-derived bounding boxes against radiologist ground truth using IoBB with threshold T(IoBB) = 0.25.The comparison includes the baseline, AGL, and AGCL-2, denoted AGCL in the localization table.
- Weakly supervised localization: AGCL generates 3,498 boxes, including 1,518 true positives, achieving the highest precision of 0.44 and recalling 73% of ground-truth boxes with 3.5 boxes per image.Its gains over AGL are larger for Effusion, Mass, and Infiltration, whereas curriculum learning provides little benefit for predominantly small Nodule cases.
4 Conclusion
The paper uses NLP-mined disease severity information to guide curriculum learning and combines it with iterative attention-guided refinement. Experiments on ChestXray14 validate improvements in thoracic disease classification and weakly supervised localization.
- Conclusion: NLP-mined disease severity levels facilitate curriculum learning for more accurate thoracic disease classification and localization.The severity information is extracted from radiology reports and used alongside iterative attention-guided refinement.
- Future work: Future work targets structured reports, richer lesion information, follow-up studies, and common disease patterns for more precise predictive models.The proposed extensions include coarse lesion locations and additional information mined from reports.