Source-linked AI summary
Weakly Supervised Deep Learning for COVID-19 Infection Detection and Classification from CT Images
Shaoping Hu, Yuan Gao, Zhangming Niu, Yinghui Jiang, Lao Li, Xianglu Xiao, Minhao Wang, Evandro Fei Fang, Wade Menpes-Smith, Jun Xia, Hui Ye, Guang Yang
TL;DR
Manual labelling is burdensome during outbreaks, while PCR can be time-consuming and have false negatives or low sensitivity. The paper proposes weakly supervised deep learning on multi-centre, multi-scanner chest CT data to automatically detect infected regions and support COVID-19 diagnosis.
Problem
Supervised learning requires considerable manual labelling, while PCR can be time-consuming and have high false-negative rates and low sensitivity.
Method
The study proposes a weakly supervised deep learning framework that automatically detects COVID-19 infected regions from chest CT data acquired across multiple centres and scanners.
Results
Detection results also support diagnosis of COVID-19 patients.
Takeaways & Limitations
The framework reduces reliance on manual CT labelling while providing automated infection-region detection for COVID-19 diagnosis.
Takeaways & Limitations
The retrospective study followed participating-hospital ethics procedures, with further consent waived.
Abstract
from arXiv · showhide
An outbreak of a novel coronavirus disease (i.e., COVID-19) has been recorded in Wuhan, China since late December 2019, which subsequently became pandemic around the world. Although COVID-19 is an acutely treated disease, it can also be fatal with a risk of fatality of 4.03% in China and the highest of 13.04% in Algeria and 12.67% Italy (as of 8th April 2020). The onset of serious illness may result in death as a consequence of substantial alveolar damage and progressive respiratory failure. Although laboratory testing, e.g., using reverse transcription polymerase chain reaction (RT-PCR), is the golden standard for clinical diagnosis, the tests may produce false negatives. Moreover, under the pandemic situation, shortage of RT-PCR testing resources may also delay the following clinical decision and treatment. Under such circumstances, chest CT imaging has become a valuable tool for both diagnosis and prognosis of COVID-19 patients. In this study, we propose a weakly supervised deep learning strategy for detecting and classifying COVID-19 infection from CT images. The proposed method can minimise the requirements of manual labelling of CT images but still be able to obtain accurate infection detection and distinguish COVID-19 from non-COVID-19 cases. Based on the promising results obtained qualitatively and quantitatively, we can envisage a wide deployment of our developed technique in large-scale clinical studies.
INTRODUCTION
COVID-19 CT analysis is motivated by limitations of RT-PCR and the difficulty of distinguishing COVID-19 from other pneumonia on imaging. The study therefore develops weakly supervised deep learning for automatic infection localisation and classification.
- RT-PCR can have high false-negative rates and low sensitivity, hindering early identification and treatment of presumptive COVID-19 patients.
- CT can support early COVID-19 screening and diagnosis, but overlapping imaging characteristics with other pneumonia complicate automated distinction.
- Existing CT methods commonly use supervised learning, requiring extensive manual annotation that clinicians may not have time to provide during an outbreak.
- The proposed weakly supervised framework automatically detects COVID-19 infected regions from multi-centre, multi-scanner CT data.
- The developed networks classify COVID-19 cases against community-acquired pneumonia and non-pneumonia scans.
MATERIALS AND METHODS
The study uses multi-centre CT data and combines preprocessing, lung segmentation, and multi-scale weakly supervised classification. Intermediate convolutional representations are aggregated to address lesion-size variation and support lesion detection.
- Patients and Data: The dataset contains 450 volumetric chest CT exams, comprising 150 COVID-19, CAP, and NP scans, acquired from two hospitals.
- Dataset for Lung Segmentation: A fixed-sized sliding window replaces predefined HU-window clipping to reduce bias across centres and scanners.
- Dataset for Lung Segmentation: A multi-view U-Net with multi-window voting and sequential information attention performs lung segmentation before downstream analysis.
- Detection and Classification Network: The fully convolutional backbone uses five convolutional blocks with stacked 3×3 convolutions, batch normalisation, ReLUs, and max-pooling.
- Multi-Scale Learning: Multi-scale learning feeds Conv3, Conv4, and Conv5 maps into weakly supervised classification layers, aggregates scores, and applies Softmax for final prediction.
- Multi-Scale Learning: Class-balanced weighting and focal modulation increase attention to rare classes and misclassified examples, including mild COVID-19 slices with small lesions.
A. Category-Specific Saliency
The method computes category-specific pixel importance by integrating gradients from a null baseline to the input, then combines multi-scale attributions into a joint saliency map. This map indicates how individual pixels influence the predicted class score.
- Category-Specific Saliency: Category-specific saliency is derived from gradients of the predicted class score with respect to individual input pixels.The gradient represents each pixel’s contribution to the prediction.
- Category-Specific Saliency: A large positive importance score indicates that a pixel strongly increases the prediction score, whereas a score near zero indicates little influence.
- Category-Specific Saliency: Integrated Gradients accumulates gradients along the straight-line path between a baseline image and the current input.The baseline used in this study is a null image filled with zeros.
- Category-Specific Saliency: The integral is approximated by summing gradients over m interpolation steps, with the approximation computed across the input path.
- Category-Specific Saliency: Integrated gradients are computed at Conv3, Conv4, and Conv5, then multiplied pixel-wise to produce a joint saliency map.
B. Bounding Box Extraction
The study extracts lesion bounding boxes from joint saliency maps through smoothing, thresholding, morphology, and connected-component analysis. It evaluates classification across three-way and binary tasks using standard diagnostic metrics and cross-validation.
- Bounding Box Extraction: The joint saliency map is converted into lesion bounding boxes by taking absolute values, applying Gaussian blurring, and thresholding.The threshold is selected with Isodata thresholding.
- Bounding Box Extraction: Morphological dilation and erosion close small foreground holes before connected components above an area threshold are enclosed by minimum rectangular boxes.
- Experiments: The experiments include three-way classification of NP, CAP, and COVID-19, plus binary comparisons between each pair of categories.
- Experiments: The three-way classifiers use Conv3, Conv4, and Conv5 predictions before training a joint classifier on aggregated prediction scores.
- Experiments: Performance is evaluated with accuracy, precision, sensitivity, specificity, and AUC, while lung segmentation uses Dice score.
EXPERIMENTS AND RESULTS
Lung-segmentation performance is examined through an ablation study comparing preprocessing and multi-view-learning post-processing choices on TCIA data. Figure 3 reports Dice scores for these variants.
- Experiments and Results: The lung-segmentation evaluation randomly divides 60 TCIA cases with ground truth into 40 training, 10 validation, and 10 independent testing datasets.
- Experiments and Results: Figure 3 compares Dice scores across no preprocessing, predefined Hounsfield-unit-window normalization, and the proposed fixed-sized sliding-window normalization.
- Experiments and Results: The ablation also compares segmentation without and with multi-view-learning-based post-processing.The post-processing condition is abbreviated as W/O P versus W P.
A. Class Activation Mapping
Multi-scale COVID-19 class activation maps show where infections are located and how lesion scale relates to network feature level. Different convolutional levels capture different lesion patterns.
- Class Activation Mapping: Figure 4 presents COVID-19 class activation maps from Conv3, Conv4, and Conv5 feature levels.
- Class Activation Mapping: Hotter activation-map areas indicate regions more likely to be infected.
- Class Activation Mapping: The multi-scale model captures both large patchy lesions, including crazy paving and consolidation, and small nodule-like lesions, including GGO and bronchovascular thickening.
- Class Activation Mapping: Conv3 and Conv4 detect small, often peripheral and subpleural lesions but do not capture larger patchy lesions effectively.The passage attributes this limitation to the mid-level layers’ limited receptive field.
- Class Activation Mapping: Conv5 detects larger patchy lesions such as crazy paving and consolidation, which are often centrally and peribronchially distributed.
B. Categorical-Specific Saliency
Categorical-specific saliency provides pixel-level lesion localisation and supports interpretation of COVID-19, CAP, and NP predictions. Multi-scale joint classification performs best overall, while COVID-19/CAP remains the most difficult binary distinction.
- Saliency interpretation: Saliency maps provide pixel-level information that delineates lesion extent more precisely than CAMs.Brighter pixels indicate larger contributions to the categorical-specific score.
- Saliency interpretation: Random on-the-fly contrast adjustment helps the model learn contrast-invariant representations and precisely localise lesions.Direct histogram matching could suppress COVID-19 lesions such as GGO and reduce detection performance.
- Class-specific patterns: COVID-19 cases generally show larger, scattered, multi-instance lesions, whereas CAP cases typically show smaller, more locally constrained infection areas.COVID-19 and CAP can share GGO and airspace consolidation features.
- Classification performance: Multi-scale learning with a joint classifier achieves superior overall performance to single-scale three-way classification tasks.Conv4 and Conv5 single-scale features perform similarly and significantly better than Conv3 features.
- Classification performance: 96.2% ACC, 97.3% PRC, 94.5% SEN, 95.3% SPE, and 0.970 AUC are reported for NP/COVID-19 binary classification.Binary classification generally performs better than three-way classification.
- Classification performance: COVID-19/CAP classification is the least superior binary task, likely because both classes share radiographic features and the network may lack capacity for disease-specific representations.The proposed method nevertheless outperforms NTS-NET.
- Classification performance: In three-way classification, NP sensitivity is 91.3%, compared with 87.6% for COVID-19 and 83.0% for CAP.The COVID-19 class has the highest mean AUC at 0.923, while CAP may be confused with COVID-19 because of similar appearance.
DISCUSSIONS
The framework learns lesion detection and localisation from image-level labels while using multi-level representations and saliency fusion to address varied lesion patterns. It also supports infected-lung-area estimation and broad classification, but remains limited in separating COVID-19 from CAP and in handling slice-level training noise.
- DISCUSSIONS: Image-level labels enable the framework to detect and localise COVID-19 and CAP lesions without lesion-level manual annotations.The method is described as weakly supervised and operates from image-level labels only.
- DISCUSSIONS: High-level Conv5 representations capture large patch-like lesions but tend to discard small local lesions.Mid-level Conv4 and Conv5 representations complement this limitation by detecting lesions corresponding to reported peripheral and inferior-lobe distributions.
- DISCUSSIONS: Combining multi-scale saliency maps generated by integrated gradients enables precise localisation of multiinstance lesions.The approach combines saliency information across scales to improve localisation of lesions occurring with different extents and positions.
- DISCUSSIONS: Joint saliency estimates the percentage of infected lung area, a factor clinicians consider when evaluating COVID-19 severity.The same framework also evaluates three-way classification and binary classification for each pair of classes.
- DISCUSSIONS: The network is not discriminative enough to separate CAP from COVID-19.The authors suspect limited backbone CNN capacity contributes to this limitation and suggest increasing feature channels or using more advanced architectures.
- DISCUSSIONS: Training on individual slices can introduce noise because CAP or COVID-19 scans may contain non-infection slices between relevant slices.The authors propose attention-based multiple-instance learning over patient-specific slice bags as a future remedy.
CONCLUSION
The study presents a weakly supervised framework for fully automated COVID-19 detection and classification using CT images from multiple scanners and centres. The framework distinguishes COVID-19 from CAP and NP, localises lesions, and achieves strong reported classification metrics with promising visualisations.
- CONCLUSION: The framework performs fast, fully automated COVID-19 detection and classification on retrospectively extracted CT images from multiple scanners and centres.It is designed to distinguish COVID-19 cases from CAP and NP patients while pinpointing infected regions.
- CONCLUSION: The proposed model achieves high accuracy, precision and AUC for classification, alongside promising qualitative visualisations of lesion detections.The authors state that these findings support envisaging large-scale deployment of the developed framework.