Source-linked AI summary
Deep-COVID: Predicting COVID-19 From Chest X-Ray Images Using Deep Transfer Learning
Shervin Minaee, Rahele Kafieh, Milan Sonka, Shakib Yazdani, Ghazaleh Jamalipour Soufi
TL;DR
The paper tackles limited evidence for automated COVID-19 detection from chest X-rays, an important early-diagnosis problem. It builds a radiologist-labeled dataset and fine-tunes pretrained convolutional networks, achieving high sensitivity with roughly 90% specificity. The authors regard the findings as encouraging but preliminary because larger COVID-19 image sets are needed.
Problem
Limited public COVID-19 X-ray images constrained detailed studies of automated COVID-19 detection, despite the importance of early diagnosis.
Method
The study creates a radiologist-labeled COVID-Xray-5k dataset and fine-tunes four pretrained convolutional networks for COVID-19 detection.
Results
98% sensitivity and approximately 90% specificity were reported for COVID-19 detection, with the best model reaching 92% specificity.
Takeaways & Limitations
The dataset and models provide a publicly available benchmark for future COVID-19 chest X-ray research and comparisons.
Takeaways & Limitations
The results are preliminary because the study uses a limited number of reliably labeled COVID-19 images and only 100 COVID-19 test images.
Abstract
from arXiv · showhide
The COVID-19 pandemic is causing a major outbreak in more than 150 countries around the world, having a severe impact on the health and life of many people globally. One of the crucial step in fighting COVID-19 is the ability to detect the infected patients early enough, and put them under special care. Detecting this disease from radiography and radiology images is perhaps one of the fastest ways to diagnose the patients. Some of the early studies showed specific abnormalities in the chest radiograms of patients infected with COVID-19. Inspired by earlier works, we study the application of deep learning models to detect COVID-19 patients from their chest radiography images. We first prepare a dataset of 5,000 Chest X-rays from the publicly available datasets. Images exhibiting COVID-19 disease presence were identified by board-certified radiologist. Transfer learning on a subset of 2,000 radiograms was used to train four popular convolutional neural networks, including ResNet18, ResNet50, SqueezeNet, and DenseNet-121, to identify COVID-19 disease in the analyzed chest X-ray images. We evaluated these models on the remaining 3,000 images, and most of these networks achieved a sensitivity rate of 98% ($\pm$ 3%), while having a specificity rate of around 90%. Besides sensitivity and specificity rates, we also present the receiver operating characteristic (ROC) curve, precision-recall curve, average prediction, and confusion matrix of each model. We also used a technique to generate heatmaps of lung regions potentially infected by COVID-19 and show that the generated heatmaps contain most of the infected areas annotated by our board certified radiologist. While the achieved performance is very encouraging, further analysis is required on a larger set of COVID-19 images, to have a more reliable estimation of accuracy rates. The dataset, model implementations (in PyTorch), and evaluations, are all made publicly available for research community at https://github.com/shervinmin/DeepCovid.git
1. Introduction
The paper addresses early COVID-19 detection from chest X-rays by assembling labeled data and applying deep transfer learning. It reports encouraging model performance while emphasizing that larger datasets are needed for reliable conclusions.
- Early COVID-19 diagnosis matters because it can support immediate isolation, while RT-PCR throat-swab positivity has been reported at 30–60%.
- Limited public COVID-19 X-ray data motivated creating a radiologist-reviewed dataset combined with non-COVID images from ChexPert.
- Four convolutional networks were trained using data augmentation and ImageNet pretraining rather than training from scratch.
- 98% sensitivity and 92% specificity were achieved by the best-performing model on a 3,000-image test set.
- The study analyzes sensitivity, specificity, ROC, AUC, precision-recall behavior, predicted scores, and confusion matrices across models.
- The authors provide infected-region heatmaps and publicly release the dataset, trained models, and implementation.
- The results remain preliminary because the labeled COVID-19 data are limited, requiring further experiments on larger datasets.
2. COVID-Xray-5k Dataset
COVID-Xray-5k combines radiologist-reviewed COVID-19 images with diverse non-COVID chest radiographs from ChexPert. The dataset includes varied disease categories, image resolutions, and publicly documented category counts.
- COVID-Xray-5k contains 2,084 training images and 3,100 test images assembled from two chest-radiograph datasets.
- The COVID-19 images were selected from anterior-posterior radiographs and filtered by a board-certified radiologist for clear disease signs.
- Non-COVID samples came from ChexPert’s no-finding category and 13 additional disease subcategories.
- The test set contains 1,700 no-finding images and approximately 100 images from each remaining ChexPert subclass.
- Figure 2 illustrates four COVID-19 images, four normal images, and eight images representing other ChexPert diseases.
- The dataset spans resolutions from below 400x400 to above 1900x1400, creating variation in image quality and acquisition.
3. The Proposed Framework
The framework uses transfer learning to adapt pretrained convolutional networks for COVID-19 detection from limited chest X-ray data. It evaluates several architectures and trains them with cross-entropy loss.
- Transfer learning: Transfer learning fine-tunes pretrained models to address the limited size of the COVID-19 image dataset.The paper describes both feature extraction and fine-tuning approaches, using pretrained weights as initial values for the new task.
- Model architectures: The evaluated architectures are ResNet18, ResNet50, SqueezeNet, and DenseNet-121.ResNet50 differs from ResNet18 mainly by having more layers.
- Model architectures: ResNet uses identity shortcut connections to provide direct paths for gradient updates to earlier layers.This design supports more efficient training by easing gradient flow.
- Model architectures: SqueezeNet alternates a 1x1 squeeze layer with parallel 1x1 and 3x3 expand convolutions.Its design also replaces some 3x3 filters with 1x1 filters and delays down-sampling.
- Model architectures: DenseNet connects each layer to all preceding layers, enabling thinner networks with fewer channels and improved computational and memory efficiency.Each layer receives feature maps from earlier layers and passes its own feature maps to subsequent layers.
- Model training: Cross-entropy loss minimizes the distance between predicted probability scores and ground-truth probabilities.The models minimize this loss using stochastic gradient descent and its variations; added regularization did not improve performance.
4. Experimental Results
The experiments evaluate four CNNs on imbalanced COVID-19 chest X-ray data using threshold-based metrics and complementary curves. The models distinguish COVID-19 from non-COVID images, with strong reported sensitivity and specificity but limited reliability because few COVID-19 test images were available.
- Predicted scores: COVID-19 images received much higher predicted probabilities than non-COVID images, while non-COVID images with other diseases received slightly higher scores than normal non-COVID cases.The authors report that SqueezeNet separated COVID-19 and non-COVID scores more strongly than the other models.
- Threshold-independent comparison: Sensitivity and specificity vary with the probability cut-off threshold, so the study also compares models across thresholds using precision-recall and ROC curves.The ROC analysis reports similar AUC performance across models, with SqueezeNet slightly higher, while precision-recall is emphasized for the imbalanced test set.
- Reliability and uncertainty: The reported rates may be unreliable because only 100 COVID-19 images were available in the test set, motivating larger COVID-19 test samples and confidence-interval estimation.The study uses a 95% confidence interval; specificity intervals are around 1%, whereas sensitivity intervals are around 2.7%.
5. Conclusion
The study presents a radiologist-confirmed COVID-Xray-5k dataset and a deep learning framework using fine-tuned pretrained convolutional models for COVID-19 detection. On the test set, the models achieved high sensitivity and approximately 90% specificity, while larger cleanly labeled datasets are still needed for reliable accuracy estimates.
- 5. Conclusion: Around 5,000 chest X-ray images formed COVID-Xray-5k, with labels confirmed by a board-certified radiologist.The dataset combined images from two datasets and was made publicly available as a research benchmark.
- 5. Conclusion: Four pretrained convolutional models were fine-tuned on the training set for COVID-19 detection.The models were ResNet18, ResNet50, SqueezeNet, and DenseNet-121.
- 5. Conclusion: The dataset is publicly available for training and evaluating future machine learning models for COVID-19 binary classification.The benchmark supports subsequent model evaluation on the stated task.
- 5. Conclusion: 98% sensitivity corresponded to approximately 90% specificity on average across the evaluated models.Performance was assessed on the COVID-Xray-5k test set using sensitivity, specificity, ROC, and AUC.
- 5. Conclusion: Further experiments on larger sets of cleanly labeled COVID-19 images are needed for more reliable accuracy estimates.The study used around 200 COVID-19 images alongside 5,000 non-COVID images.