Source-linked AI summary
COVID-CT-Dataset: A CT Scan Dataset about COVID-19
Xingyi Yang, Xuehai He, Jinyu Zhao, Yichen Zhang, Shanghang Zhang, Pengtao Xie
TL;DR
COVID-19 CT research was constrained by shortages of diagnostic tests and limited public datasets. The paper releases the COVID-CT dataset, evaluates its utility, and develops multi-task and self-supervised diagnosis methods that achieve strong reported performance on original hospital CTs.
Problem
Privacy concerns and information blockade left few large, open-source COVID-19 CT datasets, limiting development of CT-based diagnosis methods.
Method
The paper constructs COVID-CT from published CT images, verifies its training utility, and combines multi-task learning with contrastive self-supervised learning for diagnosis.
Results
F1 0.90, AUC 0.98, and accuracy 0.89 were achieved on original CT images donated by hospitals.
Takeaways & Limitations
COVID-CT was reported as useful for training COVID-19 CT diagnosis models, and a senior radiologist judged the achieved performance clinically useful.
Takeaways & Limitations
The dataset uses paper-extracted images with reduced quality and selected slices, although the authors report these issues do not significantly affect diagnosis decisions.
Abstract
from arXiv · showhide
During the outbreak time of COVID-19, computed tomography (CT) is a useful manner for diagnosing COVID-19 patients. Due to privacy issues, publicly available COVID-19 CT datasets are highly difficult to obtain, which hinders the research and development of AI-powered diagnosis methods of COVID-19 based on CTs. To address this issue, we build an open-sourced dataset -- COVID-CT, which contains 349 COVID-19 CT images from 216 patients and 463 non-COVID-19 CTs. The utility of this dataset is confirmed by a senior radiologist who has been diagnosing and treating COVID-19 patients since the outbreak of this pandemic. We also perform experimental studies which further demonstrate that this dataset is useful for developing AI-based diagnosis models of COVID-19. Using this dataset, we develop diagnosis methods based on multi-task learning and self-supervised learning, that achieve an F1 of 0.90, an AUC of 0.98, and an accuracy of 0.89. According to the senior radiologist, models with such performance are good enough for clinical usage. The data and code are available at https://github.com/UCSD-AI4H/COVID-CT
1. Introduction
CT scans were used as an alternative COVID-19 diagnostic tool during testing shortages, but public CT datasets were scarce. The paper addresses this gap by releasing COVID-CT and developing diagnosis methods supported by experiments and radiologist assessment.
- Motivation: CT scans supported COVID-19 screening and diagnosis during shortages of RT-PCR test kits.A radiologist noted that CT is useful during outbreaks because it can identify viral pneumonia, which was then likely to be caused by SARS-CoV-2.
- Motivation: Privacy concerns limited publicly available COVID-19 CT datasets, hindering development of CT-based diagnosis methods.Radiologists’ workload and uneven training further motivated automated screening methods.
- Dataset: 349 COVID-19 CT images from 216 patients and 397 negative images were collected from published COVID-19 preprints and released openly.The images were manually selected based on clinical findings described in image captions.
- Dataset: The dataset’s paper-extracted images have reduced quality and incomplete slice sequences, but a consulting radiologist judged these concerns unlikely to substantially impair diagnosis.The paper therefore uses these images for training, while reserving original hospital CTs for validation and testing.
- Methods and results: Multi-task learning and contrastive self-supervised learning produced an F1 of 0.90, an AUC of 0.98, and an accuracy of 0.89.The authors report that the senior radiologist considered models with this performance clinically useful.
2. The COVID-CT Dataset
COVID-CT is built by extracting and manually selecting CT images from COVID-19 preprints, with accompanying patient metadata and separate negative, validation, and test sets. The dataset contains 349 positive images from 216 patients, while evaluation uses original hospital-donated CTs.
- Dataset construction: 760 COVID-19 preprints were collected, embedded figures were located using PDF structure information, and CT images were manually separated when figures contained multiple sub-figures.The extracted figures preserved quality-related properties such as resolution and size.
- Positive dataset: 349 COVID-19-positive CT images were obtained from 216 patient cases, with age information for 169 patients and gender information for 137.Among patients with reported gender, the male:female ratio was 86:51.
- Dataset comparison: COVID-CT has more positive images and patients than two compared datasets, while COVID-19 CT Segmentation Dataset has more positive images but fewer patients.The paper notes that more patients increase image diversity because scans from the same patient are highly visually similar.
- Negative dataset: Negative training data contains 463 images from 55 patients, sourced from LUNA, MedPix, PMC, and Radiopaedia.The reported composition is 36 images from LUNA, 195 from MedPix, 202 from PMC, and 30 from Radiopaedia.
- Evaluation sets: The test set contains 173 COVID-19 CT images from 4 patients and 168 non-COVID-19 CT images, all from original hospital-donated or specified clinical datasets.The validation set contains 88 COVID-19 images from 4 patients and 64 non-COVID-19 images.
3. Study I: Is COVID-CT useful for training CT-based diagnosis models of COVID-19?
Study I tests whether paper-extracted, single-slice COVID-CT images can train diagnosis models despite concerns about quality and slice count. Models trained with more COVID-CT images outperform models trained on original CTs in the reported comparison.
- Study design: The study compares three positive training sets: 118 original COVID-Seg CTs, all 349 paper-extracted COVID-CT images, and a random sample of 118 COVID-CT images.Negative training, validation, and test images are held constant across settings.
- Results: Paper-extracted COVID-CT-349 substantially outperforms original COVID-Seg training, demonstrating utility despite lower-quality, single-slice inputs.The authors frame this as evidence that paper-extracted CTs can train COVID-19 diagnosis models.
- Results: Adding more paper-extracted images improves performance: COVID-CT-349 performs much better than the 118-image COVID-CT-118 setting.With equal image counts, COVID-Seg outperforms COVID-CT-118, indicating original CTs are more useful on average for training.
- Conclusion: The experiments conclude that COVID-CT is useful for training CT-based COVID-19 diagnosis models.This conclusion follows the comparisons among original and paper-extracted positive training sets.
4. Study II: Improve the performance to a clinically more useful level
The study improves COVID-19 CT diagnosis using lung and lesion masks, multi-task learning, and contrastive self-supervised learning. These methods address overfitting from limited training data and achieve clinically useful performance.
- Motivation: 467 positive training images make deep learning models prone to overfitting.The study uses COVID-CT and COVID-Seg images as its positive training data.
- Mask-based learning: Lung masks focus models on lung regions containing COVID-19 manifestations rather than irrelevant background.Lesion masks supervise attention toward lesion-containing regions during training.
- Mask-based learning: Multi-task learning jointly predicts COVID-19 class labels and lesion masks by minimizing classification and lesion-segmentation losses.At testing, predictions use only the CT image and lung mask; lesion masks are not required.
- Representation learning: Contrastive self-supervised learning finetunes ImageNet-pretrained representations by predicting whether two CT augmentations originate from the same scan.The representation is subsequently finetuned with CTs and class labels.
- Results: Lung and lesion masks improve diagnosis performance, with both masks outperforming either mask alone on the combined dataset.The experiments report this pattern across comparisons in Table 6.
- Results: 0.90 F1, 0.98 AUC, and 0.89 accuracy are achieved when multi-task mask learning is combined with CSSL pretraining.The senior radiologist confirmed that models with this performance are clinically useful.
5. Related works
Prior work developed deep learning methods for COVID-19 screening and diagnosis from CT scans and chest X-rays. However, publicly available COVID-19 medical-image datasets remained limited, particularly for CT, because of privacy concerns and information blockade.
- Existing methods: Deep learning studies used CNNs, 3D deep CNNs, transfer learning, and segmentation networks for COVID-19 imaging.The reviewed applications include CT classification, infection-region segmentation, and clinical diagnosis assistance.
- Dataset gap: Publicly available COVID-19 medical-image datasets were scarce because of privacy concerns and information blockade.Existing datasets were mainly X-ray collections, while cited CT datasets covered relatively few cases.
6. Conclusions
The paper releases a public COVID-19 CT dataset and develops learning methods that achieve strong diagnostic performance on hospital-donated original CT images. A senior radiologist confirms that the resulting models are clinically useful.
- Dataset: The dataset contains 349 COVID-19 CT images from 216 patients and 463 non-COVID-19 CT images.The non-COVID-19 images are used as negative training examples.
- Performance: The proposed multi-task and contrastive self-supervised learning approach achieves 0.90 F1, 0.98 AUC, and 0.89 accuracy.These results are reported on original CT images donated by hospitals.
- Conclusion: A senior radiologist confirms that models with this performance are clinically useful.