Source-linked AI summary
Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis
Ophir Gozes, Maayan Frid-Adar, Hayit Greenspan, Patrick D. Browning, Huangqi Zhang, Wenbin Ji, Adam Bernheim, Eliot Siegel
TL;DR
Limited datasets and labeling expertise constrained early COVID-19 CT analysis. This study developed deep-learning tools to detect and track disease, achieving 0.996 AUC for coronavirus classification.
Problem
Early coronavirus research had very limited datasets and limited expertise for labeling disease-specific data.
Method
The system analyzes CT cases using distinct 3D subsystems and previously developed algorithms for nodules and focal opacities.
Results
0.996 AUC (95%CI: 0.989-1.00) was achieved for coronavirus versus non-coronavirus thoracic CT studies on Chinese control and infected datasets.
Takeaways & Limitations
The study shows that rapidly developed AI-based CT tools can detect coronavirus and quantify disease burden while tracking progression or resolution.
Abstract
from arXiv · showhide
Purpose: Develop AI-based automated CT image analysis tools for detection, quantification, and tracking of Coronavirus; demonstrate they can differentiate coronavirus patients from non-patients. Materials and Methods: Multiple international datasets, including from Chinese disease-infected areas were included. We present a system that utilizes robust 2D and 3D deep learning models, modifying and adapting existing AI models and combining them with clinical understanding. We conducted multiple retrospective experiments to analyze the performance of the system in the detection of suspected COVID-19 thoracic CT features and to evaluate evolution of the disease in each patient over time using a 3D volume review, generating a Corona score. The study includes a testing set of 157 international patients (China and U.S). Results: Classification results for Coronavirus vs Non-coronavirus cases per thoracic CT studies were 0.996 AUC (95%CI: 0.989-1.00) ; on datasets of Chinese control and infected patients. Possible working point: 98.2% sensitivity, 92.2% specificity. For time analysis of Coronavirus patients, the system output enables quantitative measurements for smaller opacities (volume, diameter) and visualization of the larger opacities in a slice-based heat map or a 3D volume display. Our suggested Corona score measures the progression of disease over time. Conclusion: This initial study, which is currently being expanded to a larger population, demonstrated that rapidly developed AI-based image analysis can achieve high accuracy in detection of Coronavirus as well as quantification and tracking of disease burden.
I. Introduction
The introduction frames COVID-19 CT diagnosis as an urgent challenge because initial RT-PCR sensitivity may be limited and rapidly increasing imaging volumes require efficient evaluation. It motivates rapidly adapting deep-learning CT tools to differentiate coronavirus patients and support detection, measurement, and progression tracking.
- CT disease evolution: 28% of early, 76% of intermediate, and 88% of late-stage patients had bilateral lung involvement on CT.Late-stage disease was defined as 6-12 days, and increasing time from symptom onset was associated with greater disease severity.
- Clinical need: Rapid evaluation became necessary as thoracic CT was considered for diagnosis or screening of potentially very large numbers of imaging studies.The introduction identifies AI as a potential support for radiologist triage, quantification, and trend analysis.
- Development challenge: Limited datasets, limited labeling expertise, and uncertain sample sufficiency constrained conventional deep-learning development for the new disease.The paper notes that datasets were only beginning to be identified and annotated, raising uncertainty about clinically meaningful learning at this stage.
- Study objective: The study aimed to develop automated deep-learning CT tools that differentiate coronavirus from non-coronavirus patients and support disease detection, measurement, and progression tracking.The proposed rapid-development strategy modified and adapted existing AI models while incorporating initial clinical understanding.
II. Methods
The proposed system combines 3D volume analysis for nodules and focal opacities with newly developed 2D slice analysis for larger diffuse opacities, including ground-glass infiltrates. It integrates localization, quantitative measurements, visual explanations, and volumetric Corona scores for case review and patient-specific monitoring.
- System architecture: The system analyzes CT cases through Subsystem A, a 3D volume analysis for nodules and focal opacities, and Subsystem B, a 2D slice analysis for larger diffuse opacities.Subsystem B targets diffuse opacities, including ground-glass infiltrates described as representative of coronavirus.
- Subsystem A: Subsystem A uses existing software to detect small focal opacities and provide localization, segmentation, lesion features, and quantitative measurements.Outputs include volumetric and axial measurements, HU values, calcification detection, and texture characterization.
- Subsystem B: For positive slices, Grad-cam network-activation maps localize regions contributing to the decision and align with diffuse opacities.The system converts slice-level outputs into a positive ratio for case-level COVID-19 decisions.
III. Results
The results section presents experiments demonstrating the performance of the automated analysis.
- Experiments were conducted to demonstrate the performance of the automated analysis.
I. Classification:
The system detected suspected Coronavirus features at both slice and case levels using CT images. Case-level classification achieved high AUC and tunable sensitivity–specificity operating points in Chinese and U.S.-source evaluations.
- Slice-level classification: 0.994 AUC was achieved for slice-level detection, with 94% sensitivity and 98% specificity at threshold 0.5.The evaluation analyzed 270 slices: 150 normal and 120 COVID-19 suspected slices.
- Case-level classification: 0.996 AUC (95%CI: 0.989-1.00) was achieved for Chinese Coronavirus versus non-Coronavirus patients using the positive ratio.The analysis used 56 confirmed COVID-19 patients and 51 Chinese non-Coronavirus patients.
- Case-level classification: 98.2% sensitivity and 92.2% specificity were obtained at a 1.1% positive-ratio threshold, while 96.4% sensitivity and 98% specificity were obtained at 1.9%.The positive ratio was defined as the percent of positive detected slices relative to lung slices.
- Case-level classification: 0.996 AUC (95%CI: 0.989-1.00) was achieved using U.S.-source non-Coronavirus patients, with 98.2% sensitivity and 91.8% specificity at 1% positive ratio.At a 4.3% threshold, sensitivity was 94.6% and specificity was 98%.
II. Evaluation over time:
The evaluation over time tracked lesion volumes and opacities across serial CT scans, while the Corona score quantified disease burden and recovery. Across patients, score trajectories represented relative severity and distinguished different disease courses.
- II. Evaluation over time:: Corona score plots assessed relative coronavirus severity among patients, while Relative Corona Score plots identified different disease courses.The evaluation included tracking patients across multiple time points and reviewing both focal and multiple opacities.
- II. Evaluation over time:: 191.5 cm3 was the Corona score at the first scan, declining to 97.1 cm3 four days later, a 49% reduction in overall opacity burden.The system calculated the Corona score at each time point to quantify disease burden.
- II. Evaluation over time:: 0 was the final Corona score 15 days after the second scan, when no opacities were present, indicating CT-based resolution of disease.Opacities in this patient appeared more frequently in the mid- and upper-lungs than in the lower lobes.
IV. Discussion
This initial exploratory work presents AI software for detecting, characterizing, and tracking COVID-19 on thoracic CT, with high classification accuracy and quantitative disease monitoring. It also describes consistent high-volume screening and progression assessment to support radiologists and earlier detection.
- Discussion: AI-based automated CT image analysis detected Coronavirus-positive patients and quantified disease burden, while specifically enabling tracking of disease progression or resolution.The authors describe this as the first report, to their knowledge, of software specifically developed for COVID-19 detection, characterization, and progression tracking.
- Discussion: 0.996 AUC (95%CI: 0.989-1.00) was achieved for Coronavirus vs Non-coronavirus classification on Chinese control and infected patient datasets.Possible working points were 98.2% sensitivity, 92.2% specificity and 96.4% sensitivity, 98% specificity.
- Discussion: The system produced quantitative opacity measurements, visualized larger opacities through slice-based heat maps or 3D volume displays, and generated a Corona score for progression over time.These outputs support assessment of disease burden and longitudinal change in individual Coronavirus patients.
- Discussion: AI could rapidly and consistently evaluate high volumes of thoracic CTs, exclude negative studies, reduce radiologist workload, and monitor progression or regression more quantitatively.The authors state that this could enable broader screening, earlier detection of positive cases, and more effective identification and containment of early cases.
- Discussion: Combined with an established pulmonary CT detection platform, standard machine learning and AI applications could support screening, early detection, rapid progression assessment, and therapy guidance.The progression-assessment use applies to patients with pulmonary abnormalities associated with the virus.
Figure Legends
The figure legends describe the Chinese and U.S. CT datasets used for COVID-19 analysis and illustrate automated detection, visualization, and longitudinal disease tracking outputs.
- Datasets: 50 abnormal thoracic CT scans from China included slice-level annotations of normal (n=1036) versus abnormal (n=829).The scans were from patients radiologically suspicious for COVID-19 and referred for laboratory testing between January and February 2020.
- Datasets: 56 RT-PCR-confirmed COVID-19 patients had chest CT scans at one or more of 1–5 time points.The scans used slice thicknesses of {1,1.5,5}mm.
- Datasets: 51 normal scans, 30 historical normal scans, 19 diagnostic or screening scans, and 6,150 abnormal-case slices with lung masks supplied comparison and segmentation data.The normal scan groups and abnormal-case slices had varied slice thicknesses, as specified in the legends.
- System outputs: The system figures depict focal-opacity detection and measurement, Coronavirus-positive slice classification with heatmaps, and 3D maps of focal and diffuse opacities.The 3D volume map uses green for focal opacities and red for larger diffuse opacities.
- Longitudinal analysis: Additional figures show COVID-19 detection ROC curves and disease-progression tracking with Corona Score, Relative Corona Score, and multi-time-point monitoring.Day 0 represents 1–4 days after the first signs of the virus.