Source-linked AI summary
Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia
Xi Ouyang, Jiayu Huo, Liming Xia, Fei Shan, Jun Liu, Zhanhao Mo, Fuhua Yan, Zhongxiang Ding, Qi Yang, Bin Song, Feng Shi, Huan Yuan, Ying Wei, Xiaohuan Cao, Yaozong Gao, Dijia Wu, Qian Wang, Dinggang Shen
TL;DR
Early COVID-19 diagnosis from CAP is important, but RT-PCR can be slow or unavailable during outbreaks, motivating rapid chest-CT-based assistance. The paper proposes a dual-sampling attention network with online infection-region guidance and reports AUC 0.944 on an independent test set, while noting limitations in longitudinal validation and clinical interpretation of attention maps.
Problem
The paper addresses the need for rapid automatic COVID-19 diagnosis from CAP using chest CT when RT-PCR results may be delayed or testing capacity limited.
Method
The method uses lung-masked 3D CNNs with online attention refined by segmented infection regions, dual sampling for infection-size imbalance, and ensemble prediction integration.
Results
AUC 0.944 was achieved on the independent testing dataset, compared with 0.934 for RN34 + US.
Takeaways & Limitations
The proposed algorithm could potentially aid radiologists in diagnosing COVID-19 from CAP, especially during the early outbreak stage.
Takeaways & Limitations
The model still requires testing on longitudinal data and further analysis linking attention localizations to specific clinical imaging signs.
Abstract
from arXiv · showhide
The coronavirus disease (COVID-19) is rapidly spreading all over the world, and has infected more than 1,436,000 people in more than 200 countries and territories as of April 9, 2020. Detecting COVID-19 at early stage is essential to deliver proper healthcare to the patients and also to protect the uninfected population. To this end, we develop a dual-sampling attention network to automatically diagnose COVID- 19 from the community acquired pneumonia (CAP) in chest computed tomography (CT). In particular, we propose a novel online attention module with a 3D convolutional network (CNN) to focus on the infection regions in lungs when making decisions of diagnoses. Note that there exists imbalanced distribution of the sizes of the infection regions between COVID-19 and CAP, partially due to fast progress of COVID-19 after symptom onset. Therefore, we develop a dual-sampling strategy to mitigate the imbalanced learning. Our method is evaluated (to our best knowledge) upon the largest multi-center CT data for COVID-19 from 8 hospitals. In the training-validation stage, we collect 2186 CT scans from 1588 patients for a 5-fold cross-validation. In the testing stage, we employ another independent large-scale testing dataset including 2796 CT scans from 2057 patients. Results show that our algorithm can identify the COVID-19 images with the area under the receiver operating characteristic curve (AUC) value of 0.944, accuracy of 87.5%, sensitivity of 86.9%, specificity of 90.1%, and F1-score of 82.0%. With this performance, the proposed algorithm could potentially aid radiologists with COVID-19 diagnosis from CAP, especially in the early stage of the COVID-19 outbreak.
I. INTRODUCTION
The paper addresses early COVID-19 detection from CAP using chest CT, proposing an attention-guided, dual-sampling network evaluated on large multi-center datasets.
- Motivation: The study targets automatic COVID-19 diagnosis from CAP in chest CT to support early detection and reduce radiologists’ screening workload.RT-PCR may have high false-negative rates or limited availability, while chest CT can provide rapid diagnostic information.
- Motivation: COVID-19 cases tend to have more severe infection regions than CAP cases, creating an imbalanced infection-size distribution.The infection-size measure is the volume ratio of segmented infection regions to whole-lung volume.
- Contributions: The proposed method combines lung masking, online attention refinement, and dual sampling to classify COVID-19 and CAP.The online attention mechanism uses segmented infection regions to guide localization, while dual sampling addresses infection-size imbalance.
- Evaluation: The evaluation uses 2186 CT scans from 1588 patients for 5-fold cross-validation and an independent test set of 2796 scans from 2057 patients.The data are described as multi-center CT datasets collected across eight hospitals.
II. RELATED WORKS
Prior work applies deep learning to pneumonia and COVID-19 diagnosis from chest imaging, but existing CT methods were evaluated on relatively small datasets.
- Computer-Assisted Pneumonia Diagnosis: Chest X-ray CNNs and object-detection methods have been used for pneumonia diagnosis and lesion localization.Examples include RetinaNet and Mask R-CNN applied to the RSNA pneumonia detection challenge dataset.
- COVID-19 Diagnosis: Existing COVID-19 CT methods use 2D CNNs, V-Net with ResNet18, or slice-based ResNet50 processing to distinguish pneumonia categories.These approaches rely on manually delineated regions, segmented infection regions, or all CT slices, respectively.
- COVID-19 Diagnosis: The cited COVID-19 CT methods were evaluated on small datasets, whereas this paper collected 4982 CT scans from 3645 patients across eight collaborative hospitals.The authors present this as the largest multi-center COVID-19 CT dataset at the time.
B. Class Re-sampling Strategies
The method combines lung-focused preprocessing, attention-guided 3D networks, and sampling strategies designed to address imbalanced infection-region sizes.
- Class Re-sampling Strategies: Size-balanced sampling repeats COVID-19 cases with small infections and CAP cases with large infections within training mini-batches.The strategy aims to balance infection-region sizes without discarding scarce COVID-19 data.
- Lung Preprocessing: Lung masking removes non-lung context before the 3D CT images enter the network.VB-Net segments lung fields and infection regions for preprocessing and attention supervision.
- Class Re-sampling Strategies: Uniform sampling feeds each training sample once per epoch with equal probability, preserving the dataset’s intrinsic distribution.This branch is implemented as the “Attention RN34 + US” network.
- Training Pipeline: The framework trains two 3D ResNet34 models with different sampling strategies and integrates their predictions through ensemble learning.The two branches are Attention RN34 + US and Attention RN34 + SS.
- Online Attention: The online attention module converts fully connected weights into 1 × 1 × 1 convolution weights and applies ReLU to generate trainable attention maps.The maps are refined using segmented infection regions during training.
A. Network
The network uses a 3D ResNet34 backbone and an online 3D class activation mapping mechanism to generate attention maps for CT-based diagnosis.
- A. Network: The backbone is a 3D ResNet34 whose final residual block preserves feature-map resolution at 1/16 of the input in each dimension.This higher resolution improves the quality of generated attention maps before global average pooling.
- A. Network: Online 3D class activation mapping converts convolutional feature maps into trainable attention maps for classification.The fully connected weights are reused as a 1 × 1 × 1 convolution kernel followed by ReLU.
- A. Network: The attention feature map is upsampled, normalized to [0,1], and passed through a sigmoid to create a soft mask.The resulting mask is used to focus the model on relevant image regions.
- A. Network: The 1 × 1 × 1 convolution parameters are copied from the fully connected layer and updated only by the classification BCE loss.The sigmoid parameters are set to α=100 and β=0.4.
C. Size-balanced Sampling
Size-balanced sampling addresses unequal infection-size distributions by increasing exposure to small-infection COVID-19 cases and large-infection CAP cases during training.
- C. Size-balanced Sampling: Size-balanced sampling repeats small-infection COVID-19 and large-infection CAP cases within mini-batches.The strategy targets groups that are relatively underrepresented because infection-size distributions are imbalanced.
- C. Size-balanced Sampling: Training data are divided into four groups by disease label and pneumonia-to-lung volume ratio.The groups are small- and large-infection COVID-19 and small- and large-infection CAP.
- C. Size-balanced Sampling: The two targeted minority groups receive weights higher than 1, approximately 1.5 in each training fold.The weights determine each group’s sampling probability after normalization by the sum of all weights.
- C. Size-balanced Sampling: Each mini-batch first selects a group according to refined probabilities, then uniformly samples an example from that group.This increases sampling opportunities for small-infection COVID-19 and large-infection CAP cases in the Attention RN34 + SS model.
D. Objective Function
The objective combines classification and attention losses for COVID-19 cases, while CAP cases use classification loss alone.
- D. Objective Function: The training objective is composed of classification loss Lc and attention loss Lex for the attention-based models.The classification term uses binary cross entropy for COVID-19/CAP prediction.
- D. Objective Function: The attention loss uses pneumonia infection masks to refine the model’s attention maps for COVID-19 cases.Its adaptive normalization changes the loss according to infection-region size.
- D. Objective Function: CAP cases are trained using only the classification loss Lc.The additional attention loss is specified for COVID-19 cases.
E. Ensemble Learning
The dual-sampling model combines predictions from uniform- and size-balanced-sampling branches using infection-size-dependent weights.
- E. Ensemble Learning: Dual sampling combines uniform sampling and size-balanced sampling through an ensemble learning layer.The layer integrates the two branch predictions into a final diagnosis result.
- E. Ensemble Learning: The final prediction combines branch scores with weights selected according to the pneumonia-to-lung infection ratio.The weight factor is set to 0.35 for ratios below 0.001 or above 0.030, and 0.96 otherwise.
- E. Ensemble Learning: Extra weight is assigned to the size-balanced branch for COVID-19 with small infections and CAP with large infections.The uniform-sampling branch receives more weight for the remaining cases.
IV. EXPERIMENTAL RESULTS
The study evaluates COVID-19 diagnosis using a large multi-center CT dataset, with separate training-validation and testing data and standardized CT preprocessing. The datasets include demographic information and model-comparison results across TV and test sets.
- Dataset: 4,982 chest CT images from 3,645 patients comprise the study data, including 3,389 COVID-19 and 1,593 CAP images.The data were collected from multiple hospitals and include thin-slice CT scans.
- Dataset: Table I reports TV and test-set demographics, with age presented as median values and ranges.The table distinguishes the training-validation dataset from the independent testing dataset.
- Image preprocessing: CT preprocessing includes resampling images and masks to a common voxel spacing before computational down-sampling.The reported normalized spacing is 0.7168mm, 0.7168mm, and 1.25mm for the x, y, and z axes.
- Evaluation: Table II compares RN34 with uniform, size-balanced, and dual-sampling strategies using AUC, accuracy, sensitivity, specificity, and F1-score.TV results combine five validation sets, while test results report mean±STD across five trained models.
C. Training Details and Evaluation Methods
The evaluation uses five-fold cross-validation on the TV set and deployment of each fold’s model on an independent test set. Attention refinement improves key validation measures, while dual sampling achieves the strongest overall validation performance and remains robust across training folds.
- Evaluation protocol: Five-fold cross-validation is conducted on the TV set, with each fold’s trained model evaluated on the independent test set.The test-set ROC curves correspond to models trained in TV folds 1 through 5.
- TV-set results: Attention refinement improves AUC and sensitivity in the combined five-fold TV results.The comparison is made against models without the proposed attention refinement technique.
- TV-set results: Attention RN34 + DS achieves the highest TV-set AUC, accuracy, sensitivity, and F1-score among the compared models.Its specificity is slightly lower than ResNet34 with uniform sampling.
- Independent testing: 0.988 to 0.944 is the AUC decrease for Attention RN34 + DS from the TV set to the independent test set.RN34 + US decreases from 0.984 to 0.934 over the same evaluation transition.
- Statistical analysis: p-values below 0.01 indicate significant improvements of the proposed methods over RN34 + US in paired t-tests.The tests compare RN34 + US with the three proposed methods.
E. Detailed Analysis
The detailed analysis examines performance across infection-size groups and visualizes model attention in axial and coronal CT views. Size-balanced sampling improves selected extreme-group measures, while dual sampling preserves abilities from uniform sampling and attention maps better target lung infection regions.
- Infection-size analysis: CT images are divided into small, median, and large infection groups using lung-to-pneumonia volume ratios of < 0.005, 0.005 −0.030, and > 0.030.Most COVID-19 images have ratios above 0.030, whereas most CAP images are below 0.005.
- Attention visualization: Figure 5 compares original CT images, VB-Net lung and infection segmentations, Grad-CAM, and proposed attention maps across three infection-size groups.Each case is shown in axial and coronal views for RN34 + US, Attention RN34 + US, and Attention RN34 + SS.
- Sampling effects: 0.534 to 0.569 is the sensitivity increase for the small-infection group under size-balanced versus uniform sampling.Specificity for the large-infection group also increases from 0.642 to 0.667.
- Sampling effects: 0.965 to 0.955 is the sensitivity decrease for the large-infection group when only size-balanced sampling is used.Specificity for the small-infection group decreases from 0.933 to 0.896, showing trade-offs across extreme groups.
- Sampling effects: Dual sampling preserves the classification ability obtained by uniform sampling across the small- and large-infection groups.The strategy combines the strengths of the two sampling conditions through ensemble learning.
F. Failure Analysis
Failure cases show that the models can activate on regions unrelated to pneumonia, while small-infection COVID-19 cases remain insufficiently accurate. The authors identify needs for better attention localization, clinical-test integration, and broader evaluation.
- Failure cases: Two COVID-19 cases were misclassified as CAP by every model.These cases are presented as visualization failure cases in Fig. 6.
- Attention failures: All models incorrectly activated attention maps on many areas unrelated to pneumonia.The RN34 + US model also highlighted many non-lung regions instead of focusing on the lungs.
- Attention failures: Attention constraints partially alleviated irrelevant activations, but visual evidence remained insufficient for correct prediction.This limitation affected both Attention RN34 + US and Attention RN34 + SS.
- Future improvements: Future work should test longitudinal consistency and relate attention localizations to clinically used imaging signs.The authors also propose improving localization and reducing distortion from confounding visual information.
- Clinical scope: Small-infection-area COVID-19 accuracy was not satisfactory, motivating combined use of CT, clinical assessment, and laboratory tests.The authors frame this combination as necessary for precise diagnosis of early COVID-19.
- Clinical scope: The CAP dataset lacked bacterial, fungal, and non-COVID-19 viral pneumonia subtype information.The authors identify subtype assistance as a beneficial direction for future clinical diagnosis.