Source-linked AI summary
Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Images
Xiaocong Chen, Lina Yao, Tao Zhou, Jinming Dong, Yu Zhang
TL;DR
COVID-19 diagnosis faces testing delays and RT-PCR limitations, while annotated CT data are difficult to obtain. The paper develops a deep-learning few-shot method and reports superior performance to ResNet-50 when more than three samples are available.
Problem
COVID-19 testing can take several hours, while RT-PCR has reported sensitivity limitations and testing may introduce unpredictable bias.
Method
The paper develops an end-to-end trainable deep few-shot method using momentum to mitigate key-update effects and overlapping views for augmentation.
Results
The proposed method achieves superior performance to ResNet-50 when the number of available samples is larger than three, based on experiments on two datasets.
Takeaways & Limitations
The method provides a promising approach for COVID-19 diagnosis under few-shot training conditions.
Takeaways & Limitations
Testing may introduce unpredictable bias during the testing phase.
Abstract
from arXiv · showhide
The current pandemic, caused by the outbreak of a novel coronavirus (COVID-19) in December 2019, has led to a global emergency that has significantly impacted economies, healthcare systems and personal wellbeing all around the world. Controlling the rapidly evolving disease requires highly sensitive and specific diagnostics. While real-time RT-PCR is the most commonly used, these can take up to 8 hours, and require significant effort from healthcare professionals. As such, there is a critical need for a quick and automatic diagnostic system. Diagnosis from chest CT images is a promising direction. However, current studies are limited by the lack of sufficient training samples, as acquiring annotated CT images is time-consuming. To this end, we propose a new deep learning algorithm for the automated diagnosis of COVID-19, which only requires a few samples for training. Specifically, we use contrastive learning to train an encoder which can capture expressive feature representations on large and publicly available lung datasets and adopt the prototypical network for classification. We validate the efficacy of the proposed model in comparison with other competing methods on two publicly available and annotated COVID-19 CT datasets. Our results demonstrate the superior performance of our model for the accurate diagnosis of COVID-19 based on chest CT images.
1. Introduction
COVID-19 diagnosis requires rapid, accurate methods because RT-PCR can be slow and have limited sensitivity, while CT-based approaches face scarce labeled training data. The paper proposes a few-shot framework using contrastive representation learning and prototypical classification, reporting promising diagnosis with limited data.
- Diagnostic motivation: RT-PCR testing can take several hours, and reported low sensitivity and accuracy may require repeated testing.These constraints complicate timely confirmation during rapid disease spread.
- Diagnostic motivation: Chest CT imaging has been studied as a rapid COVID-19 diagnostic modality, including for patients without clinical symptoms.Reported CT features include consolidation, pleural effusion, and ground-glass opacification.
- Data limitations: Existing CT diagnosis methods are trained on limited samples and may not generalize well to new patients.The paper identifies small patient numbers and limited labeled training data as central challenges for deep learning.
- Data limitations: Hand-tuned augmentation can lead to over-fitting, while GAN-generated images may introduce unpredictable testing bias.The paper presents these issues as limitations of common approaches for alleviating scarce-data problems.
- Data limitations: ImageNet pre-training may provide irrelevant information for lung-specific representations and requires high computational cost.The paper notes that ImageNet-1B normally requires more than 50 GPU days.
- Proposed approach: The proposed framework uses instance discrimination, momentum contrastive training, lung-dataset pre-training, and prototypical networks for few-shot CT classification.The momentum mechanism updates key and query encoders at different scales, while classification uses distances to class prototypes.
- Evaluation: Experiments on two public datasets indicate that the model is a promising tool for quick COVID-19 diagnosis with very limited training data.The reported conclusion concerns accurate diagnosis under a few-shot setting.
2. Problem Definition
COVID-19 CT diagnosis is formulated as a few-shot classification problem because annotated images and training samples are limited. The task uses episodic learning with classes and only a few samples per class.
- Limited annotated COVID-19 CT images motivate formulating diagnosis as a few-shot classification problem.
- Few-shot learning addresses classification when only a few samples are available for a new class.
- An M-way, C-shot episode contains M classes with C samples available for each class.
- The dataset is represented as paired samples and labels, with d denoting the number of samples.
- COVID-19 diagnosis is represented as a two-way, C-shot learning problem using support and query sets.
3. Methodology
The method combines stochastic augmentation, contrastive representation learning, and prototypical few-shot classification. Momentum-based contrastive training maintains a larger, more consistent dictionary of negative representations.
- The proposed self-supervised COVID-19 diagnosis method has three components: data augmentation, representation learning, and few-shot classification.
- Data Augmentation: Stochastic augmentation generates two different views of each image through random cropping, resizing, flipping, and optional color distortion.
- Few-Shot Classification: The prototypical network classifies queries by comparing their embeddings with class centroids using Euclidean distance.
- Contrastive Visual Embedding: Contrastive learning treats paired augmented views from the same instance as positives and the remaining 2N −2 examples as negatives.
- Contrastive Visual Embedding: The representation-learning stage uses an encoder, projection head, and contrastive loss to learn embeddings from augmented images.
- Contrastive Visual Embedding: The momentum mechanism addresses noisy key representations caused by constantly changing encoders by updating the key encoder with a momentum coefficient.
- Contrastive Visual Embedding: A queue reuses encoded keys from previous minibatches, providing a larger dictionary while progressively removing old records to maintain consistency.
4. Experiments
The model is evaluated on two publicly available annotated COVID-19 CT datasets and uses large public lung CT datasets for pre-training. The described training algorithm updates the query and key encoders using contrastive objectives.
- The proposed model is evaluated using two publicly available annotated COVID-19 CT image datasets.
- Pre-training uses DeepLesion and LIDC-IDRI rather than ImageNet, because the method requires proper pre-training.
- DeepLesion contains over 32,000 lung CT images, while LIDC-IDRI contains 244,617 images.
- The pre-training algorithm selects two augmentation functions and computes similarities for the resulting augmented samples.
- Training updates the key encoder to minimize the contrastive loss and updates the query encoder using the momentum rule.
15 end
The study evaluates a few-shot COVID-19 CT diagnosis method using patient-level evaluation, comparisons with baseline models, and ablations of pre-training, augmentation, projection, and fine-tuning choices.
- Comparison results: The proposed two-way, one-shot method achieves very strong results compared with the evaluated baseline methods.The cited passage introduces the Table 2 comparison but does not provide the numerical values in this excerpt.
- Evaluation setup: The experiments use two-way, one-shot classification and patient-level data division to avoid possible over-fitting.The datasets are combined for testing, and CT images are resized to 512 × 512.
- Few-shot results: When the number of shots exceeds 3, the proposed model performs better than ResNet-50.Performance increases gradually as the number of shots increases.
- Ablation study: The authors report that the pre-trained encoder can capture features from unknown data, while ImageNet pre-training performs worse in the ablation.The encoder uses lung-disease datasets for unlabeled pre-training, and the ImageNet comparison is reported in Table 4.
- Ablation study: Data augmentation has a significant effect, and the projection head boosts performance in the ablation study.The experiments compare the model with and without augmentation and with and without the projection head.
5. Conclusion
The paper addresses the shortage of annotated CT scans for COVID-19 screening by proposing a deep-learning method designed for limited samples. It reports superior performance to ResNet-50 when more than three samples are available.
- 5. Conclusion: The lack of annotated CT scans is identified as the biggest challenge for COVID-19 diagnosis using CT imaging.CT imaging is presented as a screening tool that can visualize disease progression and help evaluate severity.
- 5. Conclusion: The proposed deep-learning method is intended for automatic COVID-19 screening with limited samples.The conclusion describes it as a self-supervised method belonging to the unsupervised learning field.
- 5. Conclusion: The method achieves superior performance to ResNet-50 when the number of available samples is larger than three.ResNet-50 is described as a widely used supervised learning model in medical imaging.