Source-linked AI summary
POCOVID-Net: Automatic Detection of COVID-19 From a New Lung Ultrasound Imaging Dataset (POCUS)
Jannis Born, Gabriel Brändle, Manuel Cossio, Marion Disdier, Julie Goulet, Jérémie Roulin, Nina Wiedemann
TL;DR
The paper addresses the need for fast, reliable COVID-19 assessment by investigating automated detection from lung ultrasound. It builds a processed POCUS dataset, trains POCOVID-Net, and deploys the model through a public web service, achieving 89% classification accuracy and 92% video accuracy. The authors present this as a preliminary research tool rather than a clinically validated diagnostic system.
Problem
Automatic COVID-19 detection had focused on X-ray or CT, leaving ultrasound without a comparable detection framework despite its accessibility and portability.
Method
The authors assemble and preprocess POCUS recordings, train a convolutional neural network on the resulting dataset, and provide a web service for model use and data contribution.
Results
89% classification accuracy and 92% video accuracy were achieved, with 96% sensitivity and 79% specificity for COVID-19 detection.
Takeaways & Limitations
The study provides an open dataset, model, and web interface as a starting point for further automated COVID-19 detection research using lung ultrasound.
Takeaways & Limitations
The model outputs are preliminary and should not be considered clinically significant or a replacement for other tests or physician ultrasound training.
Abstract
from arXiv · showhide
With the rapid development of COVID-19 into a global pandemic, there is an ever more urgent need for cheap, fast and reliable tools that can assist physicians in diagnosing COVID-19. Medical imaging such as CT can take a key role in complementing conventional diagnostic tools from molecular biology, and, using deep learning techniques, several automatic systems were demonstrated promising performances using CT or X-ray data. Here, we advocate a more prominent role of point-of-care ultrasound imaging to guide COVID-19 detection. Ultrasound is non-invasive and ubiquitous in medical facilities around the globe. Our contribution is threefold. First, we gather a lung ultrasound (POCUS) dataset consisting of 1103 images (654 COVID-19, 277 bacterial pneumonia and 172 healthy controls), sampled from 64 videos. This dataset was assembled from various online sources, processed specifically for deep learning models and is intended to serve as a starting point for an open-access initiative. Second, we train a deep convolutional neural network (POCOVID-Net) on this 3-class dataset and achieve an accuracy of 89% and, by a majority vote, a video accuracy of 92% . For detecting COVID-19 in particular, the model performs with a sensitivity of 0.96, a specificity of 0.79 and F1-score of 0.92 in a 5-fold cross validation. Third, we provide an open-access web service (POCOVIDScreen) that is available at: https://pocovidscreen.org. The website deploys the predictive model, allowing to perform predictions on ultrasound lung images. In addition, it grants medical staff the option to (bulk) upload their own screenings in order to contribute to the growing public database of pathological lung ultrasound images. Dataset and code are available from: https://github.com/jannisborn/covid19_pocus_ultrasound. NOTE: This preprint is superseded by our paper in Applied Sciences: https://doi.org/10.3390/app11020672
1 Introduction
The paper addresses the need for rapid, reliable COVID-19 assessment by exploring automated detection from lung ultrasound, a portable and accessible imaging modality. It introduces a POCUS dataset, a deep-learning framework, and a public web service for screening and data contribution.
- RT-PCR is reliable but can take more than an hour, while serology tests develop antibodies only at advanced disease stages.
- POCUS is noninvasive, repeatable, portable, disinfectable, and less expensive than chest X-ray or CT.The reported examination costs are $140 for ultrasound, $370 for chest X-ray, and $675–$8600 for chest CT.
- The paper targets the gap in automatic COVID-19 detection from ultrasound, which prior machine-learning work had not addressed.
- The authors propose the first framework for automated COVID-19 detection on ultrasound images.
- The released dataset contains COVID-19, pneumonia, and healthy-patient POCUS recordings that were manually pre-processed and medically checked for quality.
- POCOVID-Net achieved 89% classification accuracy and 96% COVID-19 sensitivity in 5-fold cross validation.
- The free web service supports both clinical-data uploads and access to the trained model.
2 Related work
The related work positions ultrasound as a promising but underused modality for COVID-19 imaging, contrasting it with X-ray and CT and highlighting the lack of public ultrasound datasets and automatic detection systems.
- Biomedical imaging is presented as a complement to molecular and immunological COVID-19 tests by enabling rapid patient assessment.
- Chest X-rays may be normal or only mildly abnormal in early COVID-19, whereas CT is described as more viable for early detection.
- Ultrasound can reveal pleural and interstitial thickening, subpleural consolidation, B-lines, and lower-lobe lesions associated with COVID-19.
- Before this work, automatic COVID-19 imaging studies had focused on X-ray or CT rather than ultrasound.
- Existing open-access imaging efforts included small COVID-19 X-ray collections, while CT datasets underlying reported systems remained unavailable publicly.
- Table 1 provides an overview of the study's most important data sources.
3 A lung US dataset for COVID-19 detection
The authors assembled a heterogeneous lung POCUS dataset from online sources, converting 64 videos into 1,103 processed images across COVID-19, bacterial pneumonia, and healthy classes.
- The dataset contains 64 videos: 39 COVID-19, 14 bacterial pneumonia, and 11 healthy-patient recordings.
- The collection is restricted primarily to convex probes because more such data were available, although linear-probe recordings were also collected.
- Data sources included community platforms, medical repositories, health-tech companies, and scientific literature.
- The source videos were heterogeneous, with disease-specific patterns documented in COVID-19 and pneumonia recordings.
- Sampling at 3 Hz with at most 30 frames per video produced 1,103 images: 654 COVID-19, 277 bacterial pneumonia, and 172 healthy.
- Images were cropped to remove surrounding bars and text, and five videos were manually edited to remove overlaid measurement scales and artifacts.
4 Classification with POCOVID-Net
POCOVID-Net classifies lung ultrasound frames into COVID-19, pneumonia, and healthy categories using a convolutional neural network evaluated with video-disjoint 5-fold cross-validation. It achieves strong COVID-19 sensitivity and 89% frame accuracy, while healthy-class errors and limited healthy recordings remain important concerns.
- Methods: POCOVID-Net uses a VGG-16-based convolutional architecture with a 64-neuron hidden layer, dropout, batch normalization, and softmax output.The model was pretrained on ImageNet and optimized with Adam.
- Methods: All results use 5-fold cross-validation with frames from each video restricted to a single fold, keeping recordings disjoint between training and testing.Fold sizes vary because the recordings are kept intact.
- Frame-wise results: 96% sensitivity was obtained for COVID-19, with 628 of 654 COVID-19 images classified correctly.Pneumonia achieved 93% sensitivity and 95% precision, while 75 COVID-19 images were classified as healthy.
- Model comparison: POCOVID-Net achieved 89% accuracy versus 81% for COVID-Net and 82% versus 63% balanced accuracy in the reported comparison.The comparison identifies difficulty handling unbalanced classes as a weakness of COVID-Net.
- Video-level results: 92% video accuracy was obtained when frame-wise predictions were aggregated by majority vote or averaged class probabilities.Both aggregation methods achieved the same video accuracy, while balanced video accuracy was 84%.
5 Web service (POCOVIDScreen)
POCOVIDScreen is an open-access web service that combines POCOVID-Net screening with user-submitted ultrasound data collection. It displays an averaged prediction and probability, but the authors characterize the deployed model as preliminary and not clinically significant.
- Data contribution: POCOVIDScreen lets users upload ultrasound images or videos to contribute COVID-19, pneumonia, or healthy recordings to the open-access dataset.Submitted data are reviewed by medical doctors, processed, and integrated into the GitHub database.
- Model access: The service also lets users test POCOVID-Net on their own ultrasound images or videos.The platform was designed to simplify data sharing and provide rapid model access.
- Prediction workflow: The deployed service evaluates all five cross-validation models, averages their output scores, and displays the predicted class with a probability.The authors recommend cropping inputs to a quadratic section of the relevant region, similarly to the training data.
- Scope: The authors describe the tool as a preliminary starting point whose outputs should not be considered of clinical significance.They frame the platform as support for future model development and community data collection.
6 Discussion
The discussion presents POCUS and POCOVID-Net as early steps toward automated COVID-19 assessment, supported by an open dataset, model, and web service. The authors emphasize promising preliminary performance while limiting diagnostic claims and identifying data-related improvements.
- The study explores automated COVID-19 detection from lung ultrasound to provide a quick assessment of possible infection.
- The released POCUS collection contains pre-processed, reliably labeled videos that can generate more than a thousand images and support ongoing community contributions.
- 89% detection accuracy and 92% video accuracy were reported, with 96% COVID-19 sensitivity and 79% specificity.
- The authors describe POCOVID-Net as a preliminary or complementary assessment rather than a diagnostic replacement, requiring further testing after a positive examination.
- The web service makes the model publicly available to researchers and hospitals while enabling contributions of additional lung ultrasound data.
- Because of insufficient data, the current frame-based analysis does not perform direct video inference; future work includes temporal models and ultrasound-specific pre-training.
A Details on the dataset
The dataset details document a heterogeneous collection of lung-ultrasound videos and image examples spanning COVID-19, pneumonia, and normal findings. Expert comments describe artifacts, consolidations, effusions, pleural changes, and variable image quality.
- The utilized-video table records source, length, size, frame rate, and medical-expert comments for the collected videos.
- COVID-19-related examples include B-lines, pleural irregularities or thickening, subpleural consolidations, and pleural effusions.
- One expert comment cautions that B-lines are not uniformly confirmatory and that asymptomatic positive cases may lack pneumonia.
- Pneumonia examples include larger consolidations, air bronchograms, hepatization, fluid bronchograms, and effusions.
- The collection contains variable-quality material, including low-resolution, poorly executed, or otherwise atypical images.
- Normal-lung examples are characterized by A-lines, normal pleural lines, or no detected lung-ultrasound abnormalities.