Source-linked AI summary
VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations
Ha Q. Nguyen, Khanh Lam, Linh T. Le, Hieu H. Pham, Dat Q. Tran, Dung B. Nguyen, Dung D. Le, Chi M. Pham, Hang T. T. Tong, Diep H. Dinh, Cuong D. Do, Luu T. Doan, Cuong N. Nguyen, Binh T. Nguyen, Que V. Nguyen, Au D. Hoang, Hien N. Phan, Anh T. Nguyen, Phuong H. Ho, Dat T. Ngo, Nghia T. Nguyen, Nhan T. Nguyen, Minh Dao, Van Vu
TL;DR
Existing chest X-ray datasets often lack reliable abnormality localization, limiting annotated resources for detection and localization research. VinDr-CXR releases 18,000 publicly available DICOM scans with radiologist-generated bounding boxes and disease labels, making it the largest public CXR dataset with annotations in both training and test sets.
Problem
Existing chest X-ray datasets often provide finding labels without abnormality locations, while localized datasets are too small or insufficiently detailed.
Method
The authors retrospectively collected, filtered, de-identified, and radiologist-annotated chest X-rays from two Vietnamese hospitals with local and global labels.
Results
VinDr-CXR releases 18,000 DICOM scans with bounding-box localizations and disease labels, including three-reader training annotations and five-reader test consensus labels.
Takeaways & Limitations
VinDr-CXR is presented as the largest public chest X-ray dataset with radiologist-generated annotations in both training and test sets.
Takeaways & Limitations
Access requires accepting a credentialed health-data license that restricts use to scientific research and education and prohibits data sharing and re-identification attempts.
Abstract
from arXiv · showhide
Most of the existing chest X-ray datasets include labels from a list of findings without specifying their locations on the radiographs. This limits the development of machine learning algorithms for the detection and localization of chest abnormalities. In this work, we describe a dataset of more than 100,000 chest X-ray scans that were retrospectively collected from two major hospitals in Vietnam. Out of this raw data, we release 18,000 images that were manually annotated by a total of 17 experienced radiologists with 22 local labels of rectangles surrounding abnormalities and 6 global labels of suspected diseases. The released dataset is divided into a training set of 15,000 and a test set of 3,000. Each scan in the training set was independently labeled by 3 radiologists, while each scan in the test set was labeled by the consensus of 5 radiologists. We designed and built a labeling platform for DICOM images to facilitate these annotation procedures. All images are made publicly available (https://www.physionet.org/content/vindr-cxr/1.0.0/) in DICOM format along with the labels of both the training set and the test set.
Background & Summary
Existing CXR datasets enable strong CAD performance but often rely on noisy report-derived labels and omit abnormality locations. VinDr-CXR addresses these limitations with a large public dataset containing radiologist-generated annotations for both localization and classification.
- Background: Large labeled CXR datasets and deep CNNs have enabled CAD systems to reach expert-level performance for classifying common lung diseases and related findings.Training CNNs depends heavily on high-quality datasets.
- Limitations of existing datasets: Report-derived labels can contain inconsistency, uncertainty, and errors, potentially causing poorer real-world performance than reported.Automated keyword-based and NLP labelers scale labeling but introduce noise.
- VinDr-CXR dataset: 18,000 VinDr-CXR postero-anterior scans from more than 100,000 raw DICOM images provide radiologist-annotated localization and classification labels.The raw images were retrospectively collected from Hospital 108 and Hanoi Medical University Hospital in Vietnam.
- Contribution: VinDr-CXR is presented as the largest public CXR dataset with radiologist-generated annotations in both training and test sets.The dataset is intended to accelerate machine-learning research on localization and classification of thoracic lesions and diseases.
Methods
VinDr-CXR was built by retrospectively collecting and filtering more than 100,000 DICOM chest radiographs from two Vietnamese hospitals, then labeling 18,000 adult PA-view scans with a web-based annotation tool. Seventeen experienced radiologists produced blinded annotations for separate training and test sets.
- Data collection: More than 100,000 DICOM chest radiographs were retrospectively collected from two Vietnamese hospitals between 2018 and 2020.Sources included the hospitals’ local PACS servers and a wide diversity of scanner manufacturers.
- Data filtering: A lightweight CNN-based binary classifier automatically removed non-chest, pediatric, low-quality, lateral, and incorrectly oriented scans.The filtering retained valid adult PA-view chest radiographs for labeling.
- Data labeling: Annotations covered 28 findings and diagnoses, with labels exported as JSON and organized into a CSV containing labels, bounding-box coordinates, and image IDs.Training samples include annotations from three radiologists, while the test set provides consensus annotations.
- Data labeling: 18,000 randomly selected filtered scans were divided into 15,000 training studies and 3,000 test studies.The training set contained 10,606 normal and 4,394 abnormal studies; the test set contained 2,052 normal and 948 abnormal studies.
- Data labeling: Seventeen certified radiologists remotely annotated the scans using the web-based VinDr Lab framework while blinded to clinical information.Each training scan was independently assigned to three radiologists, whereas five radiologists labeled the test set through a two-stage process.
Data Records
VinDr-CXR is publicly downloadable through PhysioNet, providing imaging data and ground-truth labels for both training and test sets. The records are separated by split and identified using anonymous encodings of DICOM SOP Instance UIDs.
- The dataset is publicly available for download through PhysioNet with imaging data and corresponding ground-truth labels for both training and test sets.
- Images are organized into separate training and testing folders.
- Each image has a unique anonymous identifier encoded from its DICOM SOP Instance UID.
Technical Validation
The dataset underwent controlled de-identification and manual quality review to remove residual patient-identifying information from DICOM metadata and chest X-ray images.
- Privacy and quality control: DICOM metadata was parsed and manually reviewed to remove individually identifiable patient health information under HIPAA, GDPR, and local privacy laws.The de-identification process was controlled and explicitly checked against these privacy requirements.
- Privacy and quality control: 10 human readers manually reviewed every image, excluding scans that retained private textual information after algorithmic de-identification.Pixel values were also carefully examined during the case-by-case review.
Usage Notes
Access to the VinDr-CXR dataset requires accepting PhysioNet’s credentialed health-data license, which restricts use to scientific research and education and prohibits data sharing or patient, institution, and hospital re-identification.
- Usage Notes: Dataset access requires accepting the PhysioNet Credentialed Health Data License 1.5.0, which prohibits data sharing and re-identification and limits use to scientific research and education.The license is called a Date Usage Agreement (DUA).
Competing interests
The work was funded by Vingroup JSC, which had no role in the study’s design, data collection, analysis, publication decision, or manuscript preparation.
- Vingroup JSC funded the work but had no role in study design, data collection and analysis, publication decision, or manuscript preparation.
Supplementary materials
The supplementary materials define the findings and diseases used in the study and list the DICOM tags retained for image loading and processing. The retained-tag list also notes that other tags were removed to protect patient privacy.
- Table 3 defines the findings and diseases used in the study.
- Table 4 lists the DICOM tags retained for loading and processing raw images.All other tags were removed to protect patient privacy.