Source-linked AI summary
Annotating anatomy and pathology in the National Lung Screening Trial computed tomography images
Deepa Krishnaswamy, Vamsi Thiriveedhi, Suraj Pai, David Clunie, Igor Octaviano, Christopher P. Bridge, Steve Pieper, Ron Kikinis, Andrey Fedorov
TL;DR
The NLST provides extensive public CT data, but precise tumor annotations and interoperable sharing of image-derived results were limited. This paper releases three complementary DICOM datasets, harmonizing prior annotations and adding anatomy segmentations and radiomics features. The datasets use standardized DICOM objects and are shared through the IDC.
Problem
The public NLST CT collection lacked precise tumor annotations, and nonstandard formats made image-derived data harder to visualize, link, and aggregate.
Source: Background & Summary · pp. 1–2Method
The authors harmonized Sybil and NLSTseg annotations and generated TotalSegmentator anatomy segmentations and radiomics features in DICOM datasets.
Source: Methods · p. 2, Background & Summary · p. 2Results
Three DICOM datasets complement NLST CT images with TotalSegmentator anatomy segmentations and features, Sybil bounding boxes, and NLSTseg lesion segmentations.
Source: Background & Summary · p. 2Takeaways & Limitations
DICOM representation enables FAIR data management and standardized sharing of the datasets through the IDC.
Source: Background & Summary · p. 2, Data Records · p. 5
Abstract
from arXiv · showhide
Large-scale public medical imaging datasets contribute critically to translational research. When accompanied by rich clinical and multi-omics data, they can stimulate exploratory research and enable secondary analyses. Expert annotations of such imaging collections can support the development of new image analysis tools. Continuous enrichment of images with image-derived data makes them more usable for researchers without expertise in image analysis or access to large-scale computational resources. The National Lung Screening Trial (NLST) released a rich longitudinal dataset that includes Computed Tomography (CT) images for over 26,000 patients. We introduce three Digital Imaging and Communications in Medicine (DICOM) formatted datasets, complementing NLST CT images, shared as analysis results in the National Cancer Institute Imaging Data Commons (IDC). Two of those (IDC NLSTSeg and IDC NLSTSybil) contain DICOM-harmonized annotations and extracted measurements (for 581 and 601 NLST patients, respectively) shared earlier using research formats (Sybil and NLSTseg). The third one (TotalSegmentator-CT-Segmentations) contains volumetric segmentations generated using TotalSegmentator and radiomics features for each segment for 26,194 NLST patients.
Background & Summary
The NLST offers extensive public CT data, but precise tumor annotations and interoperable ways to share image-derived items were lacking. This work presents three DICOM datasets that complement NLST images with anatomy segmentations, tumor annotations, and derived features.
Background & Summary
- The NLST collection lacked precise tumor-location or margin annotations, while image-derived analysis artifacts were rarely shared.Source: Background & Summary · p. 1
- Earlier Sybil and NLSTseg annotation datasets were not well integrated with the NLST imaging collection.Source: Background & Summary · p. 2
- Custom JSON and NIfTI representations complicate visualization, image-linking, standardized archiving, and aggregation across datasets; DICOM provides shared metadata and identifiers for these uses.Source: Background & Summary · p. 2
- Three DICOM datasets add TotalSegmentator anatomy segmentations and radiomics features, Sybil bounding boxes, and NLSTseg lesion segmentations to NLST CT images.The datasets use DICOM SEG for pixel-level annotations and Structured Reports for slice-level boxes and segmentation-derived features.Source: Background & Summary · p. 2
Methods
- The work curates harmonized versions of Sybil and NLSTseg expert annotations, adds AI-generated anatomy annotations, and extracts quantitative features from volumetric segmentations.Source: Methods · p. 2
NLST CT collection
The NLST is a large public low-dose CT screening-trial collection with longitudinal scans and accompanying clinical data.
NLST CT collection
- The NLST includes over 26,000 patients and more than 200,000 CT scans, with up to three time points per patient and rich clinical data.Publicly available materials include radiology, digital pathology, and some clinical data; complete clinical data can be requested from CDAS.Source: NLST CT collection · p. 3
Standardized DICOM objects
DICOM SEG and SR provide standard representations for image-derived annotations and measurements, with metadata and references that support interoperability.
Standardized DICOM objects
- DICOM SEG stores voxel-level segmentations with metadata describing the segmented region, its creation, and the referenced image.Source: Standardized DICOM objects · p. 3
- DICOM SR encodes text, image annotations, measurements, or observations in a hierarchical content tree using coded terms and references to related evidence.Source: Standardized DICOM objects · p. 3
- The authors used highdicom, dcmqi, and dcm2niix as open-source tools for conversion and dataset use.Source: Standardized DICOM objects · p. 3
NLST CT Annotations
The paper converts Sybil and NLSTseg annotations and TotalSegmentator outputs into DICOM objects, adding standardized metadata and radiomics features. The resulting collections are publicly available through the IDC.
Sybil
- Sybil bounding boxes were losslessly encoded as DICOM SR TID1500 spatial coordinates, yielding 970 instances with 9,280 annotations for 581 patients.Some JSON entries lacked bounding-box definitions and were skipped during conversion.Source: Sybil · pp. 3–4
NLSTseg
- NLSTseg NIfTI segmentations were losslessly converted to DICOM SEG with coded tumor/nodule type and lung-lobe location, representing 705 lesions across 601 CT series and 601 patients.DICOM TID1500 SR objects were generated for 600 patients; feature extraction failed for one case because its segmentation and referenced CT volumes had different dimensions.Source: NLSTseg · pp. 4–5
TotalSegmentator
- TotalSegmentator processing selected suitable NLST CT series, generated organ segmentations, and extracted first-order and shape features for each segmented structure.The initial selection comprised 126,088 series from 26,194 patients; 37 series were later removed for conversion or geometry issues.Source: TotalSegmentator · p. 5
Data Records
- The resulting collections are provided as DICOM SEG and Structured Report objects, with Zenodo descriptors and access through the IDC.Source: Data Records · p. 5
Data Overview
- NLST-Sybil uses a TID1500 SR object for each annotated CT series, while NLSTSeg and TotalSegmentator collections use SEG objects for each CT series.The segmentations are accompanied by TID1500 SR objects containing first-order and shape radiomics features.Source: Data Overview · pp. 5–6
DICOM conformance
SEG and SR objects were checked for DICOM IOD conformance, and structured report content trees were checked against the TID1500 template.
DICOM conformance
- DICOM conformance was checked with dciodvfy for SEG and SR objects and PixelMed's DICOMSRValidator for structured report content trees against TID1500.dciodvfy verifies conformance to the DICOM Information Object Definition (IOD).Source: DICOM conformance · p. 6
Qualitative verification
The authors combined metadata dashboards with image viewers to inspect distributions, review selected outliers, and check annotations against clinical information. Verification included a scan-coverage outlier, tumor-location and margin examples, and a quantitative check of lesion volumes.
Workflow
- The BigQuery metadata tables supported completeness and veracity checks and data exploration, while dashboards and OHIF enabled metadata filtering and image review.Individual studies could also be examined in 3D Slicer; the authors reviewed visual outliers selected from attribute distributions.Source: Qualitative verification · pp. 6–7
TotalSegmentator-CT-Segmentations
- A low-volume right middle-lobe outlier was traced to incomplete anatomical coverage in the CT scan during visual inspection.The workflow selected extreme values from volume distributions and examined the corresponding cases in OHIF.Source: Qualitative verification · p. 8, Figure 3 · p. 8
NLST-Sybil
- The NLST-Sybil dashboard linked tumor and patient characteristics to annotations, enabling visual confirmation that a selected tumor was in the reported right upper lobe.The example used clinical metadata filtering followed by OHIF visualization.Source: Qualitative verification · p. 8, Figure 4 · p. 9
NLSTSeg
- The NLSTSeg dashboard linked lesion segmentations with clinical data and let users compare tumor margins with reported characteristics and cancer stage.Examples included checking whether spiculated margins appeared irregular and smooth margins appeared well-defined.Source: Qualitative verification · p. 9, Figure 5 · p. 10
- 0.99 Pearson correlation coefficient and 1.92*10-5土0.0029 mL mean difference were found between recalculated and previously reported NLSTSeg tumor volumes.Volumes were recalculated with pyradiomics.Source: Qualitative verification · p. 9
Cross-collection consistency evaluation
The dashboards compare Sybil bounding boxes, NLSTSeg lesion masks, and TotalSegmentator lung-lobe segmentations to assess annotation agreement. They also provide tools for visualizing and parsing the standardized annotation datasets.
Cross-collection consistency evaluation
- For overlapping Sybil boxes and NLSTSeg masks, tumor fraction measures voxels inside boxes; slice precision stayed 1.0, so recall tracked out-of-plane discrepancies.The comparison filters to series and slices containing both annotation types.Source: Cross-collection consistency evaluation · pp. 10–11
- 26% of tumor locations disagreed across 575 dual-annotated series; Figure 8 shows corresponding lung-region overlap alongside extraneous segment assignments.TotalSegmentator segments also overlap extraneous lung regions, cardiac substructures, and ribs that disagree with the expert-assigned lesion location.Source: Cross-collection consistency evaluation · p. 12, Figure 8 · p. 13
Usage Notes
- Usage notes describe viewing annotations in Slicer and OHIF and parsing DICOM SEG and SR objects with a Colab notebook.Slicer support includes 2D boxes, 3D points, and 2D lines; the notebook demonstrates obtaining referenced images for enhanced lesion visualization.Source: Usage Notes · p. 13
Dataset-specific usage notes
The NLST-Sybil resources support exploratory content-based image retrieval, and the paper provides access to converted datasets and code.
NLST-Sybil
- Nine foundation models generate tumor-region embeddings for an interactive graph linking segmentations by cosine similarity, alongside patient characteristics and clinical metadata.The interface supports exploration of data relationships and clustering patterns.Source: NLST-Sybil · p. 14, Figure 9 · p. 15
Data Availability
- The converted and generated datasets are available through Zenodo records and the IDC portal.The passage points readers to the records and portal URLs.Source: Data Availability · p. 15
Code Availability
- The code is publicly available in the idc-nlst-plus GitHub repository under a permissive MIT license.The repository includes data-record notebooks, technical validation materials, usage notes, and SQL resources.Source: Code Availability · pp. 15–16
Competing Interests
The authors declare no competing interests and report federal funding from the National Cancer Institute and National Institutes of Health.
Competing Interests
- The authors declare no competing interests.Source: Competing Interests · p. 18
Funding
- The work received federal funding from the National Cancer Institute, National Institutes of Health.Funding was provided in whole or in part under the stated task order and contract.Source: Funding · p. 18