PaperlayerView exact revision on arXiv

Source-linked AI summary

Sharing standardized image-derived data in computational pathology using DICOM

Daniela P. Schacherer, Christopher P. Bridge, David Clunie, Igor Octaviano, André Homeyer, Markus D. Herrmann, Olivier Gevaert, Tabita Ghete, Markus Metzler, Henning Hoefener, Tahsin Kurc, Curtis Lisle, Kenneth Philbrick, Joel Saltz, Yuanning Zheng, Andrey Fedorov

arXiv:2609.14530v1cs.CVcs.AIcs.LG

TL;DR

Sharing pathology image-derived data remains less developed than sharing images, despite its role in developing and evaluating computational methods. This work converts five datasets to DICOM for release through the IDC, demonstrating the format’s use for varied real-world derived data. The collections enrich existing image collections with data that can be searched, accessed, and visualized alongside the images.

  • Problem

    Pathology image-derived data such as annotations and segmentation masks lack shared conventions and public DICOM datasets, despite their role in developing and evaluating computational methods.

    Source: Introduction · pp. 1–2

  • Method

    The authors converted five representative pathology image-derived datasets from their original formats into DICOM and released them through the IDC.

    Source: Introduction · p. 2, Discussion · p. 22

  • Results

    The five converted datasets demonstrate that DICOM can support a range of real-world pathology image-derived data and are publicly available through the IDC.

    Source: Discussion · p. 22, Conclusion · pp. 23–24

  • Takeaways & Limitations

    The DICOM collections enrich existing image collections with derived data that can be searched, accessed, and visualized alongside the source images.

    Source: Discussion · p. 22, Conclusion · p. 24

    DICOMweb does not yet provide spatial querying for MBSAs or SRs, so tools must retrieve the entire object to access intersecting graphics.

    Source: Limitations · p. 22

Abstract

from arXiv · show

Development and evaluation of computational pathology methods require access to large and diverse datasets. Over the past decade, various initiatives invested significantly into collecting, centralizing, and sharing pathology imaging data. In contrast, sharing of image-derived data such as region-of-interest delineations or segmentation masks is less well developed. In this work, we describe our approach to encoding and sharing image-derived pathology data in a standardized manner within the National Cancer Institute (NCI) Imaging Data Commons (IDC), a platform that hosts and provides public access to de-identified radiology and pathology data. The IDC relies on the Digital Imaging and Communications in Medicine (DICOM) standard for data harmonization, yet the adoption of DICOM for pathology image-derived content has remained largely unexplored until now. Here, we present five representative datasets harmonized by conversion from their original representations into DICOM and shared publicly in the IDC. We demonstrate the benefits of this harmonization, describe contributions to critical open-source tooling, and discuss technical considerations relevant to broader adoption of DICOM for pathology image-derived data.

Introduction

Computational pathology depends on large, diverse datasets and image-derived annotations, but such derived data lack a shared convention for public sharing. The paper describes converting five example datasets to DICOM for access through the IDC and developing tools and infrastructure for that process.

Introduction

  • The IDC uses DICOM for data representation and DICOMweb for access, providing a standardized platform for pathology imaging data.As of Spring 2026, it hosts approximately 100 TB of radiology and pathology cancer image data.Source: Introduction · p. 2
  • Image-derived data include slide labels, regions of interest, segmentation masks, and derived quantifications, and expert annotations remain important for fine-tuning and validation.They support development and evaluation of methods for tissue detection, cell segmentation, and tumor-microenvironment characterization.Source: Introduction · p. 2
  • A single standard for file formats and metadata can support interoperability, unified interfaces, and open-source tools for archival, search, access, and visualization.The authors describe these features as supporting maintenance, sustainability, and reproducibility.Source: Introduction · p. 2
  • The work addresses the lack of public DICOM pathology image-derived datasets by releasing five converted examples and developing practical infrastructure.Contributions include notebooks, a dashboard, enhancements to a Python library and web viewer, and discussion of technical trade-offs.Source: Introduction · p. 2

DICOM for encoding data derived from whole slide images

DICOM provides distinct representations for whole-slide images and image-derived data. The four image-derived IODs span raster and vector formats, with different options for encoding masks, measurements, and spatial findings.

DICOM for encoding data derived from whole slide images

  • Whole-slide images are stored as tiled, multi-resolution pyramids, with each pyramid level in a separate DICOM instance and tiles encoded as compressed frames.Tiled-sparse storage records tile locations explicitly and can omit empty tiles, while tiled-full uses a predetermined order.Source: DICOM for encoding data derived from… · p. 3
  • Four IODs cover image-derived content: raster Segmentation (SEG) and Parametric Map (PM), and vector-based Structured Reporting (SR) and Microscopy Bulk Simple Annotations (MBSA).Their attributes encode data identification, meaning, and algorithm provenance, as illustrated in Figure 1.Source: DICOM for encoding data derived from… · p. 3, Figure 1 · p. 5
  • SEG stores raster masks, with binary masks allowing overlapping segments and labelmaps assigning each pixel to a segment in one mask.Fractional SEG represents partial segment membership using continuous values, with probabilistic or occupancy interpretations.Source: DICOM for encoding data derived from… · p. 3
  • PM stores image-derived measurements with coded quantities and units, and supports 32-bit or 64-bit floating-point pixel values.PM can also use tiled storage and optional multi-resolution pyramids.Source: DICOM for encoding data derived from… · p. 4
  • SR organizes measurements, evaluations, and vector regions in a tree, but many similar regions can make SR impractical for large cell-boundary collections.The paper considers SRs using the TID1500 Measurement Report template.Source: DICOM for encoding data derived from… · p. 4
  • MBSA groups similar annotations with shared metadata and stores their graphics in arrays, addressing pathology images with very large numbers of annotations.Groups may include corresponding per-region numerical measurements.Source: DICOM for encoding data derived from… · p. 4

Methods

The study selected five shareable pathology datasets and converted their distinct source formats into DICOM, using common conversion code and dataset-specific representations. The examples span different image-derived content, formats, and sources.

Methods

  • Five datasets were prioritized for cancer-research value and permissive CC-BY licensing; the methods cover common implementation, dataset-specific conversions, validation, and hosting.The criteria for selecting each dataset’s DICOM representation are described in Results because the selection evolved iteratively.Source: Methods · pp. 5–6

DICOM harmonization of image-derived data

  • Custom Python conversion code used pydicom, NumPy, and highdicom, which handles DICOM validation, identifiers, source-image links, and inherited context attributes.The team curated image-derived attributes and used established terminologies for semantic information.Source: DICOM harmonization of image-derived… · p. 6

Cell nuclei contours (Pan-Cancer-Nuclei-Seg-DICOM)

  • Over 6,000 TCGA slides across 14 cancer types had automatically generated nuclei contours converted to MBSA, with additional rasterized binary SEG pyramids.The contours were grouped in one MBSA instance per slide; pixel areas were converted to square microns and stored as measurements.Source: Cell nuclei contours… · p. 6

Rhabdomyosarcoma tissue typing (RMS-Mutation-Prediction-Expert-Annotations)

  • For 96 rhabdomyosarcoma slides, expert tissue contours were encoded as SRs and model predictions as tiled multi-resolution fractional SEGs.The data represent four tissue types, including ERMS and ARMS, in both expert annotations and model predictions.Source: Rhabdomyosarcoma tissue typing… · p. 7

Tumor-infiltrating lymphocytes detection (TCGA-SBU-TIL-Maps)

  • Two TCGA tumor-infiltrating lymphocyte patch-map releases were converted to binary SEG, with fractional SEG also created for the second release.The source maps use patch-level classifications; their low resolution meant multi-resolution pyramids were not needed.Source: Tumor-infiltrating lymphocytes… · p. 7

Glioblastoma tissue characterization (TCGA-GBM360)

  • Glioblastoma patch coordinates and aggressiveness scores were rasterized into one PM per slide, using a private code for the score and UCUM for its unit.The scores are dimensionless values normalized to [0,1], and their low resolution did not require a pyramid.Source: Glioblastoma tissue characterization… · p. 8

Pediatric leukemia bone marrow region and cell annotations (BoneMarrowWSI-PediatricLeukemia)

  • For pediatric leukemia slides, MBSA series encoded ROIs and hematological structures, with separate series for each annotation session when structures were labeled.The conversion stored numeric identifiers and references linking structures to their containing ROIs.Source: Pediatric leukemia bone marrow region… · p. 8

Validation

Validation checked DICOM compliance and user-facing tool behavior rather than generating or scientifically validating the image-derived content. The validated files were staged in the IDC and made accessible through multiple interfaces.

Validation

  • Validation focused on DICOM conformance and tool operation; scientific validation of source data was left to the source-dataset references.The workflow used dciodvfy, dcentvfy, and a DICOM SR validator, then ingested and visualized files before staging them for IDC release.Source: Validation · pp. 9–10

Infrastructure and access

  • IDC access includes cloud buckets on AWS and Google Cloud, a Google Healthcare DICOM Store via DICOMweb, BigQuery SQL, idc-index, and the IDC Portal.The IDC uses one DICOM store and one BigQuery table for metadata across radiology, pathology, and image-derived data.Source: Infrastructure and access · p. 10

Visualization

  • The Slim viewer retrieves image and non-image objects over DICOMweb and renders them client-side in a browser.The project improved Slim’s handling of image-derived DICOM objects.Source: Visualization · p. 10

Results

The section introduces the harmonized collections, demonstrates usability improvements, and describes tooling enhancements and technical considerations, including limitations.

  • The section introduces image-derived collections harmonized from source datasets into DICOM.Source: Results · p. 10
  • It demonstrates practical usability improvements achieved through harmonization.Source: Results · p. 10
  • It describes open-source tool enhancements and technical considerations, including encountered limitations.Source: Results · p. 10

DICOM-converted data collections

The converted image-derived data were ingested into the IDC alongside source images, with later-contributed derived data typically placed in analysis result collections. Representative examples show varied data types and DICOM encodings.

  • Later-contributed image-derived data, often from entities other than the original submitters, are typically placed in analysis result collections.Source: DICOM-converted data collections · p. 10
  • Table 2 overviews the converted collections in IDC v24 and notes nine duplicated SEG series, ten duplicated MBSA series, and one slide excluded because of corrupted tiles.Source: Table 2 · p. 10
  • The converted collections include nuclei contours encoded as SEG or MBSA, expert contours as SR, and model predictions as fractional SEG.Source: Figure 2 · p. 12
  • Examples also show TIL-positive patches encoded as binary or fractional SEG, tumor aggressiveness maps as PM, and an expert ROI with cell bounding boxes as MBSA.Source: DICOM-converted data collections · p. 13

Capabilities enabled by the harmonized DICOM representation

The harmonized representation supports browsing, filtering, and programmatic use of image-derived data across IDC collections. Its capabilities include explicit linkage, coded semantics, contextual metadata, and more uniform access.

Capabilities enabled by the harmonized DICOM representation

Unambiguous identification and formal linkage

Semantic standardization and structured querying

  • Consistent terminology codes encode data meanings and measurements, enabling cross-collection structured queries and filtering by content rather than dataset-specific labels.Source: Semantic standardization and… · p. 14

Context and provenance tracking

  • DICOM objects inherit patient and study context and explicitly encode provenance, supporting evaluation by content, production conditions, and case context, including cross-modality queries.Source: Context and provenance tracking · p. 15

Uniform programmatic access

  • A uniform representation reduces dataset-specific code, while DICOM network compatibility supports standard access and Slim viewer alignment of overlays across collections.Source: Uniform programmatic access · pp. 15–16

Enhancements to open-source tooling

The work enhanced highdicom and the Slim viewer to address tooling gaps in pathology DICOM workflows. Improvements cover efficient data handling, tiled and multi-resolution representations, and rendering at scale.

  • These software enhancements address pathology-critical gaps in existing libraries and viewers and make the released collections and future conversions easier to work with.Source: Enhancements to open-source tooling · p. 17
  • Highdicom improvements reduce SEG creation and parsing operations and memory use, support tiled full and sparse data, and add multi-resolution SEG and PM pyramids.Users can work with total pixel matrices while the tiled representation remains largely hidden.Source: Enhancements to open-source tooling · p. 17
  • Highdicom adds LABELMAP segmentations and MBSA parsing methods for filtering annotation groups and accessing graphic data and measurements as NumPy arrays.Source: Enhancements to open-source tooling · p. 17
  • Slim now supports SEG and PM resolution levels that do not match the source pyramid, avoiding impractical upsampling and implied spatial precision unsupported by the original measurements.Source: Enhancements to open-source tooling · pp. 17–18
  • Slim clusters nearby MBSA ROIs at low magnification and renders them individually when users zoom in, addressing browser performance degradation from rendering many vector graphics.Source: Enhancements to open-source tooling · p. 18

Technical considerations

The paper frames DICOM encoding as a set of choices shaped by how image-derived data were collected and how they will be used. Its recommendations balance representation fit, usability, storage, and interoperability.

Technical considerations

  • Choose raster or vector representation according to how data were collected and their intended downstream uses, which may not point to the same format.The converted datasets generally retained the creators’ submitted representation, with complementary forms added where expected to provide utility.Source: Technical considerations · pp. 18–19
  • Vectors support manual editing and object-level analysis, while raster masks suit pixel-based computational tools but lose information when resampled across pyramid levels.Vector coordinates can be scaled to other pyramid levels; raster masks make annotations in a region directly available but are harder to edit manually.Source: Technical considerations · p. 18
  • Raster-mask size scales with resolution and spatial extent, whereas vector-file size scales with contour detail and can exceed raster size for detailed annotations.Coarse regions may remain compact as vectors, while detailed contours may be larger than their rasterized counterparts; raster masks can use image-specific lossless compression.Source: Technical considerations · p. 19
  • Multi-resolution pyramids are essential for reasonable rendering of high-resolution masks, letting viewers load tiles for the visible area at the appropriate zoom level.Matching segmentation and slide pyramid levels, pixel spacing, and tile dimensions benefits viewers, although the standard does not require this.Source: Technical considerations · p. 19
  • Image-relative coordinates avoid conversion imprecision, while frame-of-reference coordinates apply across images sharing a physical coordinate system.Frame-of-reference coordinates can suit analyses spanning pyramid levels, registered images, or derived images.Source: Technical considerations · p. 19
  • Tiled-full is generally more practical for segmentations because tiled-sparse metadata can become too large for some archives, unless masks are extremely sparse.Oversized metadata can lead to archive software errors.Source: Technical considerations · p. 19
  • Labelmap segmentations were adopted into DICOM after conversion of these datasets; losslessly compressed labelmaps were 14 to 48 times smaller than equivalent uncompressed binary segmentations.The labelmap representation was proposed in this work, and the authors plan to use it for some future datasets.Source: Technical considerations · pp. 19–20
  • SR suits manageable numbers of heterogeneous ROIs with rich metadata, while MBSA efficiently stores large homogeneous groups but cannot describe further qualitative or categorical ROI information.Either object suits a small number of simple, homogeneous ROIs; SR interoperability depends on mutually compatible implementation subsets.Source: Technical considerations · p. 20

Limitations

The limitations arise from both DICOM object capabilities and current implementations. The paper identifies constraints on geometry, querying, provenance, rendering, and instance segmentation.

Limitations

  • The paper distinguishes limitations of DICOM objects, addressable through supplements or correction proposals, from implementation limitations, which require tooling improvements.Source: Limitations · p. 21
  • DICOM MBSAs and SRs lack complex geometry such as regions with holes, while PMs lack standardized attributes for recording the generating algorithm or software.The paper notes work to address these gaps through a proposed MBSA supplement and a proposed PM correction.Source: Limitations · p. 22
  • Without spatial querying for MBSAs or SRs, tools must retrieve all graphic data even when only a small slide region is needed, increasing latency and memory requirements.DICOMweb also lacks an API or convention for reduced-detail graphic representations, so viewers may render the same data differently.Source: Limitations · p. 22
  • Instance segmentation is impractical with SEGs because descriptive metadata grows unwieldy when many instances must be separately identified.This applies whether instances are encoded as binary masks or labelmaps.Source: Limitations · p. 22

Discussion

The authors report that DICOM can represent varied real-world pathology image-derived data and make it available alongside source images in the IDC. They also describe benefits and implementation challenges, and identify further work needed for broader adoption.

Discussion

  • DICOM represented ROIs and dense quantifications from human experts and AI tools, with both vector and raster ROI formats suited to different uses.The collections enrich 26 existing image collections; vector ROIs can support visualization and raster ROIs can support deep-learning training.Source: Discussion · p. 22
  • Harmonization required considerable effort to collect often-missing metadata and involved design decisions and technical trade-offs.The work also led to improvements to reusable highdicom and Slim tooling.Source: Discussion · p. 22
  • DICOM encoding provides unambiguous identification, linkage, and spatial alignment, structured querying, contextual and provenance information, and uniform programmatic access.It also allows pathology and radiology data to be indexed, stored, and presented together, reducing technical overhead for cross-modality studies.Source: Discussion · pp. 22–23

Conclusion

  • The five converted collections span varied content, scales, representations, and sources, supporting the authors’ conclusion that DICOM is viable for sharing pathology image-derived data.The authors also describe the collections as a reference set for further tool development and say co-location with source images in the IDC streamlines reuse.Source: Conclusion · pp. 23–24

Data Availability

  • The converted collections are accessible through the IDC portal, each with a Zenodo record, and a dashboard is available for exploring them.The paper also provides public conversion code, highdicom and Slim code, and notebooks demonstrating use of DICOM SEGs, SRs, and MBSAs.Source: Data Availability · p. 24, Code Availability · pp. 24–25

Code Availability

  • Public repositories provide conversion code, while notebooks demonstrate use of DICOM SEGs, SRs, and MBSAs from the IDC.The paper also identifies code for highdicom and the Slim viewer as publicly available.Source: Code Availability · pp. 24–25

Funding

The authors report federal funding from the National Cancer Institute and National Institutes of Health.

Funding

  • The project was funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health.The funding was under Task Order No. HHSN26110071 and Contract No. HHSN261201500003I.Source: Funding · p. 31

Supplement 1: Links to images used in Figure 2 of the main article

The supplement provides links to Figure 2 cases and a table of coded values for labeled hematological structures.

Supplement 1: Links to images used in Figure 2 of the main article

  • Figure 2 case links are provided for Pan-Cancer-Nuclei-Seg-DICOM, RMS-Mutation-Prediction-Expert-Annotations, TCGA-SBU-TIL-Maps, TCGA-GBM360, and BoneMarrowWSI-PediatricLeukemia.The supplement says the Figure 2 images were generated using the IDC’s public Slim viewer.Source: Supplement 1: Links to images used in… · p. 32
  • Table S1 maps labeled hematological structures in the BoneMarrowWSI-PediatricLeukemia collection to coded values.The source labels were translated from German to English, and clinical experts supported mapping them to suitable coded values.Source: Table S1 · p. 35
  • The English translation of each original label was used for the DICOM AnnotationGroupLabel attribute.This attribute stores the free-text label alongside the coded values described in Table S1.Source: Table S1 · p. 35
Loading 2609.14530v1…