Source-linked AI summary
SynthRAD2023 Grand Challenge dataset: generating synthetic CT for radiotherapy
Adrian Thummerer, Erik van der Bijl, Arthur Jr Galapon, Joost JC Verhoeff, Johannes A Langendijk, Stefan Both, Cornelis, AT van den Berg, Matteo Maspero
TL;DR
Synthetic CT generation for radiotherapy needs public, multi-center data and shared evaluation because CBCT artifacts and MRI-only workflows create challenges for dose calculation. This paper presents SynthRAD2023, a heterogeneous brain and pelvis dataset pairing CT with MRI or CBCT across two synthesis tasks. The dataset supports development and fair comparison of image-synthesis methods, while lacking diagnostic or other medical information.
Problem
CBCT artifacts limit accurate dose calculation, MRI-only radiotherapy lacks tissue attenuation information, and public datasets for benchmarking synthetic CT methods were unavailable.
Method
The paper constructs a 1080-pair multi-center dataset of CT with MRI or CBCT for brain and pelvis radiotherapy, using validated preprocessing and challenge-oriented evaluation.
Results
The dataset enables development and comparison of synthetic CT approaches across brain and pelvis on heterogeneous clinical imaging from three centers.
Takeaways & Limitations
SynthRAD2023 supports fair evaluation of fully automatic medical image-synthesis methods and may help bring radiotherapy algorithms closer to clinical practice.
Takeaways & Limitations
Diagnostic or other medical information is unavailable, and preprocessing intentionally preserves variation without normalization or homogenization across patients or centers.
Abstract
from arXiv · showhide
Purpose: Medical imaging has become increasingly important in diagnosing and treating oncological patients, particularly in radiotherapy. Recent advances in synthetic computed tomography (sCT) generation have increased interest in public challenges to provide data and evaluation metrics for comparing different approaches openly. This paper describes a dataset of brain and pelvis computed tomography (CT) images with rigidly registered CBCT and MRI images to facilitate the development and evaluation of sCT generation for radiotherapy planning. Acquisition and validation methods: The dataset consists of CT, CBCT, and MRI of 540 brains and 540 pelvic radiotherapy patients from three Dutch university medical centers. Subjects' ages ranged from 3 to 93 years, with a mean age of 60. Various scanner models and acquisition settings were used across patients from the three data-providing centers. Details are available in CSV files provided with the datasets. Data format and usage notes: The data is available on Zenodo (https://doi.org/10.5281/zenodo.7260705) under the SynthRAD2023 collection. The images for each subject are available in nifti format. Potential applications: This dataset will enable the evaluation and development of image synthesis algorithms for radiotherapy purposes on a realistic multi-center dataset with varying acquisition protocols. Synthetic CT generation has numerous applications in radiation therapy, including diagnosis, treatment planning, treatment monitoring, and surgical planning.
1 Introduction
Medical imaging supports radiotherapy workflows, but CBCT artifacts limit accurate dose calculation while MRI-based workflows lack tissue attenuation information. Public, multi-center datasets and benchmarks are needed to compare synthetic CT generation methods.
- CT provides high-resolution patient geometry and electron-density conversion for radiotherapy dose calculations, while MRI and CBCT support positioning and monitoring.
- CBCT scatter noise and truncated projections produce artifacts that limit accurate dose calculations and replanning, motivating CT-quality synthetic images.
- MRI offers superior soft-tissue contrast, but MRI-only radiotherapy requires synthetic CTs to provide tissue attenuation information for treatment planning and dose calculation.
- Artificial intelligence methods perform well for MRI- or CBCT-based synthetic CT generation, yet public datasets and challenges for ground-truth benchmarking were lacking.
2.1 Overview dataset
SynthRAD2023 provides a multi-center dataset organized across MRI-to-CT and CBCT-to-CT tasks for brain and pelvis imaging. It includes diverse clinical data, standardized splits, and acquisition information for challenge development and evaluation.
- The dataset contains 1080 CT and MRI/CBCT pairs from three Dutch university medical centers, covering brain and pelvis subsets in two synthesis tasks.
- Task 1 pairs MRI with CT, whereas task 2 pairs CBCT with CT; each task includes brain and pelvis anatomies.
- Patient selection preserved clinical variation across centers, anatomies, tumor types, ages, genders, and imaging protocols, with 64% male and 36% female subjects aged 3 to 93 years.
- Each subset was divided into 180 training, 30 validation, and 60 test subjects to support deep-learning applications and the challenge.
- Images were acquired using clinically used protocols that reflect routine imaging, with institution-specific case contributions summarized in Table 1.
2.2 Task 1 (MRI-to-CT)
Task 1 supports MRI-to-CT synthesis for brain and pelvis radiotherapy cases. The MRI data use clinically acquired sequences, including contrast variation across centers, alongside corresponding planning CTs and acquisition-parameter tables.
- Task 1 provides MRI and corresponding planning CT pairs for all subjects in the brain and pelvis subsets.
- MRI sequences included T1-weighted gradient echo or inversion-prepared TFE acquisitions, with contrast-enhanced scans from centers B and C and non-contrast scans from center A.
- 2.2.1 Brain: Brain MRI acquisition parameters are documented in Table 2, while corresponding brain CT parameters are documented in Table 3.
- 2.2.2 Pelvis: Pelvis MRI acquisition parameters are documented in Table 4, while corresponding pelvis CT parameters are documented in Table 5.
2.3 Task 2 (CBCT-to-CT)
Task 2 supports CBCT-to-CT synthesis for brain and pelvis radiotherapy cases. The CBCT scans were selected from clinical image-guided radiotherapy and paired with corresponding planning CTs, with acquisition parameters reported by anatomy.
- Task 2 provides CBCT scans acquired for image-guided radiotherapy together with corresponding planning CTs for all subjects.
- 2.3.1 Brain: CBCT acquisition parameters for task 2 brain are reported in Table 6, with corresponding brain CT parameters reported in Table 7.
- 2.3.2 Pelvis: CBCT and CT acquisition parameters for task 2 pelvis are reported in Tables 8 and 9, respectively.
2.4 Preprocessing
The dataset was preprocessed to anonymize, standardize, align, and compact multimodal images while preserving realistic variation across patients and centers.
- Preprocessing converted, resampled, registered, anonymized, segmented, and cropped the imaging data for dataset use.The listed steps covered file conversion, resampling, image registration, anonymization, patient-outline segmentation, and cropping.
- Images were converted from DICOM to compressed NIfTI files, retaining full 3D volumes while reducing file size.
- Brain images used 1 x 1 x 1 mm3 spacing, whereas pelvis images used 1 x 1 x 2.5 mm3 spacing.
- Rigid registration aligned CBCT or MR images with resampled CT, while deformable registration parameters were provided but not used.
- Brain images were defaced by removing voxels inferior and anterior to the eyes, with Figure 3 illustrating the overwritten region and background values.
- Patient-outline masks were generated by thresholding and hole filling, dilated with an air margin, and used for field-of-view consistency and evaluation.
- Images were cropped to the patient-outline bounding box with a margin of 20 voxels.
2.5 Data validation
Validation emphasized preserving realistic multicenter variation while checking preprocessing and splits for bias and reviewing registration abnormalities.
- Preprocessing and train/validation/test splitting were carefully validated to avoid introducing bias, with visual overviews used for quality checks.
- Post-registration checks identified artifacts, implants, air pockets, and positioning variations, especially frequent abnormalities in pelvis data.
- Significant outliers were preferably assigned to the training set to limit their impact on validation and test phases.
3 Data format and usage notes
The dataset is organized by task and anatomy into NIfTI-based patient folders, distributed through Zenodo with accompanying software usage guidance.
- 3.1 Data structure and file formats: The dataset is divided into task 1 for MR and task 2 for CBCT, with each task separated into brain and pelvis anatomies.
- 3.1 Data structure and file formats: Patient folders use unique names encoding task, anatomy, data-providing center, and a three-digit patient identifier.
- 3.1 Data structure and file formats: Figure 4 presents the folder structure of the SynthRAD2023 dataset.
- 3.1 Data structure and file formats: The dataset is available under a CC-BY-NC 4.0 license through Zenodo, with training data released before validation and test sets.
- 3.2 Usage notes: Compressed NIfTI files can be read and modified with ITK or SimpleITK, including through Python examples in the preprocessing scripts.
- 3.2 Usage notes: The dataset’s graphical viewing workflow uses 3DSlicer for displaying NIfTI images.
4 Discussion
The dataset supports development and comparison of synthetic CT methods through heterogeneous multicenter imaging and common evaluation metrics, while retaining important data limitations.
- The collection enables evaluation and comparison of existing synthetic CT approaches and development of new methods for brain and pelvis imaging.
- Synthetic CT generation may support MRI-only radiotherapy planning, CBCT-based adaptive radiotherapy, diagnosis, and surgical planning.
- The Grand Challenge targets rapid, automated, patient-specific synthetic CT generation for radiotherapy with shared image-based and dose-based evaluation metrics.
- The dataset combines heterogeneous MRI, CBCT, and CT acquisitions from multiple centers, scanners, protocols, and patient conditions relevant to clinical practice.
- Diagnostic and other medical information is unavailable, so challenging clinical conditions are not labeled.
- Retrospective collection, clinically limited reconstruction parameters, and unavailable raw image data restrict investigation of alternative reconstruction approaches.
5 Conclusion
SynthRAD2023 supports development and evaluation of radiotherapy image-synthesis algorithms using a realistic multi-center population with varied acquisition protocols.
- 5 Conclusion: The dataset supports fair comparison and clinical translation of fully automatic image-synthesis algorithms across varied acquisition protocols.It also has applications in radiotherapy, diagnostic tasks, and surgical planning.