Source-linked AI summary
Extending TotalSegmentator: Predicting Patient and Acquisition Characteristics from CT and MR Images
Jakob Wasserthal, Joshy Cyriac, Michael Bach, Kimia Mozahheb Yousefi, Minh-Son To, Máté Sik, Cédric Hémon, Thomas Weikert, Martin Segeroth
TL;DR
Missing or unreliable patient and acquisition metadata can hinder clinical, quality-control, and research workflows. This study developed one fast 3D multitask model per modality to infer these characteristics from heterogeneous CT and MR examinations, with strong core-target performance and practical archive applications.
Problem
Patient and acquisition characteristics may be missing, outdated, or unreliable in imaging archives despite their relevance to clinical decisions, quality control, and automated research pipelines.
Method
Separate 3D multitask ensembles for CT and MR predicted patient, acquisition, coverage, and image-quality characteristics directly from complete heterogeneous image volumes.
Results
The models outperformed a segmentation-derived XGBoost baseline for all four core targets in both modalities and produced outputs with short CPU runtimes.
Takeaways & Limitations
The open-source models can support metadata plausibility checks, retrospective cohort curation, protocol quality control, and pipeline routing rather than replace verified clinical measurements.
Takeaways & Limitations
Ground truths may contain missing, estimated, or erroneous values, younger patients were underrepresented, and external validation was limited to a small CT cohort.
Abstract
from arXiv · showhide
Background: Patient details and acquisition metadata are important for clinical decisions, image quality control, and automated research pipelines, but may be missing or unreliable in imaging archives. Purpose: To develop and evaluate a fast open-source model that predicts patient and acquisition characteristics directly from CT and MR images. Materials and Methods: Separate 3D ResNet-10 ensembles for CT and MR were trained on 57,291 and 43,200 clinical examinations acquired from 2011 to 2025. Both predicted weight, height, age, sex, contrast presence, vertebral coverage, and image noise. The CT model additionally predicted scanner manufacturer, tube voltage, tube current, convolution kernel, and post-injection time; the MR model predicted sequence class. Performance was evaluated on internal CT (n=501) and MR (n=636) test sets and an external CT dataset (n=54). Results: Internal CT MAEs were 3.90 kg, 3.68 cm, and 4.42 years for weight, height, and age, with sex F1=0.990; corresponding MR results were 4.34 kg, 4.62 cm, 7.13 years, and F1=0.970. The CNN outperformed a segmentation-derived XGBoost baseline for all four core targets in both modalities (adjusted P<=.042). F1 scores were 0.963 for CT contrast, 0.953 for MR sequence, and 0.823 for MR contrast. External CT MAEs were 4.45 kg, 4.05 cm, and 5.17 years, with sex F1=0.971. CPU inference required 20 seconds for CT and 12 seconds for MR. Conclusion: One 3D multitask model per modality can rapidly recover patient and acquisition characteristics from heterogeneous CT and MR examinations. Models are available in TotalSegmentator: https://github.com/wasserth/TotalSegmentator
1 Introduction
Patient and acquisition metadata influence clinical decisions, quality control, and downstream imaging pipelines, yet may be missing or unreliable. The study therefore develops one fast multitask CNN per modality to recover these properties from CT and MR volumes.
- Patient weight, height, age, and sex affect contrast administration, dosing, radiation optimization, and risk stratification, but may be missing or outdated in routine metadata.
- Acquisition characteristics influence CT dose, image appearance, quantitative reproducibility, enhancement-phase identification, MR pipeline selection, and anatomical-coverage routing.
- Prior systems estimated individual body characteristics or classified selected acquisition properties, but did not establish one open-source tool jointly covering both CT and MR volumes.
- The study developed and evaluated one fast 3D multitask CNN per modality, comparing core outputs with a segmentation-derived XGBoost baseline and testing external CT generalization.
2 Materials and Methods
The study trained separate five-fold 3D multitask CNN ensembles on large clinical CT and MR cohorts to predict patient, acquisition, coverage, noise, and derived characteristics. Labels came mainly from DICOM metadata and automated procedures, with preprocessing and statistical evaluation defined across internal and external datasets.
- Cohorts: Training included 57,291 CT examinations from 34,257 patients and 43,200 MR examinations from 29,073 patients, with internal and external evaluation cohorts.Internal tests contained 501 CT and 636 MR examinations; external evaluation used 54 CT examinations.
- Targets: Both models predicted weight, height, age, sex, contrast presence, vertebral boundaries, and noise; CT additionally predicted scanner and protocol properties, while MR predicted sequence class.
- Labels: Core and acquisition labels were obtained from DICOM metadata, with contrast, vertebral boundaries, convolution kernels, and noise targets derived or encoded using specified rules.
- Model: Each modality used a 3D ResNet-10 with shared outputs and jointly optimized standardized continuous and categorical targets; five-fold predictions were averaged after training.
- Model: The complete resampled volume was processed by one modality-specific multitask model, while a segmentation-derived XGBoost baseline used region measurements and targeted the four core outputs.
- Derived measures and analysis: BMI used predicted weight divided by height in meters squared, and BSA used the Mosteller formula; performance was assessed with MAE, F1, paired tests, and Holm correction.
3 Results
The multitask CNN outperformed the segmentation-derived XGBoost baseline on all core targets in both modalities and accurately predicted several additional acquisition properties. Performance generalized to external CT data, while extended multitask outputs increased CT age error and runtime remained short.
- Internal test sets: The CNN had lower error than XGBoost for every core regression target in both modalities and higher sex classification performance.CT MAEs were 3.90 kg, 3.68 cm, and 4.42 years; MR MAEs were 4.34 kg, 4.62 cm, and 7.13 years, with sex F1 values of 0.990 and 0.970.
- Additional targets: CT contrast F1 was 0.963 and manufacturer micro-F1 was 0.988, while MR sequence micro-F1 was 0.953 and MR contrast F1 was 0.823.
- Multitask comparisons: Adding extended targets preserved CT weight, height, and sex performance but increased age MAE from 4.02 to 4.42 years.The combined core model did not differ significantly from separate target-specific models for the four core targets.
- External CT test set: External full-coverage CT produced MAEs of 4.45 kg, 4.05 cm, and 5.17 years, with sex F1 of 0.971; coverage-specific comparisons varied by target.
- Runtime: Five-fold CPU inference required 20 seconds for CT and 12 seconds for MR, with all modality-specific outputs produced in the same forward passes.
4 Discussion
The study presents open-source multitask models that jointly recover patient, acquisition, coverage, and image-quality characteristics from heterogeneous CT and MR volumes. Results support practical metadata and routing uses, while also showing coverage-dependent performance and important validation boundaries.
- One open-source multitask model per modality predicts patient, acquisition, coverage, and image-quality characteristics from complete CT and MR volumes.
- External CT performance varied with anatomical coverage, and the authors recommend returning predicted coverage alongside body-statistic estimates.Full coverage produced balanced performance, abdomen-pelvis images the lowest weight error, and thorax-only images the highest.
- Outputs are intended for metadata plausibility checks, cohort curation, protocol quality control, and pipeline routing rather than replacing verified clinical measurements.Direct measurement remains necessary when errors could affect high-risk dosing or ventilator settings.
- Ground truths may be missing, estimated, or erroneous; kernel and noise targets lack universal clinical units; younger patients and additional targets require broader validation.External validation was limited to a relatively small CT cohort, with independent MR and multicenter testing still needed.
Supplementary Material
Supplementary procedures define automated targets for image noise, vertebral coverage, CT kernel sharpness, and MR sequence encoding. The procedures use tissue patches, segmentation-derived regions, and ordinal label rules tailored to each modality.
- Noise-label generation: Noise labels were generated from spatially distributed 10-mm three-dimensional patches sampled within segmented tissue regions after mask erosion and affine intensity-trend removal.
- CT noise: CT regional noise remained in image-intensity units, while the quality-control summary used the median across valid tissue regions and required at least two regions.The CT target was the median across regions of each region’s 75th-percentile patch-noise estimate.
- MR noise: MR residual patch noise was normalized by median patch signal, filtered at signal-to-noise ratio 3, pooled across tissues, and required at least 10 valid patches.The pooled 75th percentile was multiplied by 100 and then by 1.5 during export.
- Coverage and categorical labels: Visible vertebrae were identified from C1 through L5 using segmented volumes exceeding 100 voxels, with the smallest and largest ordinal indices defining cranial and caudal boundaries.A value of zero denoted unavailable vertebral information.
- Coverage and categorical labels: CT convolution kernels were converted to ordinal sharpness codes using manufacturer-specific rules, while MR sequence classes were encoded in appearance-related order.