PaperlayerView exact revision on arXiv

Source-linked AI summary

TotalSegmentator: robust segmentation of 104 anatomical structures in CT images

Jakob Wasserthal, Hanns-Christian Breit, Manfred T. Meyer, Maurice Pradella, Daniel Hinck, Alexander W. Sauter, Tobias Heye, Daniel Boll, Joshy Cyriac, Shan Yang, Michael Bach, Martin Segeroth

arXiv:2208.05868v2eess.IVcs.CV

TL;DR

Anatomical segmentation supports radiologic biomarker extraction and clinical planning, motivating a publicly available model for robust body-wide segmentation. The authors developed a model for 104 anatomical structures and reported high accuracy across clinical CT data, with a Dice score of 0.943.

  • Problem

    The study targets the need for a publicly available, easy-to-use model that robustly segments anatomically relevant structures throughout the body for radiologic analysis and clinical planning.

    Source: §1 · p. 2

  • Method

    The authors developed a model for 104 anatomical structures from annotated CT data and applied it to a separate 4,004-scan dataset to examine age-related volume and attenuation.

    Source: §1 · p. 2, §2.1 · p. 2, §2.2 · p. 3

  • Results

    Dice score 0.943; the model showed high accuracy and robust performance on diverse clinical CT data, outperforming other freely available segmentation tools.

    Source: §4 · p. 11

  • Takeaways & Limitations

    The model can serve as a basis for large radiologic population studies.

    Source: §4 · p. 12

    Male patients were overrepresented in the study datasets.

    Source: §4 · p. 12

Abstract

from arXiv · show

We present a deep learning segmentation model that can automatically and robustly segment all major anatomical structures in body CT images. In this retrospective study, 1204 CT examinations (from the years 2012, 2016, and 2020) were used to segment 104 anatomical structures (27 organs, 59 bones, 10 muscles, 8 vessels) relevant for use cases such as organ volumetry, disease characterization, and surgical or radiotherapy planning. The CT images were randomly sampled from routine clinical studies and thus represent a real-world dataset (different ages, pathologies, scanners, body parts, sequences, and sites). The authors trained an nnU-Net segmentation algorithm on this dataset and calculated Dice similarity coefficients (Dice) to evaluate the model's performance. The trained algorithm was applied to a second dataset of 4004 whole-body CT examinations to investigate age dependent volume and attenuation changes. The proposed model showed a high Dice score (0.943) on the test set, which included a wide range of clinical data with major pathologies. The model significantly outperformed another publicly available segmentation model on a separate dataset (Dice score, 0.932 versus 0.871, respectively). The aging study demonstrated significant correlations between age and volume and mean attenuation for a variety of organ groups (e.g., age and aortic volume; age and mean attenuation of the autochthonous dorsal musculature). The developed model enables robust and accurate segmentation of 104 anatomical structures. The annotated dataset (https://doi.org/10.5281/zenodo.6802613) and toolkit (https://www.github.com/wasserth/TotalSegmentator) are publicly available.

1. Introduction

The introduction motivates a publicly available, easy-to-use model for robust whole-body CT segmentation, addressing the limited anatomical coverage and clinical representativeness of existing models. The authors also describe applying their tool to 4004 whole-body CT scans to analyze age-related changes.

1. Introduction

  • Existing segmentation models often target single organs [7–11] and cover few structures using datasets unrepresentative of routine clinical imaging, leaving researchers to build costly models.Routine imaging varies in contrast phase, acquisition settings, and pathologies.Source: §1 · p. 2
  • Segmentation supports biomarker extraction, pathology detection, tumor-load quantification, and surgical or radiotherapy planning, but building models requires tedious annotation and technical expertise.Source: §1 · p. 2
  • The authors aimed to provide a public, easy-to-use model covering most relevant whole-body structures with robust clinical performance, and applied it to 4004 polytrauma whole-body CT scans to study age-related volume and attenuation changes.Source: §1 · p. 2

2. Materials and Methods

The methods combine a split CT training cohort with a separate standardized whole-body aging cohort, expert-reviewed annotations, nnU-Net models, and prespecified segmentation and age-association analyses.

Training dataset

  • The final 1204-series training cohort was resampled to 1.5 mm isotropic resolution and split into 1082 training, 57 validation, and 65 test patients.From 1368 sampled CT images, 164 were excluded for extremity coverage, missing sections, or ambiguous anatomy. Dice and NSD were calculated against human-approved test annotations; a supplementary comparison with the BTCV-trained nnU-Net was limited to 13 structures and used bootstrap confidence intervals and Wilcoxon signed-rank tests.Source: Training dataset · pp. 2, 3, 5, 6

Aging-study dataset

  • The aging cohort comprised 4004 contrast-enhanced whole-body CT examinations after excluding 98 of 4102 eligible polytrauma patients.All scans used the same arteriovenous split-bolus protocol; the authors assumed this made contrast-related attenuation variation minimal and attenuation values comparable.Source: Aging-study dataset · p. 3

2.2. Data Annotation

  • Annotations covered 104 structures and were developed through iterative nnU-Net predictions, physician refinement, and manual review of all 1204 examinations.Two physicians supervised annotation; retraining followed review of 5, 20, and 100 patients, and a final model was trained independently of the intermediate models to limit test-set leakage.Source: §2.2 · pp. 3–4, Figure 1b · p. 4

2.3. Model

  • The nnU-Net framework trained one model at 1.5 mm and a second at 3 mm isotropic resolution to reduce RAM and GPU requirements.Prediction runtime was measured on a workstation with an Intel Core i9 3.5GHz CPU and Nvidia GeForce RTX 3090 GPU.Source: §2.3 · p. 5

Aging study dataset

  • Age associations were assessed between age and structure volume or mean attenuation, excluding failed segmentations defined by implausibly small volume.Spearman correlation and age-quartile comparisons using Kruskal-Wallis, followed by Wilcoxon rank-sum tests with Bonferroni correction, were used; p-values below 0.0001 were considered significant.Source: Aging study dataset · p. 6

3. Results

The 1.5 mm model achieved high segmentation accuracy and outperformed a BTCV-trained nnU-Net in test-set comparisons. Results also showed failure cases, robust performance on pathologies, and age-associated changes in attenuation and volume.

3.1. Characteristics of the Study Sample

  • The 1204-image training dataset spanned varied CT protocols, kernels, sites, scanners, and pathologies; the 4004-patient aging dataset covered ages 18–100 years.The training set included images from eight sites and 16 scanners; the aging dataset was 63.5% male.Source: §3.1 · p. 6, Figure 3 · p. 7

3.4. Segmentation Evaluation

  • Dice was 0.932 vs 0.871 and NSD was 0.971 vs 0.921 for the 1.5 mm model versus BTCV-trained nnU-Net on the test set (both p<0.001).The 1.5 mm model’s overall Dice was 0.943 (95% CI [0.938, 0.947]) and NSD was 0.966 (95% CI [0.962, 0.971]).Source: §3.4 · p. 7
  • The 3 mm model scored 0.840 Dice and 0.966 NSD; on the BTCV dataset, the 1.5 mm model scored 0.849 Dice and 0.932 NSD.Its lower-resolution borders were less precise, but minor inaccuracies remained within the 3 mm distance threshold; the model also exceeded BTCV-trained nnU-Net on that dataset (Dice 0.839; NSD 0.915; both p<0.001).Source: §3.4 · p. 7

Typical Failure Cases

  • The model sometimes missed small colon or iliac-artery parts and confused neighboring vertebrae and ribs.These were described as typical failure cases despite the model’s high Dice and NSD.Source: Typical Failure Cases · p. 8, Figure 4 · p. 8

Performance on Pathologic Cases

  • The model produced robust, accurate segmentations despite pathologies that distorted, displaced, removed, or duplicated structures.Examples included broken bones, bowels displaced by inguinal hernia, splenectomy, and a transplanted kidney.Source: Performance on Pathologic Cases · p. 8, Figure 5 · p. 8

Runtime

  • Runtime, RAM, and GPU-memory requirements were assessed for both resolution models on small abdomen, medium thorax-and-abdomen, and large head-to-knee CT studies.The runtime measurements used a local workstation with an Intel Core i9 3.5 GHz CPU and Nvidia GeForce RTX 3090 GPU.Source: Runtime · p. 9, Table 1 · p. 9

Evaluation of Age-related Differences

  • Age correlated positively with aortic volume (rs = 0.64; p<0.0001) and negatively with kidney and pancreas volume (rs = −0.49; p<0.0001 each).Age also correlated negatively with iliopsoas volume (rs = −0.61) and attenuation (rs = −0.57), both p<0.0001. Attenuation was negatively correlated with age in the hips (rs = −0.61), autochthonous dorsal musculature (rs = −0.74), and gluteal muscles (rs = −0.51 to −0.79); p<0.0001 for each reported correlation.Source: Evaluation of Age-related Differences · p. 9

4. Discussion

The study presents a robust, publicly available model for segmenting 104 anatomical structures in diverse CT data and demonstrates its use in large-scale analysis of age-related organ changes. The discussion emphasizes accessibility and broad research applications while noting a sex imbalance and planned improvements.

4. Discussion

  • The tool segmented 104 anatomical structures across 1,204 CT datasets from varied scanners, acquisition settings, and contrast phases, achieving Dice 0.943 and supporting age-related organ analyses on over 4,000 examinations.It works robustly on a wide range of clinical data.Source: §4 · p. 11
  • Prior multi-structure models covered fewer relevant structures or were limited by homogeneous data, small training cohorts, or demanding hardware.Chen et al. segmented 50 structures but did not publicly release the model or dataset (20); Sundar et al. segmented 120 but required 256 GB RAM and trained on fewer than 100 people (21); Trägårdh et al. segmented 100 using 339 homogeneous samples, limiting performance across diverse orientations and contrast phases (22).Source: §4 · p. 11
  • The pretrained Python package requires less than 12 GB of RAM, runs without a GPU, and accompanies a freely downloadable dataset requiring no access request.The authors describe one-click downloading for the dataset.Source: §4 · p. 11
  • The nnU-Net-based model was chosen for accuracy across tasks, although optimization or newer architectures could improve it.Potential applications include dosimetry, organ-volume and attenuation biomarkers, tumor-load estimation with lesion detection, and pathology-model development; more than 4,500 researchers had downloaded the model.Source: §4 · p. 11
  • More than 4,000 polytrauma CT examinations enabled age-related organ-volume and attenuation analyses that were previously impractical or required substantial expert time.The discussion notes that common organ-size and development reference values typically rely on samples of only a few hundred patients, and frames population studies as a way to develop new reference values and disease-evaluation methods.Source: §4 · pp. 11–12
  • Male patients were overrepresented, and future work will add structures and expand the aging study with more patients, confounder correction, and additional correlations.The authors propose the model as a basis for large radiologic population studies.Source: §4 · p. 12

Supplemental Materials · S1: List of all segmented structures

S1 inventories segmented anatomy across visceral organs, cardiopulmonary and digestive structures, vessels, skeleton, musculature, and the urinary bladder.

S1: List of all segmented structures

S2: List of all pretrained models which were used during data annotation

The annotation used pretrained models and datasets spanning abdominal and thoracic organs, ribs, vertebrae, and lungs.

S2: List of all pretrained models which were used during data annotation

S3: More details on data annotation · Transfer of segmentations between CTs with and without contrast agent

Heart-subpart segmentations were transferred from contrast-enhanced CTs to aligned native CTs, enabling an nnU-Net trained on both image types to predict these structures across CT sequences.

Transfer of segmentations between CTs with and without contrast agent

S4: Model details

The model used modified nnU-Net augmentation and training settings, divided the standard-resolution classes across smaller models to manage memory, and used a separate configuration for the 3 mm model.

  • Mirroring was removed because it impaired left-right structure discrimination, and training increased from 1,000 to 4,000 epochs for the large dataset.Source: S4: Model details · p. 17
  • To limit memory use, the 104 classes were divided among five smaller nnU-Nets: four with 21 classes and one with 20.Source: S4: Model details · p. 17
  • The 3 mm model combined all 104 classes in one model and trained for 8,000 epochs because training took longer at low resolution.Source: S4: Model details · p. 17

Challenge” which we use as baseline

The baseline includes the portal vein and splenic vein.

Challenge” which we use as baseline

S6: Reasons for low performance on some structures

The model is highly accurate and robust for most structures, but recurring minor errors affect some structures.

S6: Reasons for low performance on some structures

S7: Details of evaluation on BTCV dataset

The supplement describes a cross-validated BTCV comparison and age-quartile analyses of volume and attenuation across segmented structures. Most listed structures showed significant age-quartile differences in both measures, while several attenuation or volume comparisons did not.

Details of evaluation on BTCV dataset

  • The 30-subject BTCV dataset was resampled to 1.5 mm isotropic resolution, then evaluated with 5-fold cross-validation to generate predictions for comparison with the proposed model.For comparison against the BTCV model on the authors’ test set, they used an ensemble of the five cross-validation models.Source: S7: Details of evaluation on BTCV… · p. 19
  • Figure S1 presents high-resolution model results per class, grouping similar classes and displaying Dice and NSD scores.Source: Figure S1 · p. 19
  • The aging-study illustration reports Spearman correlations and p-values for volume and attenuation across 104 structures, with boxplots by four age quartiles and post-hoc group differences.Age groups were Q1 < 41 years, Q2 41–59, Q3 59–78, and Q4 > 78; group comparisons used Kruskal–Wallis tests, followed by Mann–Whitney U tests, and structures below lower volume cutoffs were excluded.Source: S7: Details of evaluation on BTCV… · pp. 21, 42, Figure S3 · p. 21
  • For most listed structures, age-quartile comparisons were significant for both volume and attenuation (p < 0.0001); esophagus and colon attenuation and bladder volume were not.The nonsignificant values were p = 0.0010 and p = 0.0006 for esophagus and colon attenuation, and p = 0.0041 for bladder volume; significance used a p < 0.0001 threshold after Bonferroni correction.Source: S7: Details of evaluation on BTCV… · pp. 42–44

Related papers

Loading 2208.05868v2…