Source-linked AI summary
Improving Cross-Site Whole-Heart Segmentation
Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao, Xin Wang, Shu Hu
TL;DR
Whole-heart segmentation must generalize across heterogeneous CT and MRI acquisition sites with limited labeled data. The paper combines site characterization with modality-routed pretrained nnU-Netv2 models and label-preserving Bias Field + Bezier augmentation. Across primary held-out-site splits, the retained recipe improves both modalities, with the largest gain in CT.
Problem
Whole-heart segmentation remains challenging when site and modality shifts alter spacing, intensity, reconstruction texture, and apparent anatomy.
Method
The pipeline uses modality-routed TotalSegmentator-initialized nnU-Netv2 models with site-motivated Bias Field + Bezier augmentation and class-wise connected-component cleanup.
Results
Across primary held-out-site splits, the retained recipe improves both modalities, with the largest gain observed for CT.
Takeaways & Limitations
Site-motivated label-preserving appearance augmentation can improve robustness when limited cardiac training data contain site-specific acquisition differences.
Takeaways & Limitations
The site-characterization analysis is specific to the available CARE-WHS training data.
Abstract
from arXiv · showhide
Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in https://github.com/Purdue-M2/Improving-Cross-Site-Whole-Heart-Segmentation
1 Introduction
Whole-heart segmentation supports quantitative cardiac assessment but must remain reliable across heterogeneous sites and modalities. The paper uses site characterization to select label-preserving appearance augmentation for cross-site generalization.
- Whole-heart segmentation enables quantitative measurements supporting disease assessment, longitudinal follow-up, and treatment-response monitoring.
- Multi-center variation in hardware, protocols, reconstruction, spacing, populations, and imaging physics creates shifts that can degrade performance on unseen settings.
- Existing generalization strategies modify initialization, training signals, structural constraints, or the training distribution at different pipeline stages.
- Generic augmentation can distort anatomy or remove discriminative detail when transformations are poorly matched to the task.
- Site characterization guides candidate augmentation routes, with held-out-site validation determining which components are retained.
- The proposed CARE-WHS pipeline combines modality-routed TotalSegmentator-initialized nnU-Netv2 models, Bias Field + Bezier augmentation, and connected-component cleanup.
2 Task and Dataset Background
CARE-WHS evaluates seven-structure whole-heart segmentation across cardiac CT and MRI acquisition sites. The study characterizes site differences and selects augmentations that expand measured variation while preserving anatomical labels.
- CARE-WHS segments seven foreground structures in cardiac CT and MRI, treated as separate domains because their imaging properties and pretrained initialization differ.
- Cross-site validation trains CT on sites A+B and evaluates on G, while MRI trains on pooled C/D and evaluates on E.
- Site characterization measures spacing, physical whole-heart volume, intensity, and frequency content, showing variation across scale, appearance, and reconstruction texture.
- The generalization strategy evaluates augmentations along measured axes while preserving underlying anatomical labels.
3 Methods
The method routes CT and MRI through separate pretrained nnU-Net branches, applies retained appearance augmentation during training, and cleans predictions using class-wise largest-connected components.
- The objective is seven-structure segmentation across unseen acquisition sites using label-preserving appearance augmentation to broaden feature variability.
- A deterministic intensity heuristic routes each input to a CT or MRI branch, whose seven-class predictions undergo class-wise largest-connected-component cleanup.
- Both modality branches use TotalSegmentator-initialized nnU-Net 3D full-resolution models fine-tuned on corresponding CARE-WHS cases.
- Bias-field augmentation applies a smooth multiplicative intensity perturbation generated from a low-frequency random field while preserving labels.
- Bezier augmentation applies a random monotone nonlinear contrast remapping through a cubic Bezier curve.
- The combined appearance transform changes image appearance without changing anatomical geometry, with modality-specific clipping and normalization.
- LCC cleanup independently partitions each predicted class into 26-connected regions and removes all but the largest component.
4 Experiments
Experiments evaluate modality-specific models and augmentations under held-out-site protocols for CT and MRI. The retained Bias Field+Bezier recipe and LCC cleanup improve segmentation performance, with especially clear gains for CT.
- Experimental setup: CT trains on sites A+B and evaluates on held-out site G, while MRI trains on C/D and evaluates on held-out site E.Final submissions are retrained on all available labeled cases for each modality.
- Experimental setup: The evaluation uses mean Dice as the primary metric and HD95 to assess boundary-distance performance.
- External baselines: 0.9135 mean Dice is achieved on CT, improving over the strongest non-final baseline at 0.8343 on the primary held-out-site split.
- External baselines: The proposed method achieves the best MRI mean Dice, with improvements in Myo, RA, RV, and LV while AO and PA remain challenging.The margin over competing methods is smaller than on CT.
- Augmented distribution: Bias+Bezier expands intensity and frequency variability while preserving each site's geometric spacing, matching its intended label-preserving appearance-augmentation role.The distribution analysis interprets the selected recipe, while Dice and HD95 determine configuration retention.
- Ablation study: Bias Field+Bezier is the strongest augmentation-only setting, and paired smooth spatial perturbation with nonlinear remapping outperforms either transform alone in the ablations.On CT, Bias Field gives the stronger single-transform gain; on MRI, the ordering is less pronounced.
- Ablation study: LCC cleanup mainly reduces HD95 by removing small disconnected components and improves final mean Dice in both modalities.Its effect is especially clear for CT, motivating retention as lightweight inference-time processing.
5 Conclusion
The paper presents a modality-routed 3D whole-heart segmentation pipeline using TotalSegmentator-initialized nnU-Netv2 models, site-motivated Bias Field + Bezier augmentation, and class-wise LCC cleanup. Across held-out-site splits, the retained recipe improved both modalities, with the largest gain for CT, while the site-characterization workflow remains specific to CARE-WHS training data.
- Method: The pipeline combines TotalSegmentator-initialized nnU-Netv2 models, Bias Field + Bezier augmentation, and class-wise LCC cleanup.Augmentation was selected by characterizing measurable site variation.
- Results: The retained recipe improved performance for both modalities across primary held-out-site splits, with the largest gain observed for CT.MRI improvements were more modest.
- Results: LCC cleanup reduced boundary outliers, particularly in CT.
- Limitations: The site-characterization analysis is specific to the available CARE-WHS training data.The broader contribution is framed as a workflow for quantifying site variation and selecting augmentation axes through held-out-site validation.
- Future Work: Future work should extend site characterization beyond input-space augmentation toward feature-space regularization.The proposed direction aims to make representations less predictive of acquisition site while preserving cardiac-anatomy prediction.