Source-linked AI summary
LISynSeg: Data-Centric Label-to-Image Synthesis for Cross-Modality Whole-Heart Segmentation
Jiacheng Wang, Ivana Isgum, Ipek Oguz
TL;DR
Cross-modality whole-heart segmentation must handle acquisition shifts and heterogeneous cardiac annotations. LISynSeg keeps nnU-Net fixed while adding cohort-calibrated label-to-image synthesis to real-image training and modeling cardiac label variability; real–synthetic training improves MRI and overall segmentation, while synthetic-only training loses context absent from labels.
Problem
Acquisition shifts and heterogeneous cardiac annotations create a cross-modality whole-heart segmentation challenge in CT and MRI.
Method
LISynSeg mixes real CT/MRI examples with cohort-calibrated label-to-image synthesis, targeted cardiac label augmentation, and relaxed distal-vessel supervision while holding nnU-Net fixed.
Results
Real–synthetic training improves MRI and overall segmentation over real-only nnU-Net, whereas synthetic-only training loses image context absent from labels.
Takeaways & Limitations
Training-data augmentation design is a practical route for cross-modality whole-heart segmentation with heterogeneous cardiac data.
Takeaways & Limitations
Qualitative AO and PA comparisons reflect agreement with released annotations rather than the challenge’s standardized vessel-length evaluation.
Abstract
from arXiv · showhide
Whole-heart segmentation (WHS) in computed tomography (CT) and magnetic resonance imaging (MRI) is affected by acquisition shifts and heterogeneous cardiac annotations. Existing WHS systems combine architectural design, transfer learning, and generic spatial or intensity augmentation. We investigate whether changes to data augmentation and training supervision can improve cross-modality WHS while the segmentation architecture is held constant. We present LISynSeg, a data-centric approach that augments real-image nnU-Net training with label-to-image synthesis. Synthetic volumes are generated from cardiac label maps using contrast and acquisition perturbations calibrated to the training cohort, then mixed with real images to retain thoracic context absent from the labels (and thus the synthesized images). We model cardiac label variation through controlled changes in myocardial wall thickness and partial supervision of uncertain vessel endpoints. On the CARE Whole-Heart benchmark, synthetic-only training performs worse than the real-image nnU-Net baseline, whereas calibrated real-synthetic training improves cross-modality segmentation without changing the architecture; the improvement is larger for MRI than for CT. The results show that modifying the training data strategy can benefit model development for heterogeneous cardiac data. Code and trained weights will be released at https://github.com/MedICL-VU/Care26_LISynSeg.
1 Introduction
CARE-WHS cross-modality segmentation is challenged by acquisition differences, missing thoracic context in label-derived images, and structure-specific annotation variability. LISynSeg addresses these issues by combining cohort-calibrated label-to-image synthesis with targeted label and supervision changes while keeping the segmentation architecture fixed.
- CARE-WHS images vary across scanners, protocols, centers, contrast, voxel spacing, orientation, and field of view.
- Great-vessel annotations vary in distal extent, with AO labels ending near the ascending segment or continuing through the arch and PA labels ending before or after bifurcation.
- Existing methods modify architectures or training procedures but do not explicitly expand cardiac label geometries or annotation extents represented in supervision.
- Label-to-image synthesis can model acquisition-driven appearance variation, but labels alone omit surrounding thoracic tissues present in real scans.
- LISynSeg mixes real CT and MRI examples with cohort-calibrated synthetic images, changes the Myo–LV boundary, and relaxes supervision near uncertain distal vessel endpoints.
- Controlled comparisons keep the segmentation architecture and inference pipeline common, isolating the effect of training-data construction.
2 Method
LISynSeg keeps the 3D full-resolution nnU-Net architecture and inference pipeline fixed while changing training-example construction and supervision. It mixes calibrated label-to-image synthesis with real images, models myocardial boundary variation, and masks uncertain distal vessel supervision.
- Label-to-Image Synthesis as Training Augmentation: LISynSeg augments real-image nnU-Net training with label-to-image synthesis while keeping preprocessing, optimization, architecture, and inference fixed.Synthetic images are generated from cardiac labels, while selected examples retain the augmented real image–label pair.
- Label-to-Image Synthesis as Training Augmentation: Synthetic images use the seven cardiac labels and independently sampled class intensities, bias fields, gamma variation, noise, and resolution profiles.The synthesis parameters are sampled for each selected example, and additional affine or elastic deformation is omitted because nnU-Net already spatially transforms the data.
- Myo–LV Boundary Exchange: Myo–LV boundary exchange explicitly thickens or thins myocardium by transferring interface voxels while preserving the combined Myo/LV extent.Thickening transfers voxels from LV to Myo, thinning transfers voxels from Myo to LV, and volume-ratio constraints bound the exchange.
- Distal Great Vessel Loss Masking: AO/PA loss masking retains supervision for the closest 90% of annotated vessel voxels and ignores distal regions selected from physical distance to the proximal reference.The mask is computed immediately before loss calculation and can be skipped when tied distances would mask more than 40% of a vessel.
- Distal Great Vessel Loss Masking: The masking operation changes which voxels contribute to Dice and cross-entropy losses without creating extended or truncated vessel labels.It is applied to both real and synthetic training examples, but not during validation or inference.
3 Experiments and Results
Experiments compare segmentation architectures, real-versus-synthetic training, cardiac label operations, and qualitative cases under controlled CT/MRI evaluation. LISynSeg’s strongest results come from combining real and synthetic images with both cardiac-specific operations, while retaining the fixed nnU-Net architecture and inference procedure.
- Data, Training, and Evaluation Protocol: The development split contains 85 training cases and 21 validation cases, including 12 CT and 9 MRI cases, using the CARE-WHS data partition.The released training set comprises 106 labeled volumes from five site groups, with 60 CT and 46 MRI scans.
- Data, Training, and Evaluation Protocol: DSC is reported separately for CT and MRI, with the overall WHS score weighted by the reference annotation volume of each structure.Each predicted label is postprocessed by retaining its largest connected component before evaluation.
- Segmentation Model and Training Strategy Comparison: nnU-Net achieved the highest DSC among the four real-image models on the evaluated development split, while UNETR and SwinUNETR did not outperform it.This comparison concerns the specific implementations and settings tested.
- Segmentation Model and Training Strategy Comparison: Synthetic-only nnU-Net training reduced MRI DSC by 5.87 percentage points relative to real-only training, whereas real–synthetic training increased MRI and overall DSC by 0.22 and 0.05 points.CT DSC decreased by 0.14 percentage points for real–synthetic training; the full LISynSeg setting achieved the highest DSC in CT, MRI, and overall.
- Ablation of Cardiac Label Operations: Enabling both the Myo–LV boundary operation and AO/PA loss masking produced the highest numerical DSC in CT, MRI, and overall, with small positive differences from the base setting.The Myo–LV operation alone reduced MRI DSC by 2.40 percentage points, while AO/PA masking alone remained close to the base setting.
- Qualitative Comparison: In representative cases, LISynSeg improved the CT Myo–LV interface and reduced LV under-segmentation, while MRI examples showed closer agreement around AO and PA extents.For the CT case, LV Dice rose from 0.83 to 0.86 and Myo Dice from 0.75 to 0.79; MRI AO Dice rose from 0.73 to 0.75 and PA Dice from 0.67 to 0.71.
- Qualitative Comparison: The qualitative vessel examples compare agreement with released AO and PA annotations, whose distal endpoints vary, rather than the challenge’s standardized vessel-length evaluation.This limits direct interpretation of those panels as standardized vessel-length performance.
- Segmentation Model and Training Strategy Comparison: The official ensemble achieved DSCs of 0.9360 for CT, 0.8640 for MRI, and 0.9072 for the combined track, ranking among the top five on the combined track at writing.These values describe the submitted ensemble rather than the isolated contributions in Tables 1 and 2.
4 Discussion
LISynSeg combines calibrated label-to-image synthesis with cardiac label augmentation and loss masking while keeping nnU-Net unchanged. The approach improves cross-modality WHS, with larger gains for MRI and closer agreement around great-vessel annotations.
- Discussion: LISynSeg combines cohort-calibrated label-to-image synthesis with cardiac label augmentation and loss masking without modifying the segmentation architecture.The cardiac operations target myocardial boundaries and distal great-vessel supervision.
- Discussion: The improvement is larger for MRI than for CT, and the authors associate this pattern with the usefulness of added contrast and acquisition variability for MRI.Improvements are also observed for the great vessels, particularly around branches and distal extents.
- Discussion: CARE-WHS has greater cross-center variation in scan coverage and AO/PA annotation extent than the predecessor MMWHS setting.Because distal vessel endpoints lack a consistently visible anatomical boundary, image randomization alone cannot account for all label variation.
5 Conclusion
The conclusion identifies training-data construction as a practical route for cross-modality whole-heart segmentation. LISynSeg combines real and synthetic training with cardiac-specific operations while keeping nnU-Net fixed.
- Conclusion: Real–synthetic training improves MRI and overall segmentation over real-only nnU-Net, whereas synthetic-only training loses image context absent from the labels.The method therefore uses synthetic examples alongside real images rather than replacing them.
- Conclusion: Cardiac boundary augmentation and AO/PA loss masking address label heterogeneity that generic deformation does not directly model.The targeted variation concerns myocardial wall thickness and great-vessel annotation extents.
- Conclusion: The results identify training-data augmentation design as a practical route for cross-modality WHS.