Source-linked AI summary
Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation
Junlin Yang, Nicha C. Dvornek, Fan Zhang, Julius Chapiro, MingDe Lin, James S. Duncan
TL;DR
Domain shifts make models trained on one imaging modality perform poorly on another, motivating unsupervised adaptation between CT and MRI. The paper disentangles shared anatomical content from modality-specific style and uses the shared representation for segmentation. It reports higher liver-segmentation DSC than a CycleGAN baseline, along with joint-domain learning and effectiveness on diverse multiphasic MRI.
Problem
Domain shifts between labeled source images and unlabeled target images, especially across CT and MRI, make cross-modality liver segmentation difficult.
Method
DADR uses disentangled representations with a shared domain-invariant content space and domain-specific style spaces, then segments content-only images.
Results
The method achieved a DSC of 0.81 versus 0.72 for CycleGAN in cross-modality liver segmentation and remained effective on diverse multiphasic MRI.
Takeaways & Limitations
The approach generalizes beyond CT-to-MRI adaptation to joint-domain learning and multimodal target domains with substantial variation.
Takeaways & Limitations
The approach assumes a shared semantic content space and modality-specific style codes for images from different domains.
Abstract
from arXiv · showhide
A deep learning model trained on some labeled data from a certain source domain generally performs poorly on data from different target domains due to domain shifts. Unsupervised domain adaptation methods address this problem by alleviating the domain shift between the labeled source data and the unlabeled target data. In this work, we achieve cross-modality domain adaptation, i.e. between CT and MRI images, via disentangled representations. Compared to learning a one-to-one mapping as the state-of-art CycleGAN, our model recovers a many-to-many mapping between domains to capture the complex cross-domain relations. It preserves semantic feature-level information by finding a shared content space instead of a direct pixelwise style transfer. Domain adaptation is achieved in two steps. First, images from each domain are embedded into two spaces, a shared domain-invariant content space and a domain-specific style space. Next, the representation in the content space is extracted to perform a task. We validated our method on a cross-modality liver segmentation task, to train a liver segmentation model on CT images that also performs well on MRI. Our method achieved Dice Similarity Coefficient (DSC) of 0.81, outperforming a CycleGAN-based method of 0.72. Moreover, our model achieved good generalization to joint-domain learning, in which unpaired data from different modalities are jointly learned to improve the segmentation performance on each individual modality. Lastly, under a multi-modal target domain with significant diversity, our approach exhibited the potential for diverse image generation and remained effective with DSC of 0.74 on multi-phasic MRI while the CycleGAN-based method performed poorly with a DSC of only 0.52.
1 Introduction
The paper addresses cross-modality domain shifts between CT and MRI for liver segmentation, where differing modality characteristics and CycleGAN's one-to-one pixelwise transfer limit adaptation. It proposes disentangled representations that preserve shared anatomical content while modeling modality-specific style and reports validation across cross-modality, joint-domain, and diverse-target settings.
- 1 Introduction: CT and MRI provide complementary clinical information but differ in contrast, cost, robustness, artifacts, and radiation exposure, motivating shared analysis methods.Both modalities may be used for hepatocellular carcinoma diagnosis and follow-up, including liver segmentation.
- 1 Introduction: CycleGAN's pixelwise style transfer may lose semantic feature information and produce insufficient output diversity because cycle consistency encourages one-to-one mappings.The paper links this limitation to difficulty representing complex target-domain distributions.
- 1 Introduction: Disentangled representations separate domain-invariant anatomical content from domain-specific modality style for CT-to-MRI adaptation.The approach assumes a many-to-many mapping between domains rather than CycleGAN's one-to-one mapping.
- 1 Introduction: The method is presented as the first medical-imaging approach to perform unsupervised segmentation adaptation through disentangled representations.The paper also evaluates joint-domain learning and adaptation to a highly varied multimodal target domain.
2 Methodology
DADR disentangles images into a shared content representation and domain-specific style representations, then uses content-only images for segmentation. Its DRLModule combines reconstruction and adversarial training, while the SegModule supports cross-domain and joint-domain learning.
- DRLModule: The DRLModule combines a VAE for in-domain reconstruction with a GAN for cross-domain translation and content-style disentanglement.Its encoders, generators, and discriminators jointly support reconstruction, cross-domain generation, and a shared content distribution.
- DRLModule: The total training objective combines reconstruction, adversarial, and latent-space reconstruction losses as Ltotal = αLrecon + βLadv + γLlatent.Latent reconstruction includes content and style reconstruction terms for both domains.
- Domain Adaptation with Content-only Images: DADR embeds images into a shared domain-invariant content space and domain-specific style spaces, enabling content-only segmentation across CT and MRI.The shared content representation is intended to preserve anatomical structure while excluding modality appearance.
- Domain Adaptation with Content-only Images: Content-only images are reconstructed without style codes, and a UNet trained on CT content representations is applied directly to MRI content representations.The same content-based framework also supports joint-domain learning by training the segmentation module with content-only images from both modalities.
- Implementation details: The implementation uses a standard UNet segmentation module, convolutional and residual encoders, style-conditioned generators with AdaIN, and convolutional discriminators.The style code has length 8 and the content feature map has dimensions 64x64x256.
3 Experiments and Results
Experiments evaluated DADR against CycleGAN and no-adaptation baselines for cross-modality segmentation, joint-domain learning, and multi-phasic MRI adaptation. DADR achieved stronger segmentation performance and remained effective on diverse target-domain data.
- 3.1 Datasets and Experimental Setup: The experiments compared DADR and CycleGAN with a UNet without domain adaptation using unpaired CT and MRI data.The setup included subject-wise cross-validation and separate experiments for pre-contrast and multi-phasic MRI.
- Experiment 1: Segmentation with Domain Adaptation: 0.81 DSC with DADR exceeded CycleGAN’s 0.72 in cross-modality liver segmentation.The comparison used content-only CT for training and content-only MR for validation, with 5-fold cross-validation.
- Experiment 1: Segmentation with Domain Adaptation: DADR trained a segmentation model on shared content representations, applying CT-trained segmentation directly to content-only MR.The shared representation removes modality appearance while retaining anatomical structure.
- Experiment 2: Joint-domain Learning: Joint-domain learning used content-only CT and MR together to produce one model that outperformed separately trained fully supervised UNets.The experiments used unpaired CT and MRI data and evaluated performance on both modalities.
- Experiment 3: Multi-modal Target Domain: DADR remained effective for multi-phasic MRI, whereas CycleGAN could not handle the target domain’s large variety.Content-only images preserved anatomical information across the multi-modal target domain.
A. Robust Domain Adaptation
DADR supports robust adaptation across joint and multi-modal domains by separating shared anatomical content from modality-specific style. It also enables diverse MRI style generation while preserving the same content structure.
- A. Robust Domain Adaptation: Joint-domain learning used content-only images from both modalities to train a single model for CT and MRI.This result is reported as outperforming two separately trained fully supervised UNets.
- A. Robust Domain Adaptation: DADR generated diverse modality styles by changing the style code while preserving anatomy through a fixed content code.Styles could be randomly sampled or encoded from a reference image.
B. Diverse Style Transfer
The model remained robust on a diverse multi-modal target domain and showed potential for diverse multi-modal style transfer.
- B. Diverse Style Transfer: The model remained robust under a multi-modal target domain with large diversity.This robustness was demonstrated in Experiment 3.
- B. Diverse Style Transfer: The disentangled representation enables a many-to-many mapping between domains.This design recovers complex cross-domain relationships rather than a single deterministic mapping.
- B. Diverse Style Transfer: The approach showed potential for diverse multi-modal style transfer.Its representation separates domain-invariant content from modality-specific style.
4 Conclusions and Discussions
The proposed disentangled-representation pipeline supports robust CT–MRI domain adaptation, joint-domain learning, and diverse multi-modal target handling.
- 4 Conclusions and Discussions: The pipeline preserves semantic feature-level information while modeling CT–MRI relations as many-to-many rather than one-to-one.The authors connect this design to robust cross-modality segmentation adaptation.
- 4 Conclusions and Discussions: The model generalized to joint-domain learning and handled multi-phasic MRI with substantial modality diversity.It also showed potential for multi-modal image generation.
- 4 Conclusions and Discussions: Task-relevant losses could further focus the content space on anatomical structures such as liver and tumor.The authors identify this as a direction for improving results.