Source-linked AI summary
Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss
Qi Dou, Cheng Ouyang, Cheng Chen, Hao Chen, Pheng-Ann Heng
TL;DR
Domain shift degrades ConvNet segmentation, especially across biomedical imaging modalities with substantially different distributions and expensive target annotation. The paper introduces an unsupervised adversarial framework that adapts an MRI-trained segmenter to unpaired CT data through feature mapping and domain discrimination, achieving promising cardiac-segmentation results.
Problem
ConvNets degrade under domain shift, while cross-modality biomedical adaptation is difficult and supervised transfer learning requires expensive target-domain annotations.
Method
A dilated ConvNet uses a plug-and-play DAM to map target inputs toward source features, while a DCM discriminates domain feature distributions through adversarial learning.
Results
Extensive cardiac-segmentation experiments validated the feasibility of adapting ConvNets across radiology modalities and reported promising results.
Takeaways & Limitations
The framework supports unsupervised transfer of an MRI segmenter to unpaired CT data without labeling additional medical datasets.
Takeaways & Limitations
The approach assumes source and target label spaces are identical and that cross-modality changes are primarily low-level rather than geometric.
Abstract
from arXiv · showhide
Convolutional networks (ConvNets) have achieved great successes in various challenging vision tasks. However, the performance of ConvNets would degrade when encountering the domain shift. The domain adaptation is more significant while challenging in the field of biomedical image analysis, where cross-modality data have largely different distributions. Given that annotating the medical data is especially expensive, the supervised transfer learning approaches are not quite optimal. In this paper, we propose an unsupervised domain adaptation framework with adversarial learning for cross-modality biomedical image segmentations. Specifically, our model is based on a dilated fully convolutional network for pixel-wise prediction. Moreover, we build a plug-and-play domain adaptation module (DAM) to map the target input to features which are aligned with source domain feature space. A domain critic module (DCM) is set up for discriminating the feature space of both domains. We optimize the DAM and DCM via an adversarial loss without using any target domain label. Our proposed method is validated by adapting a ConvNet trained with MRI images to unpaired CT data for cardiac structures segmentations, and achieved very promising results.
1 Introduction
Domain shift can substantially degrade ConvNet performance, and cross-modality biomedical images pose an especially difficult adaptation problem. The paper addresses this gap with unsupervised adversarial feature adaptation from MRI to CT for cardiac segmentation.
- ConvNets can underperform when tested on related but non-identical target-domain samples.
- MRI and CT have significantly mismatched data distributions and differing anatomical appearances, making cross-modality medical adaptation especially challenging.
- Unsupervised adaptation avoids the prohibitively expensive annotation of additional biomedical target-domain data.
- The proposed framework maps target CT inputs toward the source MRI feature space using a plug-and-play DAM and distinguishes domain features with a ConvNet-based DCM.
- The framework transfers an MRI segmenter to CT through feature-level mapping and adversarial learning on unpaired MRI/CT images without target labels.
- Extensive cardiac-segmentation experiments reported promising results and validated the feasibility of radiology cross-modality domain adaptation.
2 Related Work
Prior unsupervised adaptation work mainly aligns source and target feature distributions, while medical-image studies have also used fine-tuning and other domain-generalization strategies. Target annotations remained necessary in some medical segmentation settings.
- Many unsupervised adaptation methods align source and target feature distributions by minimizing feature-space distances.
- Representative approaches minimize Maximum Mean Discrepancy or align correlations and first- and second-order activation statistics across domains.
- Medical-image adaptation studies address generalization across data acquired with different imaging protocols.
- Fine-tuning for brain lesion segmentation experimentally used a small amount of target-domain annotation.
3 Methods
The method adapts a source-domain ConvNet to cross-modality target images by replacing early layers with a plug-and-play DAM while reusing higher layers. Adversarially trained DAM and DCM align multi-level source and target feature distributions without target labels.
- Framework overview: The framework combines a standard ConvNet segmenter with a plug-and-play DAM and DCM for unsupervised cross-modality adaptation.The source segmenter provides the base mapping, while the DAM and DCM are optimized through adversarial learning.
- ConvNet segmenter: The source ConvNet uses a residual backbone with dilated residual blocks to extract features from a large receptive field while preserving spatial acuity.Its segmentation objective combines multi-class cross-entropy and Dice losses; class weighting addresses imbalance.
- Plug-and-play domain adaptation module: The DAM replaces the target-processing layers up to adaptation depth d, maps target inputs into source feature space, and reuses frozen source-model upper layers.During inference, the DAM directly replaces the early d layers of the source-trained model.
- Multi-level feature alignment: The method aligns multiple levels of source and target feature maps because semantic segmentation depends on both high-level semantics and fine patterns from early layers.The source higher-layer extractors are reused, while earlier layers are updated to conduct feature-space distribution mapping.
- Learning with adversarial loss: The DCM discriminates source and target feature distributions while adversarial optimization updates the DAM to generate source-like feature maps.The Wasserstein distance between the two feature distributions is minimized, and a Lipschitz constraint is applied to the DCM.
4 Experiment
Experiments evaluate MRI-to-CT cardiac segmentation using unpaired data, compare supervised and unsupervised settings, and analyze adaptation depth. The proposed method improves target-domain segmentation without target labels, with performance depending on anatomical structure and adaptation depth.
- 4.1 Dataset and Evaluation Metrics: The study uses unpaired MRI and CT images to segment four cardiac structures, evaluating Dice and average surface distance.The structures are ascending aorta, LA-blood, LV-blood, and LV-myo; higher Dice and lower ASD indicate better performance.
- 4.2 Experimental Settings: Experiments compare source-only MRI segmentation, target-domain training from scratch, supervised transfer learning, direct MRI-to-CT application, and unsupervised adaptation.The study also compares with a prior ConvNet heart-segmentation method and varies adaptation depth.
- 4.3 Results of Unsupervised Domain Adaptation: Seg-CT-STL achieved higher Dice and lower ASD than training from scratch, with results comparable to the prior method on most structures.This supervised setting used additional annotated target-domain data.
- 4.3 Results of Unsupervised Domain Adaptation: 14.3% average Dice across structures was obtained when the MRI-trained segmenter was directly applied to CT without adaptation.This result demonstrates the severe cross-modality domain shift affecting the source-domain model.
- 4.3 Results of Unsupervised Domain Adaptation: 43.4% higher average Dice across four cardiac structures was achieved by Seg-CT-UDA (d=21) compared with direct MRI-to-CT application.The adapted model localized cardiac structures and captured anatomical shapes; AA performance approached supervised transfer learning.
- 4.4 Ablation Study on Adaptation Depth: Among adaptation depths d={13,21,31}, d=21 achieved the highest Dice on three of four structures, while d=31 was best for LA-blood.The d=31 model also outperformed d=13 overall, indicating that adaptation depth affects cross-modality performance.
5 Conclusion
The paper presents an unsupervised adversarial framework for generalizing ConvNets across biomedical imaging modalities. Its cardiac-segmentation experiments report promising results validating the approach.
- The framework adapts ConvNets across biomedical imaging modalities by optimizing a plug-and-play DAM and DCM through adversarial learning.
- Extensive cardiac-segmentation experiments produced promising results that validated the approach's effectiveness.