PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Symmetric Adaptation Network for Cross-modality Medical Image Segmentation

Xiaoting Han, Lei Qi, Qian Yu, Ziqi Zhou, Yefeng Zheng, Yinghuan Shi, Yang Gao

arXiv:2101.06853v1eess.IVcs.CV

TL;DR

Cross-modality distribution shifts can degrade medical image segmentation, and existing asymmetric adaptation structures may not fully eliminate large domain gaps. The paper introduces symmetric bidirectional adaptation using a shared encoder and domain-specific decoders, alongside segmentation training on original and translated images. It reports advantages over state-of-the-art methods on Cardiac and BraTS tasks.

  • Problem

    Distribution gaps between training and test modalities can sharply reduce segmentation performance, creating a challenge for cross-modality deployment.

    Source: §I · p. 1

  • Method

    A shared encoder and private decoders form two symmetric translation sub-networks for bidirectional alignment, while segmentation training uses original and translated images from both domains.

    Source: §I · p. 2

  • Results

    78.50% and 66.45% mean Dice on MRI-to-CT and CT-to-MRI, respectively, and 67.18% Dice on BraTS; the method outperforms other state-of-the-art methods.

    Source: §I · p. 2

  • Takeaways & Limitations

    The paper concludes that its network combines bidirectional feature alignment with semantic information from different-style images and shows advantages on Cardiac and BraTS datasets.

    Source: §VI · pp. 9–10

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) methods have shown their promising performance in the cross-modality medical image segmentation tasks. These typical methods usually utilize a translation network to transform images from the source domain to target domain or train the pixel-level classifier merely using translated source images and original target images. However, when there exists a large domain shift between source and target domains, we argue that this asymmetric structure could not fully eliminate the domain gap. In this paper, we present a novel deep symmetric architecture of UDA for medical image segmentation, which consists of a segmentation sub-network, and two symmetric source and target domain translation sub-networks. To be specific, based on two translation sub-networks, we introduce a bidirectional alignment scheme via a shared encoder and private decoders to simultaneously align features 1) from source to target domain and 2) from target to source domain, which helps effectively mitigate the discrepancy between domains. Furthermore, for the segmentation sub-network, we train a pixel-level classifier using not only original target images and translated source images, but also original source images and translated target images, which helps sufficiently leverage the semantic information from the images with different styles. Extensive experiments demonstrate that our method has remarkable advantages compared to the state-of-the-art methods in both cross-modality Cardiac and BraTS segmentation tasks.

I. INTRODUCTION

Distribution shifts across imaging modalities can sharply reduce segmentation performance, while pixel-wise labeling of target images is labor-intensive. The paper proposes symmetric bidirectional adaptation and reports strong results on cross-modality Cardiac and BraTS tasks.

  • Large distribution gaps from acquisition parameters or imaging modalities can cause a drastic performance drop when a trained segmentation model is deployed.Source: §I · p. 1
  • Pixel-wise manual delineation of target images is labor-intensive, motivating unsupervised adaptation from labeled source images to unlabeled target images.Source: §I · p. 1
  • Existing feature-alignment work often uses an independent network, whereas this paper reports improved performance with two symmetric translation sub-networks.Source: §I · p. 2
  • The proposed network uses two symmetric translation sub-networks with a shared encoder and private decoders to align feature distributions in both source-to-target and target-to-source directions.Source: §I · p. 2
  • The segmentation sub-network trains on original and translated images from both domains, leveraging images with different styles and their semantic information.Source: §I · p. 2
  • 78.50% and 66.45% mean Dice for MRI-to-CT and CT-to-MRI, respectively, and 67.18% Dice on BraTS; the method outperforms other state-of-the-art methods.Source: §I · p. 2

II. RELATED WORK

The related work groups UDA segmentation methods by their alignment perspective and describes limits in feature alignment, image translation, and joint approaches. The paper motivates symmetric translation sub-networks and semantic-space adversarial losses in response.

II. RELATED WORK

  • UDA segmentation methods fall into feature-alignment, image-translation, and joint-learning categories.Source: §II · p. 2

A. Feature-alignment-based Methods

  • Feature-alignment methods reduce distances between source and target features or use adversarial learning, while image-space alignment offers a compact alternative to complex, high-dimensional latent spaces.Source: A. Feature-alignment-based Methods · p. 2

B. Image-translation-based Methods

  • Image-translation methods generate target-style images carrying source labels for supervised segmentation training, but unstable GAN-based generation may destroy semantic information.Source: B. Image-translation-based Methods · p. 2

C. Joint Learning Methods

  • Joint methods combine image translation and feature alignment, but independent feature alignment can struggle with large discrepancies and generated images may lose semantic information.Source: C. Joint Learning Methods · pp. 2–3

III. METHOD

The method shares an encoder across segmentation and two domain-specific translation sub-networks. It uses bidirectional feature alignment and adversarial learning to generate cross-domain images.

III. METHOD

  • A shared encoder, two domain-specific decoders, and a classifier form the framework; the encoder and each decoder make a translation sub-network, while the encoder and classifier make the segmentation sub-network.Source: §III · p. 3, Table I · p. 3

A. Translation Sub-network for Feature Alignment

  • The two translation sub-networks translate in opposite directions and share an encoder, while separate decoders model domain-specific representations and encourage domain-invariant shared features.Source: A. Translation Sub-network for Feature… · p. 3

1) Adversarial Loss of Generated Space:

  • For source-to-target translation, the shared encoder embeds source images and the target decoder generates target-like images to fool a target discriminator; the reverse-direction loss is also formulated.Source: 1) Adversarial Loss of Generated Space · p. 3

2) Reconstruction Loss:

Reconstruction losses are used in both translation sub-networks to preserve image content and learn domain-specific style. Their weighted components contribute to the optimization of the two sub-networks.

2) Reconstruction Loss:

  • Target-domain reconstruction uses the encoder and target decoder to reconstruct target images, helping translated images preserve content while learning target style.Source: 2) Reconstruction Loss · p. 3
  • The source translation sub-network also uses reconstruction loss so its decoder learns the source-domain distribution.Source: 2) Reconstruction Loss · p. 3
  • The target and source translation sub-networks are optimized with combined losses whose reconstruction and adversarial components have weights 1.0 and 0.01, respectively.Source: 2) Reconstruction Loss · p. 3
  • The overview depicts a shared encoder with separate source and target translation paths alongside the segmentation sub-network.Source: Figure 2 · p. 4

B. Segmentation Sub-network for Semantic Mining

The segmentation sub-network mines shared semantics by training on source images in both original and translated styles, while adversarial objectives align source- and target-domain prediction maps. It also applies semantic-space adversarial learning to generated images.

B. Segmentation Sub-network for Semantic Mining

  • The sub-network uses original and translated source images for semantic learning, and raw and translated target images in its adversarial task.This combination is intended to leverage semantic information across different image styles.Source: B. Segmentation Sub-network for… · p. 4

1) Segmentation Loss:

  • Soft Dice and weighted cross-entropy address class imbalance, while translated source images are supervised using inherited source labels.The translated images retain some original content information and are used for supervised segmentation training.Source: 1) Segmentation Loss · p. 4

2) Adversarial Loss of Semantic Space:

  • Discriminators distinguish source- and target-domain prediction maps, while the segmentation sub-network attempts to fool them to mine shared semantics.The approach uses similarities in segmentation outputs, including spatial layout and local context.Source: 2) Adversarial Loss of Semantic Space · p. 4
  • The semantic-space adversarial objective also includes corresponding losses for translated source and target images.Translated source images inherit source labels.Source: 2) Adversarial Loss of Semantic Space · pp. 4–5

C. The Overall Loss

The network's overall unsupervised domain-adaptation objective is formulated with module-weighting hyperparameters. The experiment uses corresponding values of 1.0, 1.0, and 0.1.

C. The Overall Loss

D. Network Architecture

The experiments evaluate the network on cardiac and BraTS cross-modality segmentation, comparing it with other adaptation methods and testing design choices. The method combines shared-encoder translation and segmentation networks with domain-specific decoders.

D. Network Architecture

  • The network uses a shared encoder for segmentation and two translation sub-networks, with domain-specific decoders and separate discriminators.The segmentation sub-network is built on Deeplab-ResNet50.Source: D. Network Architecture · p. 5

1) Datasets:

  • Experiments use MMWHS cardiac volumes for MRI-to-CT and CT-to-MRI adaptation, and BraTS with T2 as source and FLAIR, T1, and T1CE as targets.Both datasets use an 80% training and 20% test split.Source: 1) Datasets · p. 5

2) Implementation Details:

  • The cardiac model is trained for 2 × 10^4 iterations and the BraTS model for 6 × 10^4 iterations, both with batch size 8.The reported training times are 10 hours for cardiac and 24 hours for BraTS.Source: 2) Implementation Details · p. 5

B. Evaluation Metrics

  • Cardiac evaluation uses Dice and ASD, while BraTS evaluation uses Dice and HD; higher Dice and lower ASD or HD indicate better performance.ASD measures average 3D surface distance; HD measures the extreme distance between point sets.Source: B. Evaluation Metrics · p. 5

C. Comparison with the State-of-the-art Methods

  • On the Cardiac dataset, our method outperforms CyCADA by 14.10% for MRI-to-CT and 8.95% for CT-to-MRI, and SIFA by 4.40% and 3.05%, respectively; versus DSFN, it gains 2.70% for MRI-to-CT.Source: C. Comparison with the… · p. 7
  • Joint feature-alignment and image-translation methods outperform independent methods in the reported cardiac comparison.The authors attribute this pattern to image translation helping feature alignment and vice versa.Source: Table II · p. 6, C. Comparison with the… · p. 7
  • The proposed BraTS segmentation results are closer to supervised results and better than other adaptation methods.The comparison is reported alongside Table III and Fig. 4.Source: Table III · p. 7, Figure 4 · p. 7, C. Comparison with the… · p. 8

D. Ablation Study

The ablation study examines bidirectional feature alignment in generated image space and common semantic learning in semantic space.

D. Ablation Study

  • The ablations focus on bidirectional alignment between source and target feature distributions and learning common semantic knowledge in semantic space.The study evaluates how different constraints affect the full network.Source: D. Ablation Study · p. 8

1) Effectiveness of unidirectional alignment:

The ablations compare one-way source-to-target alignment with a symmetric bidirectional design. The bidirectional setup adds Dice over the unidirectional variant in both cardiac transfer directions.

1) Effectiveness of unidirectional alignment:

  • Compared with no adaptation, unidirectional alignment raises average Dice by 42.42% in MRI to CT and 40.17% in CT to MRI.Average ASD falls by 23.09 in MRI to CT and 19.61 in CT to MRI.Source: 1) Effectiveness of unidirectional… · p. 8

2) Effectiveness of bidirectional alignment:

  • Adding a symmetric target-to-source translation sub-network raises average Dice by 6.60% in MRI to CT and 7.80% in CT to MRI versus unidirectional alignment.The two translation sub-networks share an encoder and use private decoders; the decoders learn domain-specific representations while the encoder learns domain-invariant representations.Source: 2) Effectiveness of bidirectional… · p. 8

3) Effectiveness of source segmentation:

The ablations test original-source segmentation and semantic-space adversarial learning, while the discussion compares random and pretrained initialization. The paper reports stronger average Dice with pretraining but uses random initialization for method comparisons.

3) Effectiveness of source segmentation:

  • Adding original source images to segmentation raises Dice by 1.83% in MRI to CT and 0.59% in CT to MRI.The authors motivate this with possible semantic information loss in GAN-translated source images.Source: 3) Effectiveness of source segmentation · p. 8

4) Effectiveness of adversarial loss in semantic space:

  • Using original and generated images for semantic-space adversarial learning yields Dice of 78.50% in MRI to CT and 66.45% in CT to MRI.ASD is 5.92 in MRI to CT and 5.36 in CT to MRI; Fig. 5 reports intermediate performance and a test-volume boxplot, and the authors state each proposed module helps improve performance.Source: 4) Effectiveness of adversarial loss… · p. 9, Figure 6 · p. 9

V. DISCUSSION

  • The discussion reports results for different parameter settings in Fig. 6 and Table V.The cited passage does not specify those settings or their outcomes.Source: §V · p. 9, Table V · p. 9

1) Initialization of network parameters:

  • Pretrained initialization improves average Dice to 80.73% compared with random initialization, but the paper uses random initialization for fair comparison.The reused ResNet50 parameters come from layers 1–4 of an ImageNet-pretrained model.Source: 1) Initialization of network parameters · p. 9

2) Evaluation of domain-specific decoders:

The evaluations favor the proposed private-decoder and shared-discriminator design choices, and the conclusion reports advantages on Cardiac and BraTS while noting target-domain generalization can improve.

2) Evaluation of domain-specific decoders:

4) Effectiveness of the shared discriminators:

VI. CONCLUSION

  • The conclusion pairs bidirectional feature alignment with semantic use of different-style images and says target-domain generalization could improve.Source: §VI · pp. 9–10
  • The network shows significant advantages over state-of-the-art methods on both Cardiac and BraTS datasets.Source: §VI · p. 10
Loading 2101.06853v1…