Source-linked AI summary
Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGAN
Heran Yang, Jian Sun, Aaron Carass, Can Zhao, Junghoon Lee, Zongben Xu, Jerry Prince
TL;DR
Unpaired cycleGAN MR-to-CT synthesis lacks a direct constraint ensuring that synthetic CT preserves input-MR structure, despite the difficulty of obtaining paired scans. The paper adds MIND-based structure consistency and position-based slice selection, achieving better performance than conventional cycleGAN and results similar to paired-data cycleGAN.
Problem
Unpaired cycleGAN synthesis does not guarantee structural consistency between synthetic CT and input MR, while paired CT-MR data are difficult to obtain.
Method
The method augments cycleGAN with a MIND-based structure-consistency loss and a position-based strategy for selecting training images.
Results
The proposed method performs significantly better than conventional cycleGAN across all reported metrics and produces similar results to cycleGAN trained with paired data.
Takeaways & Limitations
Structure-constrained unpaired training generates better synthetic brain CT images than conventional cycleGAN while approaching paired-data cycleGAN results.
Abstract
from arXiv · showhide
The cycleGAN is becoming an influential method in medical image synthesis. However, due to a lack of direct constraints between input and synthetic images, the cycleGAN cannot guarantee structural consistency between these two images, and such consistency is of extreme importance in medical imaging. To overcome this, we propose a structure-constrained cycleGAN for brain MR-to-CT synthesis using unpaired data that defines an extra structure-consistency loss based on the modality independent neighborhood descriptor to constrain structural consistency. Additionally, we use a position-based selection strategy for selecting training images instead of a completely random selection scheme. Experimental results on synthesizing CT images from brain MR images demonstrate that our method is better than the conventional cycleGAN and approximates the cycleGAN trained with paired data.
1 Introduction
MR-to-CT synthesis can avoid separate CT scans, but unpaired cycleGAN training does not guarantee structural consistency between synthetic CT and input MR images. The proposed structure-constrained cycleGAN adds structural guidance and achieves better results than conventional cycleGAN while approximating paired-data training.
- Motivation: MR images lack electron-density information needed for MR-based radiotherapy planning and PET/MR attenuation correction, motivating CT synthesis from MR.Separate CT scanning is time-consuming, costly, potentially harmful, and requires accurate MR/CT registration.
- Motivation: Existing atlas-based approaches rely on many paired CT and MR images, which are difficult to obtain for specific MR tissue contrasts.Unpaired cycleGAN methods relax this paired-data requirement using adversarial and cycle-consistency losses.
- Problem: Cycle consistency can preserve the reconstructed MR image while allowing the synthetic CT structure to differ from the ground-truth, especially in the skull region.This exposes the lack of a direct structural constraint between input and synthetic images.
- Contribution: The proposed structure-constrained cycleGAN adds a structure-consistency loss based on the modality independent neighborhood descriptor and uses position-based training-image selection.The method is illustrated with cross-domain generators trained using adversarial, cycle-consistency, and structure-consistency losses.
- Results: The method achieves significantly better results than conventional cycleGAN and approximates a cycleGAN trained with paired data.The reported comparison concerns brain MR-to-CT synthesis using unpaired data.
2 Method
The method combines adversarial and cycle-consistency losses with a MIND-based structure-consistency loss to synthesize structurally aligned CT from unpaired MR data. It also uses position-based slice selection and a four-network CycleGAN architecture.
- Network and objective: Two generators map between MR and CT, while two discriminators distinguish real from synthetic images in each domain.The generators are GCT and GMR; the discriminators are DCT and DMR.
- Network and objective: Adversarial losses match synthetic-image distributions to the target CT or MR domains, and cycle consistency keeps reconstructed images close to their inputs.The cycle loss constrains GCT(GMR(ICT)) and GMR(GCT(IMR)) to reproduce their respective inputs.
- MIND structure feature: MIND represents local structure by comparing each image patch with neighboring patches in a non-local region, enabling comparisons across MR and CT domains.The descriptor is an |Rnl|-length vector, with components based on patch distances and local variance.
- MIND structure feature: The MIND distance is implemented with convolutional operations so the structural feature and its gradients can be computed efficiently in the network.An all-one or Gaussian kernel is applied to translated-image differences within each patch.
- MIND structure feature: The structure-consistency loss enforces voxel-wise similarity between MIND features from synthetic images and their input images.The implementation uses a 9 × 9 non-local region, a 7 × 7 patch, and a Gaussian kernel with σ = 2.
- Position-based selection: Position-based selection aligns MR and CT slices by relative volume position, with a random offset constrained to [−5, 5].The strategy avoids pairing slices from substantially different anatomical positions and is intended to strengthen discriminator training.
3 Experiments
The experiments evaluate unpaired brain MR-to-CT synthesis on clinical data using quantitative and visual comparisons against conventional, paired-data, and position-based-selection CycleGAN variants.
- Data set: The dataset includes 45 patients, split into 27 training, 3 validation, and 15 test patients.
- Methods: Experiments compare the proposed method with conventional CycleGAN, paired-data CycleGAN, and CycleGAN using position-based slice selection.
- Evaluation: Evaluation uses MAE, PSNR, SSIM, and SSIM(HG), averaged over 15 test subjects within the head region.
- Quantitative results: The proposed method significantly outperforms conventional CycleGAN on all metrics (p < 0.001) and produces results similar to paired-data CycleGAN.
- Quantitative results: Position-based selection significantly improves SSIM(HG) (p < 0.001), with marginal improvement in the other three metrics.
- Visual results: Visual comparisons show synthetic CT images and difference images for the ground truth, input MR, and four evaluated methods.
4 Conclusion
The paper proposes a structure-constrained CycleGAN for unpaired brain MR-to-CT synthesis. Its experiments show better synthetic CT images than conventional CycleGAN and results similar to paired-data CycleGAN.
- The proposed method adds MIND-based structure-consistency loss and position-based training-image selection to unpaired CycleGAN synthesis.
- The method generates better synthetic CT images than conventional CycleGAN and results similar to CycleGAN trained with paired data.