Source-linked AI summary
Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data size
Yuta Hiasa, Yoshito Otake, Masaki Takao, Takumi Matsuoka, Kazuma Takashima, Jerry L. Prince, Nobuhiko Sugano, Yoshinobu Sato
TL;DR
The paper addresses MR-to-CT synthesis for musculoskeletal imaging, where MRI has strong soft-tissue but weak bone contrast and paired registered data are limited. It extends CycleGAN with gradient consistency loss and evaluates training-data size, synthesis quality, and segmentation. Larger training sets improved synthesis and segmentation, while GC loss improved boundary-focused outcomes and selected segmentation results.
Problem
MRI has poor bone contrast, while conventional synthesis requires paired registered multimodal data; MR-to-CT synthesis could provide clearer bone boundaries and standardized intensity units.
Method
The study extends unpaired CycleGAN with gradient consistency loss to encourage edge alignment and evaluates training-data size, GC-loss inclusion, and segmentation on synthesized images.
Results
Larger training data improved synthesis accuracy and segmentation, while GC loss improved synthesis metrics and increased DICE in the gluteus minimus region.
Takeaways & Limitations
The method supports evaluating MR-to-CT synthesis for musculoskeletal images and its applicability to segmentation of synthesized images.
Takeaways & Limitations
Patients with implants were excluded, although the target THA cohort can include implants on one side.
Abstract
from arXiv · showhide
CT is commonly used in orthopedic procedures. MRI is used along with CT to identify muscle structures and diagnose osteonecrosis due to its superior soft tissue contrast. However, MRI has poor contrast for bone structures. Clearly, it would be helpful if a corresponding CT were available, as bone boundaries are more clearly seen and CT has standardized (i.e., Hounsfield) units. Therefore, we aim at MR-to-CT synthesis. The CycleGAN was successfully applied to unpaired CT and MR images of the head, these images do not have as much variation of intensity pairs as do images in the pelvic region due to the presence of joints and muscles. In this paper, we extended the CycleGAN approach by adding the gradient consistency loss to improve the accuracy at the boundaries. We conducted two experiments. To evaluate image synthesis, we investigated dependency of image synthesis accuracy on 1) the number of training data and 2) the gradient consistency loss. To demonstrate the applicability of our method, we also investigated a segmentation accuracy on synthesized images.
1 Introduction
MRI provides superior soft-tissue contrast but poor bone contrast, motivating MR-to-CT synthesis for clearer bone boundaries and standardized Hounsfield units. The study extends unpaired CycleGAN synthesis with gradient consistency and evaluates training-data size, gradient loss, and segmentation accuracy.
- MRI complements CT for muscle identification and osteonecrosis diagnosis but has poor contrast for bone structures.
- MR-to-CT synthesis could provide clearer bone boundaries and standardized Hounsfield units while avoiding additional CT radiation exposure.
- Paired, registered multimodal training data limited conventional synthesis, whereas CycleGAN enables learning from unpaired images through cycle consistency.
- MRI segmentation requires manual labels for each sequence and imaging device because MRI lacks a standard intensity unit; MR-to-CT synthesis may support modality-independent segmentation.
- The study adds gradient consistency loss and uses 302 MR and 613 CT volumes to address pelvic intensity variation and improve boundary accuracy.
2 Method
The method trains unpaired bidirectional CycleGAN generators and discriminators on MR and CT volumes, regularizing adversarial learning with cycle and gradient consistency losses. Gradient consistency explicitly targets edge alignment, while experiments vary training-data size and GC-loss inclusion.
- 2.1 Materials: The training data comprise 302 unlabeled MRI volumes, 613 unlabeled CT volumes, and 20 labeled CT volumes, with similar field-of-view but no registration.MRI volumes received N4ITK intensity inhomogeneity correction.
- 2.2 Image synthesis using CycleGAN with gradient-consistency loss: GCT translates MR to synthesized CT and GMR translates CT to synthesized MR, while DCT and DMR distinguish real from synthesized images.
- 2.2 Image synthesis using CycleGAN with gradient-consistency loss: Cycle consistency regularizes adversarial training by penalizing differences between real images and their reconstructions, discouraging arbitrary source-target permutations.
- 2.2 Image synthesis using CycleGAN with gradient-consistency loss: Gradient consistency loss compares gradients of real and synthesized images to encourage edge alignment and improve boundary accuracy.
- 2.2 Image synthesis using CycleGAN with gradient-consistency loss: The total objective balances adversarial, cycle-consistency, and gradient-consistency losses before joint generator-discriminator optimization.The reported weights are λCycle = 3 and λGC = 0.3.
3 Result
Larger training datasets improved MR-to-CT synthesis and segmentation accuracy, while gradient-consistency loss improved image similarity, boundary preservation, and selected segmentation performance.
- Image synthesis: The average MAE decreased and PSNR increased as training data size and gradient-consistency loss were incorporated.These metrics were computed between synthesized CT and registered ground-truth CT volumes.
- Image synthesis: Gradient-consistency loss preserved shapes near the femoral head and adductor muscles in representative synthesized CT volumes.The figure examples compare translations with and without the loss.
- Segmentation: Larger training datasets significantly improved DICE for pelvis, femur, gluteus medius, and gluteus minimus segmentation.The reported significance levels were p < 0.01 for pelvis, femur, and gluteus medius, and p < 0.05 for gluteus minimus.
- Segmentation: With more than 300 training cases and gradient-consistency loss, average DICE was 0.808±0.036 for pelvis, 0.883±0.029 for femur, 0.804±0.040 for gluteus medius, and 0.669±0.054 for gluteus minimus.Gradient-consistency loss also increased DICE for the gluteus minimus region (p < 0.01).
4 Discussion and Conclusion
The study extends CycleGAN with gradient consistency loss to improve boundary accuracy and evaluates synthesis and segmentation across larger training data. It also identifies implant exclusion as a limitation and notes a gap between modality-independent and modality-dependent segmentation.
- The proposed method extends CycleGAN with gradient consistency loss to improve accuracy at image boundaries.
- Figure 7 qualitatively displays 22 estimated labels, while quantitative evaluation covers pelvis, femur, gluteus medius, and gluteus minimus.The right-most column overlays all estimated labels on the real MRI.
- Patients with implants were excluded, although the target cohort can include implants on one side during secondary-surgery planning.
- The study reports a gap between modality-independent and modality-dependent segmentation accuracy and proposes end-to-end synthesis and segmentation as a potential improvement.