PaperlayerView exact revision on arXiv

Source-linked AI summary

CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration

Boah Kim, Dong Hwan Kim, Seong Ho Park, Jieun Kim, June-Goo Lee, Jong Chul Ye

arXiv:2008.05772v1cs.CVcs.LGeess.IVstat.ML

TL;DR

Existing image-registration methods can be fast but do not explicitly guarantee topology preservation, risking structural information loss during deformation. CycleMorph trains forward and reverse registration networks with image-level cycle consistency and extends the approach to multiscale large-volume registration. Across diverse 2D and 3D datasets, the method reports effective registration, topology preservation, and improved quantitative performance over VoxelMorph.

  • Problem

    Deep learning registration methods do not explicitly enforce topology preservation, which can lead to inaccurate registration and loss of structural information.

  • Method

    CycleMorph uses forward and reverse networks with image-level cycle consistency, plus a multiscale extension for large-volume registration.

  • Results

    CycleMorph significantly outperforms VoxelMorph across reported metrics, decreasing NMSE by 0.034, increasing SSIM by 0.140 versus initialization, and gaining 0.037 % on the Jacobian determinant metric.

  • Takeaways & Limitations

    Experiments across medical and non-medical datasets support topology-preserved deformation and effective registration for diverse image pairs.

Abstract

from arXiv · show

Image registration is a fundamental task in medical image analysis. Recently, deep learning based image registration methods have been extensively investigated due to their excellent performance despite the ultra-fast computational time. However, the existing deep learning methods still have limitation in the preservation of original topology during the deformation with registration vector fields. To address this issues, here we present a cycle-consistent deformable image registration. The cycle consistency enhances image registration performance by providing an implicit regularization to preserve topology during the deformation. The proposed method is so flexible that can be applied for both 2D and 3D registration problems for various applications, and can be easily extended to multi-scale implementation to deal with the memory issues in large volume registration. Experimental results on various datasets from medical and non-medical applications demonstrate that the proposed method provides effective and accurate registration on diverse image pairs within a few seconds. Qualitative and quantitative evaluations on deformation fields also verify the effectiveness of the cycle consistency of the proposed method.

I. INTRODUCTION

Image registration must align images whose anatomy and contrast vary, but existing methods either require extensive computation or fail to explicitly preserve topology. CycleMorph addresses this with image-level cycle consistency, multiscale registration, and evaluation across diverse 2D and 3D applications.

  • Motivation: Conventional registration methods are computationally expensive, while deep learning methods are faster but often lack explicit topology-preservation constraints.The latter can produce inaccurate registration with structural information loss.
  • Proposed approach: CycleMorph trains forward and reverse CNNs so a deformation can be applied in one direction and reversed back to the original image.The method imposes cycle consistency on images rather than directly enforcing inverse consistency on deformation fields.
  • Proposed approach: The inverse path with a cyclic constraint provides topology preservation with fewer folding problems during deformation.The paper presents this as an implicit regularization of the registration process.
  • Multiscale extension: CycleMorph uses coarse 3D registration on subsampled volumes followed by local deformation estimation to handle large volumes under GPU memory limits.The second stage improves registration accuracy after estimating large deformation at coarse scale.
  • Evaluation: Experiments cover 2D face registration, 3D brain MR registration, and multiphase 3D abdominal CECT registration for liver cancer evaluation.Qualitative and quantitative results are reported as evidence of robustness and topology-preservation efficacy.

II. RELATED WORKS

Prior registration methods balance similarity, smoothness, speed, and topology preservation imperfectly. CycleMorph builds on these approaches by enforcing consistency through image transformations rather than relying only on deformation-field constraints or expensive diffeomorphic methods.

  • Classical and diffeomorphic methods: Classical variational registration combines image similarity with deformation-field regularization, but diffeomorphic methods add topology-preserving properties at substantial computational cost.Diffeomorphic mappings are continuous, differentiable, and topology-preserving, yet their expense can limit clinical workflow use.
  • Learning-based methods: Learning-based registration predicts deformation fields rapidly after training and may be supervised or unsupervised.Supervised methods require ground-truth fields, whereas unsupervised methods learn from image-registration losses.
  • Consistency limitations: Existing unsupervised methods often omit consistency constraints, which can produce folding from degeneracy of the mapping.A diffeomorphic integration layer addresses this issue but requires additional complexity at inference.
  • Consistency limitations: Discrete deformation representations can violate exact inverse correspondence, motivating inverse-consistency regularization of forward and reverse deformation fields.Prior methods enforce this consistency directly on the deformation vector fields.
  • CycleMorph context: CycleMorph’s framework uses two registration networks and spatial transformation to align moving images with fixed images before applying its consistency mechanism.The networks estimate forward and reverse deformation fields for the two image-orderings.

III. THEORY

CycleMorph defines forward and reverse registration networks that warp paired images and then reprocess the deformed images in reversed order. Its optimization combines registration, cycle, and identity losses in an unsupervised framework.

  • Consistent image registration: CycleMorph defines GX and GY to estimate deformation fields from X to Y and from Y to X, respectively.A spatial transformation layer warps the moving image toward the fixed image using the estimated field.
  • Cycle consistency: The method reapplies the networks to the two deformed images with switched ordering to impose cycle consistency at the image level.This constrains the re-deformed image to return toward the original moving image and supports topology preservation.
  • Optimization: Training minimizes a joint objective over GX and GY that includes registration, cycle, and identity losses.The loss terms are weighted by hyper-parameters α and β and require no ground-truth deformation fields.

1) Registration Loss:

The registration loss combines image-shape similarity with deformation smoothness, while the broader framework uses cycle consistency to constrain successive deformations toward the original images.

  • 1) Registration Loss:: The registration loss evaluates dissimilarity between the deformed moving image and the fixed image while regularizing the deformation field for smoothness.It follows the traditional variational registration energy structure with similarity and smoothness terms.
  • 1) Registration Loss:: Local cross-correlation is used as the similarity function to reduce sensitivity to contrast variations.The local neighborhood uses w × w × w voxels in 3D and w × w pixels in 2D, with w = 9 in the study.
  • 2) Cycle Loss:: Cycle consistency deforms an image forward and then reverses it, imposing that the re-deformed image approximates the original image.The framework applies analogous constraints for both X and Y through successive deformations.
  • 2) Cycle Loss:: The cycle loss measures the L1 differences between each twice-deformed image and its corresponding original image.The two terms compare the re-deformed versions of X and Y against X and Y, respectively.

3) Identity Loss:

The identity constraint discourages deformation when moving and fixed inputs are identical, helping stabilize displacement-field estimation in stationary regions.

  • 3) Identity Loss:: The identity loss requires identical moving and fixed images to remain undeformed.This treats stationary image regions as fixed points during deformation.
  • 3) Identity Loss:: Maximizing local cross-correlation for identical inputs favors a trivial identity deformation rather than unnecessary displacement.The identity loss uses negative local cross-correlation terms for both networks.
  • 3) Identity Loss:: The identity constraint increases the stability of deformation vector field estimation in stationary regions.The spatial transform layer supports differentiable image warping for optimization through backpropagation.
  • C. Multiscale Image Registration: GPU memory limitations constrain training on large 3D volumes, motivating global registration followed by local registration.CycleMorph can operate on downsampled images and local patches to address this constraint.

1) Training Stage:

The multiscale training stage separately trains global and local cycle-consistent registration networks to handle large images across diverse 2D and 3D applications.

  • 1) Training Stage:: The global registration network is trained first on subsampled image pairs for large-deformation estimation.Its deformation fields are upsampled to obtain full-resolution deformed images.
  • 1) Training Stage:: The local registration network is then trained on patches from globally deformed images paired with the original fixed images.Global and local networks are trained separately within the proposed cycle-consistent framework.
  • 1) Training Stage:: At test time, global and local networks estimate deformation fields successively, but the moving image is warped only once with the refined field.This design avoids accumulating interpolation errors from two successive image deformations.
  • Experiments: The method was evaluated on face-expression images, a 3D brain MR benchmark, and liver contrast-enhanced CT registration.These experiments cover non-medical 2D images and medical 3D registration with large deformation.
  • Experiments: The face-expression dataset contains 1,608 images from 67 subjects, divided into 53, 7, and 7 participants for training, validation, and testing.It includes eight expressions and three gaze directions per subject.
  • Experiments: The OASIS-3 brain registration dataset provides 1,249 T1-weighted 3D MR images with corresponding volumetric segmentations.The scans were resampled to a 256 × 256 × 256 grid with 1mm3 isotropic voxels before cropping and normalization.

3) Multiphase Liver CT:

The multiphase liver CT setup uses 4D scans and a multiscale registration strategy to accommodate variable depth and GPU-memory limits. Evaluation combines deformation-field regularity with application-specific image and anatomical criteria.

  • The dataset contains four-phase 3D liver CT volumes at 512 × 512 × depth, with 555 training scans and 50 test scans.
  • Liver-containing slices are extracted and volumes are zero-padded above and below to align slice counts without resampling.
  • The multiscale liver model subsamples inputs from 512 × 512 × depth to 128 × 128 × depth globally, then upsamples deformation fields at inference.
  • Registration quality is assessed using Jacobian-based deformation regularity, while face, brain, and liver applications use distinct task-specific criteria.For faces, evaluation uses NMSE and SSIM against fixed targets; liver evaluation uses anatomical landmarks and tumor measurements.

1) Analysis of Face Expression Image Registration:

The evaluation framework uses application-specific annotations to quantify registration accuracy, including brain segmentation overlap and liver landmark alignment. Liver registration additionally examines tumor-size measurements for diagnostic relevance.

  • Brain registration is evaluated with Dice scores between deformed anatomical segmentation maps and fixed atlas labels across 30 selected structures.
  • Liver CT registration uses target registration error on 20 radiologist-marked anatomical and pathological points from 50 test scans.
  • Target registration error is computed as the average Euclidean distance between corresponding landmark coordinates in moving and fixed images.
  • The liver evaluation also measures differences in cancer-region major and minor lengths to assess registration from a tumor-diagnosis perspective.

4) Comparative Methods:

CycleMorph is compared with classical and deep-learning registration methods across qualitative and quantitative evaluations. The reported comparisons emphasize improved facial alignment, reverse consistency, and deformation regularity.

  • Comparative Methods: The comparison set includes Elastix, ANTs SyN, VoxelMorph, and MS-DIRNet, with VoxelMorph-1 used as the baseline network.
  • Qualitative Evaluation: CycleMorph produces facial registrations more similar to targets than VoxelMorph, especially around the eyes and mouth.
  • Qualitative Evaluation: CycleMorph deformed images can be reversed to the original source images, whereas VoxelMorph deformed images cannot be reversed in the illustrated evaluation.
  • Quantitative Evaluation: The reversed images from CycleMorph have lower NMSE and higher SSIM than those from VoxelMorph, supporting the role of cycle consistency in reducing folding problems.

B. Brain MR Image Registration

CycleMorph achieves accurate atlas-based brain MR registration with smooth deformation fields, while multiscale processing improves overlap but introduces a regularity trade-off. Results are summarized with anatomical Dice scores, Jacobian statistics, and runtime.

  • Brain MR Image Registration: CycleMorph accurately aligns brain MR images, as verified by overlaid boundaries of multiple anatomical structures and smooth deformation fields without singularities.
  • Brain MR Image Registration: CycleMorph achieves higher Dice scores than VoxelMorph across all evaluated brain structures in the reported global and multiscale comparisons.
  • Brain MR Image Registration: Global CycleMorph combines higher Dice measures with a lower percentage of non-positive Jacobian determinants than VoxelMorph.
  • Study on Local Patch Size: All multiscale methods improve global-registration Dice scores but produce deformation fields with more non-positive Jacobian values.
  • Study on Local Patch Size: Smaller local patches tend to yield higher anatomical Dice scores and better deformation regularity, leading the experiments to use 64 × 64 × 64 patches.

1) Qualitative Evaluation:

CycleMorph produces accurate, smooth multiscale registrations and improves qualitative and quantitative deformation quality while preserving topology and reducing folding.

  • Qualitative Evaluation:: Multiscale CycleMorph produces accurate registrations with smooth deformation vector fields across multiphase 3D images with different contrast.The qualitative evaluation covers all multiphase 3D images shown in Fig. 10.
  • Qualitative Evaluation:: Multiscale registration improves over global registration, with difference images showing better alignment between deformed and target images.Global registration deforms the whole source shape, whereas local registration provides local-region deformation.
  • Quantitative Evaluation:: CycleMorph significantly improves registration performance over VoxelMorph and MS-DIRNet, although its TRE is slightly higher than Elastix.Its tumor-size differences are smaller than those of the comparative methods, indicating accurate deformation even in small cancer regions.
  • Quantitative Evaluation:: Global CycleMorph registration takes about 4 seconds, while total multiscale registration takes 41 seconds on a single GPU.Deep learning-based models take less than 1 minute, compared with approximately 20 minutes for Elastix.
  • Ablation Study on Loss Function:: Removing cycle loss produces more deformation-field voxels with non-positive Jacobian determinants, whereas cycle consistency reduces folding and improves topological preservation.The ablation study evaluates TRE and the percentage of non-positive Jacobian determinants on multiphase liver CT registration.
  • Conclusion: CycleMorph is an unsupervised cycle-consistent model that extends to multiscale registration for large volumes and provides topology-preserved deformation across image pairs.The conclusion reports accurate registration for new data after training and significant performance improvement across varied datasets.

APPENDIX QUANTITATIVE EVALUATION OF MULTIPHASE LIVER CT

The multiphase liver CT evaluation used expert anatomical and pathological landmarks together with tumor-size measurements to assess registration performance.

  • APPENDIX QUANTITATIVE EVALUATION OF MULTIPHASE LIVER CT: An abdominal radiologist with 8 years of experience marked anatomical and pathological points for quantitative evaluation.The evaluation concerns multiphase liver CT registration.
  • APPENDIX QUANTITATIVE EVALUATION OF MULTIPHASE LIVER CT: The landmark protocol included portal-vein branching points, fissures, and hepatic cysts or calcifications when present.These landmarks supported quantitative assessment of the registered multiphase images.
  • APPENDIX QUANTITATIVE EVALUATION OF MULTIPHASE LIVER CT: Tumor measurements included the long and short diameters of hepatic HCCs, using the largest lesion when multiple nodules were present.The measurements were performed for hepatic HCC evaluation.
Loading 2008.05772v1…