PaperlayerView exact revision on arXiv

Source-linked AI summary

SynthMorph: learning contrast-invariant registration without acquired images

Malte Hoffmann, Benjamin Billot, Douglas N. Greve, Juan Eugenio Iglesias, Bruce Fischl, Adrian V. Dalca

arXiv:2004.10282v4eess.IVcs.CVq-bio.NC

TL;DR

Learning-based registration is fast but typically depends on training images with similar contrasts and geometry. SynthMorph trains on diverse synthetic label maps and images instead, achieving accurate registration across unseen MRI contrasts without acquired imaging data.

  • Problem

    Learning-based registration is limited to contrasts and geometric content resembling its training data, unlike classical methods that optimize each new image pair.

  • Method

    SynthMorph generates random label maps and contrast-varied synthetic images, including images synthesized from anatomy-specific maps, to train contrast-invariant registration networks without acquired imaging data.

  • Results

    SynthMorph accurately registers real images within and across MRI contrasts unseen during training, with sm-brains outperforming classical methods by at least 2.4 Dice points and vm-ncc by at least 0.6 Dice points on T1w-T1w registration.

  • Takeaways & Limitations

    SynthMorph combines rapid learning-based execution with generalization to MRI contrasts unseen during training, while synthetic-only training removes dependence on acquired intensity images.

  • Takeaways & Limitations

    The present work focuses on neuroimaging; extending the approach to other MRI sequences, body parts, or domains remains future work.

Abstract

from arXiv · show

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for diverse synthetic label maps and images that exposes networks to a wide range of variability, forcing them to learn more invariant features. This approach results in powerful networks that accurately generalize to a broad array of MRI contrasts. We present extensive experiments with a focus on 3D neuroimaging, showing that this strategy enables robust and accurate registration of arbitrary MRI contrasts even if the target contrast is not seen by the networks during training. We demonstrate registration accuracy surpassing the state of the art both within and across contrasts, using a single model. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images. Our code is available at https://w3id.org/synthmorph.

I. INTRODUCTION

Deformable MRI registration must handle substantial contrast variation, while classical methods repeatedly optimize each image pair and learning-based methods depend on training contrasts and shapes. SynthMorph addresses this dependency by training on synthetic label maps and images, enabling contrast-agnostic registration without acquired training images.

  • Motivation: MRI sequences produce dramatically different appearances for the same anatomy, making deformable registration important across acquisitions and modalities.Differential distortion from effects such as eddy currents and susceptibility artifacts can require deformable rather than rigid registration.
  • Existing approaches: Classical registration optimizes a similarity-and-regularity objective for every new image pair, with cost functions typically adapted to the image type.Learning-based methods instead predict deformation fields quickly after training but are limited by the contrasts and geometric content represented in their training data.
  • Existing approaches: Cross-contrast registration commonly uses mutual information or correlation ratio, but these metrics do not match within-contrast accuracy from NCC and MSE.The contrast dependence of similarity metrics remains a challenge for registering different MRI sequences.
  • SynthMorph: SynthMorph trains networks with synthetic label maps and corresponding images, incorporating labels into a loss independent of image contrast.The strategy synthesizes paired 3D label maps and images from noise distributions at every mini batch.
  • SynthMorph: SynthMorph registers unseen MRI contrasts with one model and can train on arbitrary noise-synthesized shapes without acquired data.The paper reports competitive performance from arbitrary shapes and stronger performance when images are synthesized from anatomical label maps.
  • Scope: The work extends an earlier conference paper with new experiments, analyses, discussion, and tests of generalization to MRI contrasts unseen during training.Its stated contribution focuses on neuroimaging while presenting a broader learning strategy.

II. METHOD

The method replaces acquired training images with generated label maps and intensity volumes, then trains a deformable registration network using contrast-independent label overlap. It also uses an invertible deformation parameterization with smoothness regularization.

  • Registration framework: A CNN hθ receives moving and fixed 3D images and outputs a deformation field φθ for deformable registration.The deformation field is learned by optimizing a loss with stochastic gradient descent.
  • Registration framework: Standard learning-based registration combines image dissimilarity between warped and fixed images with regularization that encourages smooth deformations.The weighting parameter λ controls the balance between these terms.
  • Limitation: Networks trained this way predict reasonable deformations mainly for contrasts and shapes resembling their training data.This limitation motivates SynthMorph’s contrast-invariant and anatomy-robust training strategy.
  • Synthetic data generation: SynthMorph generates paired 3D label maps from random seeds, then synthesizes paired intensity volumes conditioned on those maps and another seed.Available anatomical labels can replace randomly synthesized segmentation maps, while real intensity images remain unnecessary for training.
  • Contrast-agnostic loss: Generated contrasts and shapes expose the network to broad variability, while label overlap supplies a similarity loss independent of image contrast.This removes dependence on a contrast-specific image cost function.
  • Contrast-agnostic loss: The label-overlap formulation uses one-hot encoded labels, with voxel-wise multiplication and addition defining its overlap computation.The experiments use the soft Dice metric for this purpose.
  • Deformation regularization: The deformation uses a stationary velocity field integrated into the network to obtain an invertible-by-design diffeomorphism.The displacement field is regularized through the squared spatial gradient norm.

C. Generative Model Details

SynthMorph generates varied label maps and arbitrary-contrast image pairs, either from random shapes or available anatomical segmentations, then uses them for registration training.

  • Label-map synthesis: 26 smoothly varying noise images are randomly deformed and combined by voxelwise argmax to generate multilabel maps.Each voxel receives the label of the warped noise image with the highest intensity.
  • Label-map synthesis: Selected label maps are independently deformed with smooth diffeomorphic transformations to produce moving and fixed segmentation maps.Nearest-neighbor interpolation preserves segmentation labels during deformation.
  • Label-map synthesis: Anatomical training uses separate subject label maps, whereas random-shape training reuses one map to ensure topologically consistent correspondences.This choice controls the variability and correspondence structure presented during training.
  • Image synthesis: Synthetic images assign label-specific Gaussian intensities, then add partial-volume smoothing, spatial bias fields, normalization, and global exponentiation.The resulting moving and fixed images have arbitrary contrast while retaining the synthesized anatomy.
  • Random transforms: Random smooth velocity fields are sampled at lower resolutions, integrated, upsampled, and combined to create spatial variability in the synthesized data.Multiple resolutions are combined for more complex deformations in sm-shapes training.

D. Implementation Details

Implementation uses deliberately unrealistic synthetic variability, a 3D VoxelMorph-style U-Net, and GPU-based diffeomorphic integration for training and inference.

  • Synthetic variability: Hyperparameters are chosen to produce shapes and contrasts beyond realistic medical ranges rather than mimic a specific anatomy or MRI contrast.The stated purpose is to force features independent of contrast-specific characteristics.
  • Network architecture: The registration model is a convolutional U-Net that predicts a stationary velocity field from moving and fixed images, then integrates it into a warp.The encoder downsamples through stride-2 convolutions, while the decoder uses upsampling and skip connections.
  • Network architecture: The default network uses 256 filters, 3 × 3 × 3 convolutions, and three output filters, with width reduced to 64 for analyses and sweeps.The reduced width lowers computational and memory requirements.
  • Optimization: SVFs are integrated with a GPU scaling-and-squaring implementation using 5 steps, and models train with Adam using batch size one.Training typically continues for 4 × 10^5 iterations until synthetic-set Dice converges.
  • Synthesis implementation: The image-generation model improves efficiency with separated Gaussian convolutions and combines spatial augmentations into single deformation fields.Additional augmentation varies flips, anisotropic smoothing, multiresolution warps, and batch-level transformations.

III. EXPERIMENTS

Experiments evaluate SynthMorph across diverse brain MRI datasets, contrasts, processing conditions, and cardiac images, using acquired data only for testing and comparison.

  • Processing: Brain images are mapped to a common 160 × 160 × 192 affine space at 1 mm isotropic resolution for deformable registration.When segmentations are unavailable, brain and non-brain labels are derived with contrast-adaptive SAMSEG.
  • Evaluation design: Table II organizes test registration sets by contrast combinations, with superscripts identifying skull-stripping and across-dataset registration.The compiled sets are drawn from OASIS, HCP-A, and BIRN.
  • Processing: Cardiac frames are resampled to 256 × 256 × 112 volumes with isotropic 1-mm voxels, with manual contours transferred into the common space.The cardiac data contain thick 6–13 mm slices and approximately 1.5-mm in-plane resolution before resampling.
  • Data use: SynthMorph training uses Buckner40 label maps, while learning baselines use acquired T1w and T2w images from HCP-A, GSP, and UKBB.Validation uses held-out registration pairs from OASIS, HCP-A, and BIRN.
  • Datasets: Test sets combine OASIS, HCP-A, and BIRN brain MRI contrasts, plus multi-FA, multi-TI, and cardiac datasets not used for training or validation.The experiments therefore assess transfer across contrasts, datasets, and anatomy.

B. Baselines

The study compares SynthMorph with classical registration, image-trained VoxelMorph variants, contrast augmentation, and supervised synthetic-warp models.

  • Classical methods: ANTs uses NCC within contrast and MI across contrasts, while NiftyReg uses default NMI settings with diffeomorphic SVF integration.Classical parameters are adjusted to match the spatial scales of brain structures.
  • Learning baselines: VoxelMorph baselines are trained with NCC on 100 HCP-A T1w images or NMI on random T1w/T2w combinations exposing 9900 cross-subject pairs.The NMI model sees T1w-T1w, T1w-T2w, and T2w-T2w pairings.
  • Learning baselines: A larger-data VoxelMorph variant adds 7000 UKBB and GSP T1w images to test whether anatomical variability improves generalization.This evaluates whether the original finding that 100 images suffice holds in the authors’ implementation.
  • Learning baselines: An augmentation baseline retrains NCC-based VoxelMorph with random deformations applied to the input images.This isolates the contribution of spatial augmentation within an image-trained baseline.
  • Learning baselines: A hybrid baseline applies smoothed random lookup-table transformations to acquired T1w image pairs during similarity-loss training.The lookup table remaps intensities while preserving the set of intensity values.
  • Supervised variants: Supervised variants use known synthetic warps and train with MSE on either stationary velocity fields or deformation fields.These models test whether explicit knowledge of the synthetic ground-truth warp improves accuracy.

C. SynthMorph Variants

SynthMorph variants trained from synthetic shapes or anatomical labels generalize across MRI contrasts without acquired training images. sm-brains provides the strongest overall accuracy, while sm-shapes remains competitive despite lacking neuroanatomical information.

  • SynthMorph variants: sm-brains achieves the best accuracy across datasets without using real MR images during training.It outperforms classical methods by at least 2.4 Dice points and vm-ncc on T1w-T1w registration by at least 0.6 Dice points.
  • Cross-contrast performance: Across contrasts, sm-brains leads all other methods by up to 45.1 Dice points.It also exceeds classical methods by at least 2.9 Dice points on cross-contrast tasks.
  • SynthMorph variants: sm-shapes matches or exceeds baseline accuracy across multi-contrast tasks despite training on neither imaging data nor neuroanatomy.It slightly underperforms the best classical method for T1w-T1w registration.
  • Baseline comparison: Baseline VoxelMorph models perform well on familiar contrasts but break down on unseen contrast pairings and decline slightly without skull-stripping.Deformation augmentation produces negligible overall changes and does not restore cross-contrast generalization.
  • Surface-distance evaluation: sm-brains achieves the lowest mean symmetric surface distance for all contrasts, typically 0.7 mm or less.Within contrasts, it improves on classical methods by at least 0.06 mm.
  • Contrast augmentation: Random LUT contrast augmentation improves cross-contrast performance but reduces within-contrast Dice by 0.5–1.9 points relative to vm-ncc.sm-brains still exceeds the hybrid model by at least 2.4 points within contrasts and 4.5 points across contrasts.

F. Experiment 2: Contrast Invariance

The contrast-invariance experiment evaluates registration and feature variability as MRI contrast changes. SynthMorph maintains accuracy across increasingly different contrasts and shows lower deep-feature variability than VoxelMorph.

  • Experimental design: The experiment varies FLASH flip angles and MPRAGE inversion times across paired subjects to test robustness to new image types.The setup measures both registration accuracy and last-layer feature variability before stationary velocity-field formation.
  • Registration accuracy: SynthMorph remains largely unaffected as moving-image contrast diverges from the fixed T1w-like image.ANTs, NiftyReg, and learning baselines lose performance with increasing contrast differences.
  • Feature invariance: SynthMorph models exhibit substantially less response variability to contrast changes than other tested methods.The difference is especially pronounced in deeper network layers.

G. Experiment 3: Hyperparameter Analyses

Hyperparameter analyses show that SynthMorph performance depends on deformation, smoothing, network width, intensity augmentation, resolution, and regularization settings. The selected settings support robust registration across datasets and evolving contrasts.

  • Training variability: sm-brains performs best with low deformation strength when each iteration uses label maps from two distinct subjects.Single-segmentation sm-shapes training instead benefits from a larger deformation strength, with bv = 3 optimal.
  • Image and network settings: Random image blurring is optimal near bK ≈ 1, while increasing filters improves accuracy at the cost of longer training.The identified bias-field and gamma-augmentation settings are bB = 0.3 and σγ = 0.25.
  • Regularization: Reducing regularization lowers accuracy on smaller structures when training focuses on the largest labels.This anatomy-specific effect does not apply to sm-shapes, which trains on all synthetic labels.
  • Regularization: At λ = 1, the proportion of folding voxels is below 10^-6 at numerical precision.For λ ≥ 1, the average Jacobian determinant deviates from the ideal value 1 by less than 2 × 10^-3.

H. Experiment 4: Cine-Cardiac Application

Pretrained SynthMorph models are transferred to cine-cardiac MRI without retraining or task-specific optimization. sm-shapes reduces contour surface distance more than competing pretrained models, while the study treats this as evidence of transferability rather than superiority to dedicated cardiac methods.

  • Scope: The cardiac experiment demonstrates viability beyond neuroimaging but does not claim superiority over methods specifically developed for cardiac registration.The pretrained models assume affine input registration, which can be challenging in non-brain applications.
  • Transfer setting: SynthMorph transfers to cardiac images with substantially different content using brain-registration weights without retraining.The evaluation pairs end-systolic and end-diastolic frames from 33 subjects.
  • Cardiac results: sm-shapes reduces epicardial mean symmetric surface distance by 11.6 ± 1.5% on average and improves 85% of cardiac pairs.The corresponding reduction for vm-ncc is 4.3 ± 1.4%.
  • Qualitative behavior: sm-shapes focuses deformation on cardiac dilation while leaving most anatomy intact in representative registrations.The displacement fields illustrate matching the contracted anatomy to its late-diastolic shape.

C. Dice-Loss Sensitivity

SynthMorph’s synthetic-shape training supports registration beyond specific human anatomy, while the paper identifies affine alignment and neuroimaging scope as current boundaries.

  • C. Dice-Loss Sensitivity: Synthetic-shape training teaches the network to match shapes across contrast without learning specific human anatomical structures.The model samples random shapes of varied sizes rather than being exposed to specific neuroanatomy.
  • C. Dice-Loss Sensitivity: Competitive validation Dice scores are obtained for smaller structures, including the third and fourth ventricles, despite their exclusion from the dissimilarity loss.
  • C. Dice-Loss Sensitivity: Supervised training on synthetic shapes consistently under-performs the unsupervised counterpart, with deformation ambiguity proposed as one explanation.Different deformations can produce the same warped brain, making reliable predictor training challenging.
  • E. Further Work: The current input images require prior affine alignment for optimal results, although the released code includes an optimization-based affine solution.The included affine optimization estimates 12 parameters for each new 3D image pair in approximately 10 seconds.
  • E. Further Work: The present work focuses on neuroimaging, leaving extension to other body parts, MRI sequences, and applications beyond medical imaging for future investigation.
  • E. Further Work: The authors also plan to test whether combining simulation with real MRI scans improves generalization or biases networks against unseen contrast variations.

F. Invariant Representations

The study links SynthMorph’s robustness to contrast-invariant deep representations and examines performance across simulated MRI contrasts and cardiac motion. It also discusses how synthetic-data design and domain knowledge affect generalization.

  • F. Invariant Representations: SynthMorph’s deep layers show greater contrast invariance than conventionally trained networks, supporting robust registration across FLASH and MPRAGE image spectra.
  • F. Invariant Representations: In cine-cardiac registration, sm-shapes reduces fixed–moving frame MSD for most subjects more than sm-brains, vm-ncc, and vm-nmi.The result is achieved without optimization for the cardiac anatomy or image type.
  • F. Invariant Representations: Neuroimaging comparisons indicate substantially better SynthMorph performance when domain-specific anatomical knowledge is incorporated.
  • F. Invariant Representations: Generation hyperparameters trade broad target coverage and robustness against the network capacity needed to represent sampled variability.
  • F. Invariant Representations: The baseline comparison finds that neither augmenting nor adding data improves VoxelMorph performance.The baseline used images from 100 subjects with random axis flips, producing 79,200 cross-subject combinations.
  • F. Invariant Representations: The study establishes synthetic-only training as useful and presents feature invariance as a way to retain rapid, robust deep-learning registration while generalizing to any MRI contrast at test time.
Loading 2004.10282v4…