PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning the Effect of Registration Hyperparameters with HyperMorph

Andrew Hoopes, Malte Hoffmann, Douglas N. Greve, Bruce Fischl, John Guttag, Adrian V. Dalca

arXiv:2203.16680v1cs.CVcs.LGeess.IV

TL;DR

Registration accuracy is highly sensitive to hyperparameters, but conventional tuning repeatedly trains models and existing automatic methods may require explicit validation objectives. HyperMorph learns hyperparameter effects with a hypernetwork that predicts registration-network weights, enabling precise continuous test-time optimization. The paper reports that one model captures a range of separately optimized baselines, reduces initialization variability by 43 to 61%, and supports efficient tuning across registration settings.

  • Problem

    Registration hyperparameters strongly affect accuracy, yet tuning them can require expensive repeated model training and explicit validation objectives that are unavailable for some tasks.

    Source: §1 · p. 2, §2.1 · p. 4, §2.2 · p. 5

  • Method

    HyperMorph trains a hypernetwork that maps input hyperparameter values to the parameters of a registration network, learning a continuous family of registration models in one model.

    Source: §3.1 · p. 7, §5 · p. 19

  • Results

    A single HyperMorph model captures the behavior of baseline models optimized at different hyperparameter values, enabling precise test-time optimization and reducing initialization variability by 43 to 61%.

    Source: §5 · pp. 19–20

  • Takeaways & Limitations

    HyperMorph enables fast, automated, high-resolution hyperparameter tuning across image pairs, datasets, anatomical regions, loss metrics, and tasks without repeatedly retraining models.

    Source: §5 · p. 20

    Automatic tuning methods remain limited when annotated validation data are unavailable or evaluation is visual or nondifferentiable, and some optimization-specific hyperparameters may require substantial HyperMorph modifications.

    Source: §2.2 · p. 5, §5 · p. 21

Abstract

from arXiv · show

We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given image pair. In both strategies, the accuracy of the resulting spatial correspondences is strongly influenced by the choice of certain hyperparameter values. However, an effective hyperparameter search consumes substantial time and human effort as it often involves training multiple models for different fixed hyperparameter values and may lead to suboptimal registration. We propose an amortized hyperparameter learning strategy to alleviate this burden by learning the impact of hyperparameters on deformation fields. We design a meta network, or hypernetwork, that predicts the parameters of a registration network for input hyperparameters, thereby comprising a single model that generates the optimal deformation field corresponding to given hyperparameter values. This strategy enables fast, high-resolution hyperparameter search at test-time, reducing the inefficiency of traditional approaches while increasing flexibility. We also demonstrate additional benefits of HyperMorph, including enhanced robustness to model initialization and the ability to rapidly identify optimal hyperparameter values specific to a dataset, image contrast, task, or even anatomical region, all without the need to retrain models. We make our code publicly available at http://hypermorph.voxelmorph.net.

1. Introduction

HyperMorph addresses costly, task-sensitive registration hyperparameter tuning by learning how hyperparameters affect deformation fields in one model. This enables continuous test-time search while improving efficiency, initialization robustness, and precision across registration settings.

1. Introduction

  • Registration accuracy depends strongly on hyperparameters that can vary across implementations, image contrasts, anatomies, datasets, and tasks.Small hyperparameter changes can substantially alter accuracy, making tuning important for developing, testing, and distributing registration methods.Source: §1 · p. 2
  • Traditional tuning repeatedly trains and evaluates separate models at selected hyperparameter values, consuming computational and human effort and risking suboptimal choices.Researchers often adapt a small search subset iteratively, while values borrowed from prior work may not suit a specific dataset or task.Source: §1 · p. 2, Figure 1 · p. 3
  • HyperMorph trains a single hypernetwork-based model that learns a continuous mapping from registration hyperparameters to registration-network parameters and deformation fields.The resulting model approximates a spectrum of traditional registration models and can be evaluated for arbitrary hyperparameter values at test time.Source: §1 · p. 2
  • Weight sharing lets HyperMorph encompass models across hyperparameter values more efficiently than training multiple registration models separately.The shared representation exploits similarity between networks with similar hyperparameters.Source: §1 · pp. 2–3
  • HyperMorph shows lower variability across random initializations than conventional registration models, reducing the need for repeated retraining.This improves robustness while further reducing the computational burden associated with initialization variability.Source: §1 · p. 3
  • A single HyperMorph model supports rapid, precise test-time optimization across loss metrics, datasets, anatomical regions, and tasks, outperforming the resolution of grid search.Hyperparameters can be selected through automatic optimization or visual evaluation over a continuous range.Source: §1 · p. 3
  • The extended work adds an alternative monolithic architecture, improves the hypernetwork design, and analyzes network size, sampling, and multi-hyperparameter learning in 3D semi-supervised training.It also contrasts the hypernetwork approach with integrating hyperparameters directly into registration networks.Source: §1 · p. 3

2. Related Work

Related work frames registration as a trade-off between alignment accuracy, deformation regularity, and computational cost, while existing hyperparameter-search methods remain expensive or require explicit validation objectives. Hypernetworks provide a basis for learning hyperparameter effects, which HyperMorph applies to deformable registration.

2.1 Image Registration

  • Classical registration optimizes a deformation field separately for each image pair, producing precise and robust alignments but often requiring substantial computation.CPU alignment of 256^3 image volumes can take tens of minutes or more, while faster GPU methods still require GPU access.Source: §2.1 · p. 4
  • Learning-based registration predicts deformation fields rapidly, using supervised, unsupervised, or semi-supervised objectives depending on available data and annotations.Semi-supervised approaches incorporate auxiliary information such as anatomical annotations into the loss.Source: §2.1 · p. 4
  • Registration methods commonly tune hyperparameters balancing image similarity and deformation regularization, with additional parameters arising from auxiliary losses and similarity metrics.Training separate models for different values can take days or weeks and consume hundreds of GPU-hours.Source: §2.1 · pp. 4–5

2.2 Hyperparameter Optimization

  • Standard hyperparameter optimization includes random, grid, sequential, Bayesian, and gradient-based methods, but these approaches generally require multiple model optimizations or expensive differentiation through training.Gradient-based methods can optimize many hyperparameters but may be highly memory intensive when reversing the training procedure.Source: §2.2 · p. 5
  • Existing automatic tuning methods require an explicit validation objective, limiting use when annotations are unavailable or evaluation is visual or nondifferentiable.They also optimize values for one validation set and do not readily adapt to a new task without retraining.Source: §2.2 · p. 5

2.3 Hypernetworks

  • Hypernetworks output the weights of a primary network and have been used for model compression, Bayesian inference, pruning, multitask learning, and generative modeling.Their end-to-end structure supports learning parameterized families of primary networks.Source: §2.3 · pp. 5–6
  • Prior hypernetwork methods use conditional networks to approximate hyperparameter effects, while HyperMorph applies this idea to learning-based deformable registration.Related work includes SMASH for architecture-conditioned parameters and methods for regularization tuning in classification and registration.Source: §2.3 · p. 6

3. Methods

HyperMorph learns how registration hyperparameters affect deformation fields through a single hypernetwork-based model, enabling continuous test-time tuning without retraining separate models. The method supports interactive or automatic optimization, alternative integrations, and registration implementations designed for flexible hyperparameter conditioning.

3.1 HyperMorph

  • HyperMorph uses a hypernetwork to estimate registration-network parameters from hyperparameters, learning a continuous family of registration models in one end-to-end training process.The hypernetwork samples hyperparameters during training and adapts the registration network’s convolutional kernels accordingly.Source: §3 · p. 6, §3.1 · pp. 6–7, Figure 2 · p. 7
  • The registration objective combines image similarity and deformation regularization, with λ balancing the terms and λsim and λreg controlling their components.The method constrains λ to [0, 1] by scaling the similarity term, while similarity metrics include MSE, NCC, or MI depending on image contrast.Source: §3.1 · pp. 7–8

3.2 Hyperparameter Tuning

  • A trained HyperMorph model supports interactive slider-based tuning or automatic validation optimization by freezing the hypernetwork and optimizing only the hyperparameters.Automatic tuning uses validation segmentations and a validation loss, whereas interactive tuning can target particular anatomical regions.Source: §3.2 · pp. 8–9

3.3 Implementation

  • The implementation uses a VoxelMorph U-Net registration network with an SVF output integrated by scaling and squaring, while a fully connected hypernetwork predicts its parameters.The base registration network has 313,507 trainable parameters, and the combined model has approximately 40.5 million parameters concentrated in the hypernetwork.Source: §3.3 · p. 9
  • Continuous hyperparameter sampling and endpoint over-sampling improve boundary behavior, while discrete hyperparameters are normalized and interpolated for test-time optimization.The experiments sample continuous values uniformly and use an endpoint fraction r=0.2; discrete values such as NCC window size are normalized to [0, 1].Source: §3.3 · pp. 9–10

3.4 Alternative Models

  • Alternative integrative models provide hyperparameters to an auxiliary network that is joined to the registration network at its input, output, or convolutional channels.The pre-integrative, post-integrative, and full-integrative variants differ in where the auxiliary network’s output enters the registration architecture.Source: §3.4 · pp. 10–11, §3.3 · p. 10

4. Experiments

Across six experiments, HyperMorph approximates separately trained registration models while reducing search cost, improving initialization robustness, and supporting task- and region-specific tuning without retraining.

4. Experiments

  • The experiments evaluate whether one HyperMorph model captures individually trained baselines across similarity metrics, datasets, contrasts, registration tasks, anatomical regions, and alternative architectures.The study uses 3D brain MRI groups spanning within-contrast, multi-contrast, and longitudinal registration, with anatomical segmentation-based metrics.Source: §4 · pp. 11–12

4.1 Experiment 1: HyperMorph Efficiency and Capacity

  • HyperMorph requires 6.1× fewer GPU-hours for one hyperparameter and 12.3× fewer for two than corresponding grid searches, while enabling arbitrarily fine test-time resolution.Its selected hyperparameters closely match grid-search optima, with maximum Dice differences of 0.06 ± 0.42 and 0.02 ± 0.09 on held-out manually labeled data.Source: §4.1 · pp. 13–14

4.2 Experiment 2: Robustness to Initialization

  • HyperMorph is more robust to initialization: Dice variability is 1.7× lower than baselines with MSE and 2.6× lower with MI, both with P < .001.The comparison retrained five HyperMorph and five baseline models across four regularization weights and 50 image pairs.Source: §4.2 · p. 15

4.3 Experiment 3: Hyperparameter-Tuning Utility

  • Optimal regularization weights vary across datasets, contrasts, tasks, and anatomical regions, and mismatched values reduce maximum Dice by up to 3.16 ± 2.14.Automatic optimization takes 12.3 ± 1.8 seconds per test pair, while interactive tuning requires 2 GB of memory.Source: §4.3 · pp. 16–17

4.4 Experiment 4: Hypernetwork Size

  • HyperMorph’s ability to match baseline accuracy increases with hypernetwork size, with the large network appropriate for learning λ in 3D registration.Despite approximately 7.6× more trainable weights than the small network, training and inference times differ by less than 0.4%.Source: §4.4 · p. 17

4.5 Experiment 5: Hyperparameter Sampling

  • Endpoint oversampling improves accuracy at λ ∈ {0, 1} but reduces accuracy across the full range; r = 0.2 provides the most consistent matching overall.With r = 0.0, deviations from baseline Dice are 0.08 ± 0.26 at λ∗ and 2.96 ± 1.57 at endpoints; with r = 0.8, they are 0.87 ± 0.40 and 0.49 ± 0.51.Source: §4.5 · pp. 17–18

4.6 Experiment 6: Alternative Models

  • None of the three input-integrative alternatives reaches the accuracy of baseline or hypernetwork-based HyperMorph models.The full-integrative model takes 1.1× longer to train, while pre- and post-integrative models take 1.8× longer than hypernetwork-based HyperMorph.Source: §4.6 · pp. 18–19

5. Discussion and Conclusion

HyperMorph replaces repeated fixed-hyperparameter training with a hypernetwork that supports continuous, test-time registration tuning. The discussion highlights computational efficiency, initialization robustness, adaptation across registration settings, and boundaries for future extensions.

Conclusion

  • HyperMorph uses one hypernetwork to predict registration-network weights across continuous hyperparameter values, enabling precise test-time optimization without repeatedly training separate models.This captures the behavior of individually optimized baseline models while allowing optima between sparse grid points.Source: §5 · pp. 19–20

Test-time adaptation

  • HyperMorph supports automatic or interactive optimization using different metrics, while regular deformation quality must still be checked when optimizing Dice alone.The authors analyze voxel-wise Jacobian determinants to assess deformation regularity.Source: §5 · p. 19

Function vs. Input Space

  • The hypernetwork approach outperformed integrative alternatives in registration quality, potentially because its convolutional filters adapt flexibly to each hyperparameter value.The fixed filters in integrative approaches may face a more difficult optimization problem, but the authors present this explanation as a hypothesis.Source: §5 · pp. 19–20

Computational efficiency

  • Weight-sharing lets HyperMorph tune hyperparameters across a continuous interval with substantially less compute and human time than sparse-grid training of multiple registration networks.The approach estimates deformation fields for arbitrary image pairs and hyperparameter values at test time.Source: §5 · p. 20

Initialization robustness

  • HyperMorph reduced variability across training runs by 43 to 61% relative to individually trained networks, indicating greater robustness to random initialization.The authors attribute this likely to weight-sharing across the hyperparameter landscape.Source: §5 · p. 20

Test-time adaptation

  • A single HyperMorph model can identify hyperparameter values for different datasets, image contrasts, tasks, and anatomical regions, supporting adaptation beyond a single global optimum.The discussion also reports region-specific λ∗ identification and suggests spatially varying regularization fields as a future direction.Source: §5 · pp. 20–21

Further work

  • HyperMorph may extend to additional regularization, dropout, and architectural hyperparameters, but architecture effects may be harder to learn and optimization parameters may require substantial modifications.The authors specifically identify learning rate and batch size as requiring likely modifications.Source: §5 · p. 21
  • For noise-weighted similarity terms, Dice sensitivity can vary sharply with σ, so non-uniform hyperparameter distributions may improve performance beyond the uniform distribution used experimentally.Figure 12 reports good performance across several noise estimates despite using p(λ) = U(0, 1).Source: §5 · p. 21, Figure 12 · p. 21

Ethical Standards

The authors report adherence to high ethical standards and state that the models used publicly available data except for FSM data.

Ethical Standards

  • The study reports high ethical standards and use of publicly available data, except for the FSM data.The authors relate the UK Biobank data use to public-health interests.Source: Ethical Standards · p. 22
Loading 2203.16680v1…