Source-linked AI summary
Learning Where to Trust: Reliability-Guided Expert Routing and Velocity Gating for 3D Deformable Registration
Chenyu Guo, Wenjie Dai, Yiqian Chang, Zhiyuan Zhang, Yunfan Wei, Shuhan Qi
TL;DR
Learning-based deformable registration lacks an explicit way to connect local correspondence reliability with spatial context selection and update strength. ReliMorph uses reliability-guided expert routing and residual-velocity gating with symmetric coarse-to-fine refinement; experiments report improved anatomical overlap across unimodal and multimodal benchmarks.
Problem
Learning-based registration has not sufficiently coupled spatial context selection and deformation-update strength to local correspondence reliability, especially where matching is ambiguous.
Source: §1 · p. 1Method
ReliMorph uses local correlation statistics, modality-adapted shared features, and registration-state cues to route spatial experts and gate residual velocities during symmetric coarse-to-fine refinement.
Source: §1 · p. 2Results
Experiments on IXI, OASIS, and AbdomMRCT report improved anatomical overlap across unimodal brain and multimodal registration, with ReliMorph highest in DSC and lowest in HD95 across all three benchmarks.
Source: §4.2 · p. 7, §5 · p. 8Takeaways & Limitations
ReliMorph links correspondence reliability to both context selection and update modulation, while propagating inverse-inconsistency feedback to guide finer-level refinement.
Source: §1 · p. 2Shared estimation parameters and bidirectional feedback do not guarantee exact inverse consistency or fold-free discrete transformations.
Source: §3.4 · p. 5
Abstract
from arXiv · showhide
Deformable image registration relies on accurate anatomical correspondences. However, explicitly coupling local matching reliability with spatial context selection and deformation update strength remains insufficiently explored in learning-based registration. This challenge is particularly pronounced in homogeneous, weakly textured, and severely deformed regions, where ambiguous matches may induce erroneous deformation updates. We propose ReliMorph, a reliability-routed symmetric framework for 3D deformable registration that couples correspondence assessment with spatial context selection and update strength.Local correlation distributions characterize correspondence ambiguity using shared anatomical features with modality-specific adaptation. These reliability cues guide spatial expert fusion and gate residual velocities during coarse-to-fine refinement. Updated inverse-inconsistency feedback guides routing and gating at finer pyramid levels. To our knowledge, ReliMorph is the first 3D deformable registration framework to jointly condition spatial expert routing and residual velocity gating on correspondence reliability. Extensive experiments demonstrate that ReliMorph consistently outperforms competing methods in anatomical overlap, highlighting its effectiveness across unimodal and multimodal registration tasks.
1. Introduction
Deformable registration needs precise anatomical correspondences, but learning-based methods still lack a joint way to assess local match reliability, select spatial context, and regulate deformation updates. ReliMorph couples these operations in a symmetric, coarse-to-fine framework.
1. Introduction
- ReliMorph jointly uses correspondence reliability to guide spatial expert routing and residual-velocity gating, with bidirectional consistency feedback conditioning finer-scale refinement.Its stages combine shared anatomical encoding with modality-specific adaptation, local correlation descriptors, and symmetric transformation composition.Source: §1 · p. 2
2.1. Cross-Modal Representations for Deformable Registration
- Cross-modal appearance strategies do not resolve ambiguous anatomical matches in homogeneous tissue, repeated structures, or regions with modality-dependent boundary visibility.The challenge is preserving fine distinctions while judging whether local evidence supports deformation updates.Source: §2.1 · p. 2
2.2. Coarse-to-Fine and Diffeomorphic Registration
- Coarse-to-fine estimation and geometric regularity do not ensure anatomical correctness when earlier errors alter the correspondence evidence available at later stages.The remaining challenge is to use local reliability and bidirectional consistency feedback to guide updates across scales.Source: §2.2 · p. 2
2.3. Reliability-Aware Matching and Adaptive Routing
- Uncertainty weighting does not specify which spatial context to use or how strongly to update deformation, while adaptive kernels and expert weights do not establish correspondence reliability.The open challenge is explicitly connecting reliability assessment to context selection and update modulation in cross-modal 3D registration.Source: §2.3 · pp. 2–3
3. Methodology
ReliMorph estimates bidirectional 3D deformations through shared, modality-adapted features and reliability-conditioned correspondence modeling. Its coarse-to-fine refinement integrates residual velocities and trains without segmentation supervision.
3.1. Overview of ReliMorph
- ReliMorph estimates dense displacement fields in both directions, combining shared encoding, modality-specific adaptation, reliability-conditioned deformation estimation, and bidirectional coarse-to-fine refinement.The forward field resamples the moving image onto the fixed-image grid, with the backward field defined analogously.Source: §3.1 · p. 3
3.2. Shared-Anatomy Modality-Residual Encoding
- A shared encoder processes both images while local-mean residuals add intensity variation, and zero-initialized modality-conditioned modulation adapts shared features without separate modality encoders.The local residual uses a 5 × 5 × 5 average-pooling kernel.Source: §3.2 · pp. 3–4
3.3. Reliability-Routed Symmetric Correspondence Modeling
- Local correlation distributions provide reliability cues through normalized entropy, peak probability, top-response margin, and expected offset.The resulting correspondence descriptor combines these statistics with the matching representation.Source: §3.3 · p. 4
- Correspondence and registration-state descriptors condition voxel-wise expert fusion and confidence-gated residual velocities, with a positive gate floor limiting velocity attenuation.The experts provide local-detail, axial-context, and dilated-context processing; the gate floor is cmin = 0.25.Source: §3.3 · pp. 4–5, Figure 3 · p. 4
3.4. Symmetric Coarse-to-Fine Diffeomorphic Refinement
- Bidirectional residual transformations are integrated and composed at each level; displacement fields propagate to finer grids, where inverse-inconsistency feedback conditions routing and confidence estimation.After the finest update, the model returns registered images and consistency-adjusted confidence maps; shared parameters and bidirectional feedback do not guarantee exact inverse consistency or fold-free discrete transformations.Source: §3.4 · p. 5
3.5. Training Objective
- Training uses image alignment, geometric regularization, and confidence self-supervision without segmentation supervision.The confidence targets derive from local registration dissimilarity, with gradients stopped through the targets.Source: §3.5 · p. 5
4. Experiment
The experiments evaluate ReliMorph on brain MR and abdominal CT–MR registration, comparing it with methods spanning correspondence modeling, adaptive processing, diffeomorphic registration, and efficient convolution. Reported results cover benchmark performance, qualitative alignment, computational cost, and component ablations.
4.1. Experimental Setup
- The evaluation spans IXI and OASIS inter-subject brain MR registration and AbdomMRCT intrapatient moving-CT to fixed-MR registration, using fixed image pairs for evaluation.The selected baselines cover correspondence and context modeling, adaptive feature processing, diffeomorphic modeling, and efficient convolutional registration.Source: §4.1 · pp. 5–7
4.2. Performance Comparison
- ReliMorph achieves the highest DSC and lowest HD95 across IXI, OASIS, and AbdomMRCT; its OASIS accuracy comes with slightly higher folding than LGANet++.On AbdomMRCT, it also reduces folding and MIND-SSC dissimilarity relative to TransMorph.Source: Table 1 · p. 6, §4.2 · p. 7
4.1. Experimental Setup
- Anatomical overlap is measured with DSC, boundary discrepancy with HD95, and deformation regularity with Fold; AbdomMRCT additionally uses LMIND-SSC for modality-independent structural agreement.Fold is the percentage of voxels with det(∇ϕ) ≤0.Source: §4.1 · p. 7, §4.3 · p. 8
4.2. Performance Comparison
- Qualitative examples show weaker cortical-boundary residuals and smooth Jacobian maps on IXI, close anatomical layout on OASIS, and less elongated or fragmented abdominal propagated labels.Figure 4 encodes displacement components in RGB and Jacobian regions by folding, near-unit volume change, compression, and expansion; Figure 5 shows warped images, fields, and abdominal masks.Source: Figure 4 · p. 6, §4.2 · p. 7, Figure 5 · p. 8
- ReliMorph reaches DSC 0.8093 on IXI with inference taking 0.78 ± 0.12 s per image pair and 1.75 GiB peak allocated GPU memory.The comparison covers accuracy, inference time, and peak GPU memory across methods.Source: §4.2 · p. 7, Figure 6 · p. 8
4.3. Ablation study
- Removing explicit correspondence descriptors, adaptive routing, velocity gating, or cross-scale inverse-inconsistency feedback weakens IXI alignment; removing the positive gate floor causes the largest accuracy drop.Without the floor, performance is 0.7745 DSC with 0% folding, consistent with conservative under-registration from over-suppressed velocities.Source: §4.3 · p. 7, Table 2 · p. 8
- As matching ambiguity increases, routing shifts from the local expert toward the dilated expert, and suppressing each expert most harms its corresponding ambiguity regime.Suppressing the local expert increases disagreement by +1.42 pp in low-ambiguity regions; suppressing the dilated expert increases it by +1.56 pp in high-ambiguity regions.Source: §4.3 · pp. 7–8, Table 3 · p. 8
5. Conclusion
ReliMorph couples correspondence reliability with context selection and deformation updates in symmetric 3D registration. Experiments report improved anatomical overlap across unimodal brain and multimodal abdominal registration.
5. Conclusion
- ReliMorph combines shared anatomical encoding with modality-specific adaptation and uses correlation statistics, deformation-state cues, and inverse-inconsistency feedback to route spatial experts and gate residual velocities during coarse-to-fine refinement.Stationary-velocity integration and transformation composition refine both directions, with updated consistency feedback guiding later routing and modulation.Source: §5 · p. 8
Supplementary Material
The supplementary material provides default hyperparameters in Table S1.
Supplementary Material
- Table S1 lists default hyperparameters; merged entries apply to all datasets, and per-level vectors run from coarse to fine.Source: Table S1 · p. 11
A1. Reproducibility Details
The reproducibility details specify dataset preprocessing and fixed hyperparameters, then describe a temperature comparison on IXI that tracks registration performance and correlation-distribution entropy.
A1. Reproducibility Details
- IXI, OASIS, and AbdomMRCT use dataset-specific preprocessing, while hyperparameters are fixed a priori and model selection uses validation Dice.The section specifies affine initialization and intensity normalization for each dataset.Source: A1. Reproducibility Details · p. 11
A1.1. Effect of Correlation Temperature
- The IXI temperature ablation compares learnable per-level temperatures with fixed temperatures under the same search radii and training protocol.Fixed temperatures remain unchanged during training and inference.Source: A1.1. Effect of Correlation Temperature · p. 11
- Table S2 reports registration performance and finest-level mean normalized entropy across the same pairs, directions, and spatial regions.Per-level entropy statistics are also recorded to examine temperature-related changes across scales; the caption notes that entropy has no preferred direction.Source: A1.1. Effect of Correlation Temperature · p. 11, Table S2 · p. 11
A2. Extended Comparisons
The extended comparisons describe shared evaluation protocols and additional baselines, and define a modality-independent structural evaluation for abdominal CT–MR registration.
A2.1. Additional Baseline Comparisons
- Additional baseline comparisons report Jaccard alongside DSC, HD95, and folding rates, with distributions and qualitative examples spanning the evaluated datasets.The section also describes shared splits, evaluation pairs, preprocessing, resolutions, and affine initialization; method-specific architectures and regularizers are retained.Source: A2.1. Additional Baseline Comparisons · p. 11, Figure S1 · p. 12
- For abdominal CT–MR, the comparison substitutes MIND-SSC dissimilarity where cross-modal adaptation is required, while methods with modality-independent objectives retain them.The section identifies additional supplementary baselines but supplies no comparative outcome in these passages.Source: A2.1. Additional Baseline Comparisons · p. 11
A2.2. Modality-Independent Structural Evaluation
- The modality-independent evaluation measures MIND-SSC descriptor agreement over a common region formed from the fixed-MR body mask and valid descriptor neighborhoods.It uses 12-channel descriptors with patch radius two voxels and dilation two voxels under identical preprocessing and normalization.Source: A2.2. Modality-Independent Structural… · p. 12
A3. Gate and Geometric Regularity
The gate-floor sensitivity experiment retrains configurations on IXI and OASIS and finds a measured trade-off between anatomical alignment and folding.
A3. Gate and Geometric Regularity
- The experiment evaluates cmin values of 0, 0.1, 0.25, 0.5, and 1 on IXI and OASIS using the velocity gate defined in the main paper.Configurations share data splits, training budget, remaining hyperparameters, validation-based checkpoint selection, and integration depth T = 7.Source: A3. Gate and Geometric Regularity · p. 12
- Among tested settings, cmin = 0.25 achieves the highest DSC and lowest HD95 and ICE on both datasets, but also the highest measured folding rate.Because each configuration is retrained, the results reflect jointly learned velocity and confidence predictions.Source: A3. Gate and Geometric Regularity · p. 12
A4. Expert and Routing Analysis
The analysis examines expert composition and routing choices, including their effects on registration, computational cost, and region-dependent specialization.
A4.1. Effect of Expert Composition
- The default expert set combines local-detail, axial-context, and dilated-context branches, and the analysis tests branch removal and alternative homogeneous or dilation-only sets.The configurations retain soft routing and the same encoder, matching representation, velocity decoder, confidence gating, and objective; Table S5 compares performance, parameter counts, and FLOPs.Source: A4.1. Effect of Expert Composition · pp. 12–13
- Qualitative composition comparisons show warped-image overlays and signed residuals for IXI and OASIS, or label-disagreement maps for AbdomMRCT.The figure caption specifies the displayed comparisons but these passages do not state their outcomes.Source: Figure S2 · p. 13
A4.2. Comparison of Routing Policies
- Routing alternatives include uniform fusion, soft routing, and top-1 or top-2 selection, with selected weights renormalized over the retained experts.Top-k policies use a straight-through estimator during training and selected, renormalized weights at inference.Source: A4.2. Comparison of Routing Policies · pp. 13–14
- The routing comparison measures accuracy, folding, inference time, and peak GPU memory, but top-k selection does not skip expert convolutions because all expert feature maps are computed first.A qualitative IXI comparison also visualizes warped grids and displacement fields.Source: A4.2. Comparison of Routing Policies · p. 14, Figure S3 · p. 14
A5. Reliability Evaluation
Reliability Evaluation tests how confidence supervision, velocity modulation, and reliability scores relate to registration accuracy and quality assessment. Consistency-adjusted confidence ranks highest for detecting low-quality registration pairs among the evaluated scores.
A5. Reliability Evaluation
- The section evaluates confidence supervision and velocity modulation, confidence against anatomical disagreement, and confidence-based identification of low-quality pairs.Table S7 covers accuracy, folding, and AURC; its variants include independently trained configurations and a frozen full-model intervention.Source: A5. Reliability Evaluation · pp. 14–15, A5.1. Confidence Supervision and… · p. 15, Table S7 · p. 15
- Expert-composition and routing-policy comparisons are conducted on IXI using soft routing for composition variants.The routing-policy comparison uses the same local, axial, and dilated experts and reports inference time and peak GPU memory.Source: Table S5 · p. 15, Table S6 · p. 15
A5.1. Confidence Supervision and Velocity Modulation
- Controlled comparisons test confidence supervision and velocity modulation separately, including a frozen intervention that replaces spatial gates with mean-matched constants.The intervention removes spatial gate variation, while mean velocity magnitudes are not constrained to match.Source: A5.1. Confidence Supervision and… · p. 15
A5.2. Confidence and Registration Error
- Voxel-level evaluation compares confidence scores and registration cues against anatomical label disagreement using error-detection and risk–coverage metrics.The evaluation distinguishes foreground voxels from anatomical boundary voxels; lower AURC means fewer errors among high-confidence predictions.Source: A5.2. Confidence and Registration Error · p. 16, Table S8 · p. 17
A5.3. Inference-Time Quality Assessment
- Pair-level reliability aggregates voxelwise scores in an image-derived foreground, with low quality defined relative to the validation cohort.The DSC cutoff is the lower quartile of the full model’s validation DSC distribution and is fixed for evaluation.Source: A5.3. Inference-Time Quality Assessment · p. 16
- Consistency-adjusted confidence achieves the highest AUROC and AP among evaluated scores for detecting low-quality registration pairs.The low-quality cutoff is set from the validation DSC distribution, and score thresholds target a 10% validation flag rate.Source: A5.3. Inference-Time Quality Assessment · p. 17, Table S9 · p. 17
A6. Inference Efficiency and Memory Analysis
The section benchmarks inference latency and peak GPU memory for baseline methods on IXI. It specifies the hardware, timing protocol, and scope of the measured pipeline.
A6. Inference Efficiency and Memory Analysis
- Inference efficiency is compared across baseline methods on IXI with input size 160 × 192 × 224.Neural models use evaluation mode, batch size one, and FP32 without automatic mixed precision on one NVIDIA A6000 GPU; SyN uses the same workstation with a fixed CPU thread count.Source: A6. Inference Efficiency and Memory… · p. 17, Table S10 · p. 17
- Latency covers the pipeline from a preprocessed image pair to the full-resolution transformation and warped image, including both registration directions for ReliMorph.ReliMorph timing includes feature extraction, correlation volumes, reliability statistics, expert routing, confidence gating, and full bidirectional refinement.Source: A6. Inference Efficiency and Memory… · p. 17
- The protocol times five passes per test pair after 20 warm-ups and records peak allocated GPU memory.CUDA is synchronized around each timed run, and latency is reported as mean ± standard deviation across pairs.Source: A6. Inference Efficiency and Memory… · p. 17, Table S10 · p. 17