Source-linked AI summary
Recursive Cascaded Networks for Unsupervised Medical Image Registration
Shengyu Zhao, Yue Dong, Eric I-Chao Chang, Yan Xu
TL;DR
Deformable registration needs learning methods that avoid limited-quality ground-truth flows and handle complicated deformations. The paper introduces recursively trained cascades that progressively warp images using any base network, achieving consistent gains across medical-image datasets and evaluation metrics. Shared-weight testing can further refine alignment, but may over-deform images and does not always improve performance.
Problem
Limited-quality ground-truth flows constrain supervised registration, while existing unsupervised networks struggle with complicated deformations and large displacements.
Method
Recursive cascades repeatedly align the current warped image to the fixed image, compose their flow fields, and train end-to-end using only final-image similarity.
Results
The architecture achieves consistent, significant gains across liver and brain datasets and diverse evaluation metrics, independently of the base network.
Takeaways & Limitations
Recursive cascading supports unsupervised deformable registration with deeper cascades and shared-weight test-time refinement across base networks.
Takeaways & Limitations
Shared-weight cascading does not always improve performance and may cause over deformation; recursion can also amplify folding areas in composed fields.
Abstract
from arXiv · showhide
We present recursive cascaded networks, a general architecture that enables learning deep cascades, for deformable image registration. The proposed architecture is simple in design and can be built on any base network. The moving image is warped successively by each cascade and finally aligned to the fixed image; this procedure is recursive in a way that every cascade learns to perform a progressive deformation for the current warped image. The entire system is end-to-end and jointly trained in an unsupervised manner. In addition, enabled by the recursive architecture, one cascade can be iteratively applied for multiple times during testing, which approaches a better fit between each of the image pairs. We evaluate our method on 3D medical images, where deformable registration is most commonly applied. We demonstrate that recursive cascaded networks achieve consistent, significant gains and outperform state-of-the-art methods. The performance reveals an increasing trend as long as more cascades are trained, while the limit is not observed. Code is available at https://github.com/microsoft/Recursive-Cascaded-Networks.
1. Introduction
Deformable registration methods face limited supervised training data and difficulty handling complicated, large-displacement deformations. The paper proposes recursive cascades that cooperatively learn progressive alignments through unsupervised end-to-end training.
- Supervised registration methods are constrained by the limited quality of ground-truth flows from traditional algorithms or simulated deformations.
- Existing unsupervised networks often make straightforward predictions, burdening them with complicated deformations and large displacements.
- Recursive cascades take the current warped image and fixed image as inputs, measuring similarity only on the final warped image for cooperative progressive alignment.
- Each cascade learns a simple alignment, while composing recursively predicted flow fields allows deeper cascades to refine small displacements.
- The architecture supports shared-weight cascades and can be built on existing base networks, enabling deeper unsupervised cascades without requiring a specific base network.
2. Related Work
Prior work includes cascaded computer-vision networks, iterative traditional registration algorithms, and learning-based supervised or unsupervised registration. VoxelMorph and VTN predict dense flow fields, while deeper cascades had not been shown to advance deformable registration performance.
- Cascade architectures have been used for pose regression, object detection, semantic segmentation, image super-resolution, and several medical-image tasks.
- Traditional registration methods commonly perform recursive iterations that optimize energy functions or progressively solve alignment problems.
- Supervised registration requires labeled or simulated deformation data, whereas unsupervised methods use image similarity through differentiable warping.
- VoxelMorph and VTN predict dense flow fields with deconvolutional layers, while DLIR predicts a sparse displacement grid interpolated by a third-order B-spline kernel.
- The paper identifies no prior work showing that training deeper cascades advances deformable image-registration performance.
3. Recursive Cascaded Networks
Recursive cascades progressively warp the moving image toward the fixed image, enabling jointly trained unsupervised registration and optional shared-weight refinement at test time.
- Recursive architecture: The registration function predicts a dense flow field from the moving and fixed images, with each cascade operating on the currently warped moving image.The cascades compose flow fields recursively rather than making one straightforward prediction.
- Recursive architecture: Cascaded refinements decompose potentially large displacements into successive small-displacement alignments.The final prediction is formed by composing the flow fields predicted across cascades.
- Network design: The architecture can use different networks or a common base network, and it includes an affine top-level cascade for medical images that are roughly pre-aligned.The affine cascade predicts only an affine transform, while deformable cascades predict dense flow fields.
- Training: All cascades can be jointly trained end-to-end without supervision by backpropagating through differentiable image warping and measuring similarity on the final warped image.Each predicted flow field also receives smoothness regularization through an L2 variation loss.
- Shared-weight cascading: Shared-weight cascading reuses cascade parameters so additional recursive refinements can be applied during testing after training.The method can substitute each cascade with repeated applications to form r×n cascades.
- Limitations: Shared-weight test-time refinement may cause over deformation: similarity is ensured to increase, but the aggregate flow can become less natural when images are too perfectly matched.Training depth is also constrained by GPU memory, while distinct parameters may perform better with sufficiently large datasets.
4. Experiments
The experiments evaluate recursive cascaded registration on liver CT and brain MRI using segmentation overlap and landmark distances, comparing multiple baselines and cascade configurations. Results show consistent gains across base networks and datasets, with benefits from jointly trained recursive cascades and robustness to atlas choice.
- Datasets and settings: Experiments cover liver CT pairwise registration and brain MRI atlas-based registration across multiple medical-image datasets.Liver evaluation uses SLIVER, LiTS, and LSPIG; brain evaluation uses LPBA with an atlas-based setup.
- Evaluation metrics: Performance is quantified using Dice scores for anatomical overlap and average landmark distance for datasets with landmark annotations.Dice averages overlap across annotated structures, while landmark distance averages distances between fixed-image landmarks and warped moving-image landmarks.
- Overall results: Recursive cascaded networks outperform existing methods across all evaluated datasets with significant gains, while a 10-cascade VTN remains comparable in GPU runtime to baseline networks.The architecture applies to both VTN and VoxelMorph, and runtime increases linearly with the number of cascades.
- Number of cascades: Increasing the number of recursive cascades produces consistent performance gains independently of whether VTN or VoxelMorph is used as the base network.A 3-cascade VTN already outperforms the comparable VTN baseline, which the authors attribute to progressively learned intermediate alignments and final-image similarity supervision.
- Shared-weight cascading: Shared-weight cascading increases image similarity, but generally performs worse than jointly trained cascades and is likely to deteriorate quality beyond three repetitions.The reported similarity measure is the correlation coefficient between the warped moving image and the fixed image.
- Cascades versus capacity: Cascade-based improvements exceed those from simply widening or deepening comparable networks, despite VM x2 using four times as many parameters as VoxelMorph.VM x2 performs better than the original VoxelMorph but worse than both cascade variants; similarly, deeper VM variants are outperformed by 2-cascade VM.
5. Discussion
The discussion identifies composed-field smoothness as a limitation of recursive cascades, particularly when folding is amplified during recursion. It also notes that deeper cascades and shared-weight training may require additional resources and careful design.
- Future directions: Deeper cascades may improve performance when additional resources or distributed learning are available.The authors also identify training or fine-tuning shared-weight cascades and lightweight base networks as avenues for exploration.
- Limitations: Composed-field smoothness is a possible limitation because recursive composition may amplify folding areas.The issue is especially challenging for weight-sharing techniques.
- Limitations: Careful regularization or an invertible base network could reduce the risks associated with folding during recursion.
6. Conclusion
The paper presents a deep recursive cascade architecture for deformable medical image registration and evaluates it on liver and brain datasets. Across diverse metrics, it reports significant gains over state-of-the-art methods, with unsupervised learning and base-network independence supporting broader potential applicability.
- Conclusion: The architecture achieves significant gains over state-of-the-art methods on both liver and brain datasets.The evaluation uses diverse metrics for deformable medical image registration.
- Conclusion: The method combines good performance, unsupervised learning, and independence from the base network.
Abstract
The supplementary material visualizes recursive-registration results across liver and brain evaluation datasets. It shows input images, output flows, warped images or grids, and comparisons involving landmarks or anatomical annotations where available.
- Visualization procedure: The supplementary visualizations are produced by drawing output flows, computing warped images and grids, and comparing warped annotations with fixed images.
- Evaluation datasets: The evaluation datasets include liver CT scans from SLIVER, LiTS, and LSPIG, plus brain MRIs from LPBA.
- Visualizations: SLIVER visualizations show flow fields, landmark locations, sliced segmentations, and examples from different methods.Flow displacement components are mapped to red, green, and blue channels.
- Visualizations: LiTS examples compare images, flows, and segmentations across different registration methods.
- Visualizations: LSPIG examples compare median slices of images, flows, and segmentations across different methods.
- Visualizations: LPBA examples compare median image and flow slices, with five chosen anatomical structures projected onto the plane.