Source-linked AI summary
Slice-to-volume medical image registration: a survey
Enzo Ferrante, Nikos Paragios
TL;DR
Slice-to-volume registration is important for image-guided interventions and volume reconstruction, but its extensive literature lacked a comprehensive survey. This paper systematically reviews the field through a taxonomy of algorithms and analyzes category-level advantages, disadvantages, and future directions. The survey identifies dominant methodological patterns while emphasizing validation limitations and emerging opportunities for new approaches.
Problem
Slice-to-volume registration supports image-guided interventions and volume reconstruction, yet the field lacked a comprehensive survey despite extensive literature and has limited open benchmarks with gold-standard annotations.
Method
The paper systematically reviews slice-to-volume registration studies using a taxonomy covering matching criteria, transformation models, optimization, and number of slices.
Results
Most reviewed methods use rigid registration with iconic matching criteria and continuous optimization, while the discussion identifies deep convolutional networks as a future paradigm for the field.
Takeaways & Limitations
The survey provides a categorical framework for comparing slice-to-volume registration algorithms and identifies future directions beyond the dominant geometric approaches.
Takeaways & Limitations
The field lacks open public benchmarks with gold-standard annotations, limiting fair comparison of slice-to-volume registration methods.
Abstract
from arXiv · showhide
During the last decades, the research community of medical imaging has witnessed continuous advances in image registration methods, which pushed the limits of the state-of-the-art and enabled the development of novel medical procedures. A particular type of image registration problem, known as slice-to-volume registration, played a fundamental role in areas like image guided surgeries and volumetric image reconstruction. However, to date, and despite the extensive literature available on this topic, no survey has been written to discuss this challenging problem. This paper introduces the first comprehensive survey of the literature about slice-to-volume registration, presenting a categorical study of the algorithms according to an ad-hoc taxonomy and analyzing advantages and disadvantages of every category. We draw some general conclusions from this analysis and present our perspectives on the future of the field.
1. Introduction
Slice-to-volume registration aligns a 2D image with its corresponding plane in a 3D volume and supports image-guided intervention and volume reconstruction. This survey addresses the field's missing comprehensive synthesis by organizing prior work through a taxonomy of registration components and application settings.
- Definition: Slice-to-volume registration identifies the slice in a 3D volume corresponding to an input 2D image.It is a specialized image-registration problem involving images with different dimensions.
- Applications: The problem supports image-guided interventions by fusing pre-operative 3D data with real-time intra-operative 2D images.It also supports volume reconstruction by mapping individual slices to a reference volume to correct inter-slice misalignment.
- Challenges: Slice-to-volume registration cannot be obtained by straightforwardly extrapolating 3D-3D methods because sparse slices provide less information and impose planarity and deformation constraints.Similarity descriptors that assume balanced image information require explicit adaptation to the slice-to-volume setting.
- Survey scope: The survey fills an identified gap by reviewing the literature, proposing a taxonomy, and analyzing advantages and disadvantages across algorithm categories.The taxonomy is adapted to slice-to-volume registration and organizes work by matching criterion, transformation model, optimization, and number of slices.
- Formal framework: The review defines registration through an objective combining image similarity and regularization while allowing rigid or non-rigid mappings.The surveyed framework also includes multi-slice methods that register sparse slices directly to a volume, excluding approaches that reconstruct a volume beforehand.
2. Matching Criterion
Matching criteria quantify alignment between a slice and its mapped volume representation using intensity, geometric, or sensor information. The survey describes modality-dependent criteria and highlights difficulties caused by sparse information, multimodality, noise, and weakly distinctive medical-image features.
- Criterion families: Matching criteria quantify image alignment and guide optimization using iconic intensities, geometric landmarks, or sensor information.The survey distinguishes these strategies according to the information exploited during matching.
- Iconic matching: Iconic criteria compare intensity information, but multimodal registration requires more complex similarities or alternative geometric and sensor-based strategies.Monomodal images usually have more directly correlated intensity values than multimodal images.
- Iconic matching: Mutual information is common for multimodal registration, yet sparse low-resolution slices make slice-specific joint-density estimation difficult and can degrade results.The survey describes informative probability-density priors as one strategy for improving such estimates.
- Alternative criteria: Region-based, gradient-based, and robust modality-specific criteria extend intensity matching when segmentation, borders, or multimodal image formation provide useful structure.Examples include segmentation-based Chan–Vese matching, gradient-magnitude criteria, MIND, and simulation-based similarities.
- Geometric matching: Geometric methods minimize discrepancies between salient landmarks in the slice and volume, but extracting distinctive features is difficult in low-resolution medical images with limited texture.Super-resolution or registration-based interpolation can improve image quality before feature extraction.
3. Transformation Model
Transformation models describe how a slice and volume are brought into spatial correspondence, ranging from rigid motion to local deformation. Their expressive power increases with model complexity, creating a trade-off between flexibility and parameterization.
- Model spectrum: Rigid transformations model global rotation and translation, whereas deformable models can represent local in-plane and out-of-plane changes.The survey relates model richness to the number of parameters that must be specified.
- Rigid models: The common rigid model uses 6 degrees of freedom—three rotations and three translations—and handles simple in-plane and out-of-plane motion.Such models can be sufficient for clinical settings without image distortion, including basic inter-slice motion correction.
- Rigid models: Rigid alignment is often used before deformable registration to account for large-range displacements before estimating local deformations.The survey reports this two-step strategy across several deformable-registration studies.
- Non-rigid models: Non-rigid models include linear operations such as scale, flip, and shear as well as complex local deformations.The survey includes affine, thin-plate spline, free-form deformation, and finite-element models among the approaches reviewed.
- Non-rigid models: Finite-element models can incorporate dynamic tissue behavior through physically meaningful constraints on deformation.This capability is presented as an advantage for modeling tissue deformation during registration.
4. Optimization Method
Slice-to-volume registration is optimized through continuous or discrete formulations. The review contrasts gradient-based and derivative-free strategies, then summarizes discrete methods’ flexibility and accuracy trade-off.
- Continuous optimization: Most published slice-to-volume registration methods use continuous optimization because the search space is continuous.
- Gradient-based methods: Gradient-based methods use objective-function derivatives but may require difficult derivative calculations and can converge to local minima.
- Non-gradient-based methods: Derivative-free methods support noisy, non-differentiable, or black-box objectives that frequently arise in medical image registration.
- Non-gradient-based methods: Local search perturbs each parameter and selects the greatest improvement, but its behavior depends on the initial step size.
- Non-gradient-based methods: Powell’s method searches along multiple directions and can be initialized with principal components to reduce an affine transformation’s search space.
- Discrete optimization: Discrete methods avoid differentiability requirements and can control label-space range and resolution, but their accuracy is bounded by discretization.
5. Number of Slices
The review distinguishes single-slice from multi-slice registration. Multi-slice methods use several slices to improve matching and robustness, while managing contextual information and computational cost.
- Definitions: Single-slice methods estimate a mapping from one 2D slice, whereas multi-slice methods map several slices to the 3D volume.
- Single-slice extensions: Applying a single-slice method independently to every slice enables parallelization but loses contextual information between slices.
- Multi-slice registration: Multi-slice registration can produce results close enough to volume-to-volume registration while using only a few slices instead of a complete volume.
- Multi-slice registration: Some approaches register multiple slices consistently without reconstructing them into a complete volume before registration.
- Multi-slice registration: Multi-slice registration improves matching robustness by augmenting image support, although it requires more computation than single-slice registration.
- Multi-slice registration: Key-slice selection exploits redundant neighboring frames by choosing informative slices, including those with the highest image entropy.
6. Applications
Slice-to-volume registration supports image fusion in guided interventions and motion correction during volume reconstruction. The review describes how these applications align intra-operative or motion-affected slices with volumetric references.
- Image fusion and guided interventions: Image fusion combines pre-operative 3D images with intra-operative 2D images, allowing physicians to navigate annotated high-resolution data during interventions.
- Image fusion and guided interventions: Figure 4 depicts an intra-operative ultrasound slice registered to a pre-operative CT image, with both images shown side-by-side for complementary information.
- Motion correction and volume reconstruction: Figure 5 illustrates patient motion during slice-by-slice acquisition, which produces a corrupted stacked volume that registration aims to improve.
- Motion correction and volume reconstruction: Motion correction and volume reconstruction map individual slices to a reference volume to correct inter-slice misalignment.
- Motion correction and volume reconstruction: The MSV approach independently estimates six-degree-of-freedom rigid transformations for fMRI slices using mutual information and Nelder–Mead optimization.
- Motion correction and volume reconstruction: When no anatomical reference exists, a reference volume can be calculated during registration from the current slice estimates.
7. Validation of Slice-to-Volume Registration Methods
Validation studies use phantoms and clinical images, with metrics based on transformations, landmarks, voxel distances, or segmentation masks. The review highlights limited purpose-built benchmarks and annotation constraints, especially for sparse slices and clinical data.
- Validation datasets: Purpose-built public benchmarks with gold-standard annotations are scarce for slice-to-volume registration.MNI BITE is used in several studies but was not designed specifically for this task, particularly single-slice methods.
- Future validation resources: Future validation datasets should provide landmarks, manual segmentations, and gold-standard 2D-to-3D transformations.Such resources would support more consistent comparisons between slice-to-volume registration methods.
- Type of images: Validation commonly combines phantoms for accurate measurements with clinical images for realistic assessment.Clinical datasets better reflect application settings but are harder to annotate because specialist input is often required.
- Transformation-based metrics: Transformation-parameter errors compare estimated transformations with known ground truth, but exact slice-to-volume transformations are rarely available.Researchers therefore generate synthetic ground truth from known volumes or use bronze-standard annotations when possible.
- Point-based metrics: Landmark-based validation measures distances between corresponding points, commonly reporting target registration error or related fiducial measures.Sparse slices can make shared landmark annotation extremely difficult, forcing dataset-specific workarounds that reduce usable slices.
- Segmentation-based metrics: Segmentation-based validation is more available than landmarks or ground-truth transformations but requires small, localized structures to provide reliable evidence.Combining overlap and contour-distance measures can address inconsistencies in assessing registration quality.
8. Discussion and future directions
The survey finds that slice-to-volume registration research is dominated by rigid models, iconic similarity measures, and continuous optimization, while identifying opportunities for richer deformation models, combined sensing, discrete optimization, and learned approaches.
- The survey classifies methods by matching criterion, transformation model, optimization method, slice count, application, and validation strategy.
- Most methods use rigid transformations, iconic matching criteria, and continuous optimization techniques.
- Rigid models suit simple slice-to-volume mappings and basic motion correction, but complex imaging involves tissue deformation and natural organ motion.
- Multiple slices can improve matching quality by increasing image support, while sensor-based strategies can complement iconic measures and refine rigid estimates.
- Discrete optimization remains underused but may help with many parameters, non-convex energies, or wide search spaces.
- Deep convolutional networks offer a different registration paradigm by learning similarity measures or registration processes from large datasets.