Slice-to-volume medical image registration: a survey

1. Introduction

• Slice-to-volume registration identifies the slice in a 3D volume corresponding to an arbitrary-plane 2D image, supporting image fusion during interventions and correction of misaligned slices during volume reconstruction.In interventions, pre-operative 3D data are fused with real-time intra-operative 2D images; reconstruction methods map individual slices onto a reference volume.Source: §1 · pp. 1–2
• Projective and sliced/tomographic 2D/3D registration require different strategies because their image geometries and pixel-to-voxel relationships differ.Projective-image pixels represent projections of multiple voxels, whereas slice-to-volume pixels can be directly compared with corresponding volume voxels.Source: §1 · p. 2
• Slice-to-volume registration cannot straightforwardly reuse 3D/3D methods because sparse slices provide unbalanced information and require planarity and in-plane deformation constraints.These constraints are specific to the slice-to-volume setting and are not required when both images have corresponding dimensions.Source: §1 · p. 2
• The survey addresses a gap by categorizing slice-to-volume algorithms according to matching criterion, transformation model, optimization, slice number, applications, and validation strategy.The taxonomy is adapted to slice-to-volume registration from previous image-registration surveys and is illustrated in Figure 1.Source: §1 · pp. 2–3, Figure 1 · p. 3

1.1. Definition of Slice-to-Volume Registration

• The registration objective combines an image-matching term M with regularization R, and the mapping may be rigid or non-rigid depending on the allowed transformation.Regularization becomes crucial for complex non-rigid models, encoding planarity, deformation limits, tissue elasticity, or consistency among multiple slices.Source: §1.1 · pp. 3–4
• The survey includes direct registration of multiple sparse slices to a volume but excludes methods that first reconstruct a volume, reducing the problem to 3D/3D registration.Included slices may be orthogonal, parallel, arbitrary, or lack a known spatial relation.Source: §1.1 · p. 4
• The catalog includes Rivaz et al. (2014b), Rivaz et al. (2014a), Schulz et al. (2014), Xu et al. (2014a), Xu et al. (2014b), and Ferrante et al. (2015a).Each entry is accompanied by an X/dash pattern whose meaning is not specified in the supplied passages.Source: §1.1 · p. 6
• The catalog also lists Ferrante et al. (2015b), Hallack et al. (2015), Kainz et al. (2015), Yavariabdi et al. (2015), Chen et al. (2016), and Guzman et al. (2016).The passages report only the associated coded X/dash entries, without defining their categories.Source: §1.1 · p. 6
• Additional listed studies are Marami et al. (2016b,a), Murgasova et al. (2016), Porchetto et al. (2016), and Xiao et al. (2016).Their entries likewise contain X/dash patterns whose interpretation is not supplied.Source: §1.1 · p. 6

2. Matching Criterion

• Matching criteria classify slice-to-volume registration as iconic, geometric, hybrid, or sensor-based according to whether they exploit intensities, landmarks, both, or non-image technologies.Image-based methods can accommodate elastic changes but face noise and ambiguity from sparse slices, whereas sensor-based methods estimate rigid transformations and require calibration.Source: §2 · p. 7

2.1. Iconic

• Iconic criteria use image intensities or derived statistics: SAD and SSD suit monomodal correspondence, whereas CC, CR, MI, and NMI model statistical intensity relationships, including multimodal registration.MI requires joint and marginal intensity PDFs, suffers when slice information is sparse, and varies with changing overlap. NMI is overlap-invariant and ranges from 0 to 1.Source: §2.1 · pp. 8–9

2.2. Geometric

• Geometric methods register salient anatomical landmarks or shapes, offering simple optimization, broad capture range, and low initialization sensitivity but depending critically on difficult feature extraction and matching.Sparse landmarks can yield insufficient deformation support; ICP, signed distance maps, and related methods avoid or jointly solve correspondence, while low resolution and manual segmentation remain important limitations.Source: §2.2 · pp. 11–13

2.3. Sensor Based Methods

• Sensor-based methods use optical or electromagnetic tracking to update slice pose, providing reliable rigid localization when calibrated but remaining limited by line-of-sight, metal distortion, lower accuracy, or accumulated errors.Neither OTS nor EMTS handles elastic volume–slice deformation; optical tracking requires visibility, whereas electromagnetic tracking avoids line-of-sight constraints but is sensitive to nearby metal.Source: §2.3 · pp. 13–14

2.4. Hybrid

• Hybrid criteria combine iconic or geometric image information with each other or with tracking, using sensors for initialization or updates and image matching for refinement.Mitra et al. (2012) combined shape-context and intensity similarities but required manual prostate segmentations and assumed parallel TRUS and MR planes; Penney et al. (2004a) and Xu et al. (2008) used tracking with image registration for motion compensation.Source: §2.4 · pp. 14–16
• Ghanavati et al. (2010) registered multiple freehand ultrasound slices to a pelvic CT statistical atlas using ultrasound simulation and iconic similarity, later converting the two-step process into one-shot optimization in Ghanavati et al. (2011).The method first rigidly aligned the atlas mean shape, then optimized deformable atlas parameters against simulated and actual ultrasound images.Source: §2.4 · p. 16
• Wein et al. (2008b) used a skin-mounted position sensor to estimate respiratory anterior-posterior liver motion and combined it with an iconic criterion to optimize a 12-DOF affine transformation.The transformation mapped pre-operative plans and imaging into the interventional scenario.Source: §2.4 · p. 16
• Huang et al. (2009) combined ECG synchronization, optical tracking, and mutual-information refinement to register ultrasound slices to dynamic cardiac CT in real time.ECG selected the corresponding cardiac phase, tracking supplied near-optimal initialization, and mutual-information optimization refined each slice; Hummel et al. (2008) similarly combined EM and OTS sensors with iconic registration for endoscopic ultrasound and CT.Source: §2.4 · p. 16
• Rigid slice-to-volume registration commonly models transformation with six degrees of freedom for rotation and translation.The survey notes that this simple transformation has been widely adopted by reviewed methods.Source: Figure 2 · p. 17

3. Transformation Model

• Transformation models are classified by degrees of freedom, with rigid models describing global rotations and translations and deformable models allowing local in-plane or out-of-plane changes.Model complexity and expressive power increase with the number of parameters.Source: §3 · p. 17

3.1. Rigid

• 6-DOF rigid transformations are the most common model, handling in-plane and out-of-plane motion and often providing initial alignment before deformable registration.Restricted models can simplify optimization when out-of-plane motion is avoided, but complete slice-to-volume alignment may require all six degrees of freedom.Source: §3.1 · pp. 17–18

3.2. Non-Rigid

• Non-rigid transformations range from global scale, flip, shear, and affine operations to models that produce local structural deformations.Affine models can remove large-scale respiratory differences in intra-operative 2D US–CT registration, but cannot capture localized deformations such as those observed near the rectum in prostate histology–MRI registration.Source: §3.2 · p. 18, §3.2.1 · p. 19

3.2.1. Linear Models

• Affine models can initialize or correct slice-to-volume alignment, using methods such as the Generalized Hough Transform for starting-position detection and mutual-information refinement.Their suitability depends on the deformation scale because global affine transformations cannot express local deformations.Source: §3.2.1 · p. 19

3.2.2. Deformable Models

• Deformable models include TPS, FFD, and finite-element meshes, offering local or physically constrained deformation fields for complex slice-to-volume mapping.TPS outperformed rigid registration in a multi-slice fMRI scenario with local deformations, while results depended strongly on control-point number and placement; FFDs trade flexible local support for regular-grid constraints, and FE meshes incorporate physically meaningful tissue behavior.Source: §3.2.2 · pp. 20–21

4. Optimization Method

• Most slice-to-volume registration methods use continuous optimization, while discrete methods operate on quantized solutions and heuristics trade optimality guarantees for broader applicability.Continuous methods search the full parameter space; heuristics are useful when optimal inference is impractical or when an approximately good initialization is sufficient.Source: §4 · p. 21, §4.3 · p. 26

4.1. Continuous

• Continuous methods iteratively update transformation parameters using gradient- or non-gradient-based search directions, with comparative analysis reported for histological-slice to MR registration (Osechinskiy and Kruggel, 2011b).Gradient methods can exploit derivatives, whereas derivative-free methods avoid differentiability requirements; continuous strategies include gradient descent, conjugate gradient, quasi-Newton, and Gauss-Newton methods.Source: §4.1 · pp. 21–22

4.1.1. Gradient based Methods

• Gradient-based methods can provide optimality guarantees for convex differentiable objectives, but otherwise may reach local minima, depend on initialization, and require derivative calculation.Gradient descent is widely used for slice-to-volume registration, while conjugate gradient can converge faster in long, narrow valleys; quasi-Newton and Gauss-Newton methods reduce Hessian-computation demands.Source: §4.1.1 · pp. 22–23

4.1.2. Non-Gradient based Methods

• Derivative-free methods handle noisy, non-differentiable, or black-box objectives, but local search is sensitive to stepsize and Nelder-Mead or Powell methods can still converge to local optima.Powell’s method is generally described as robust and fast, and PCA initialization can improve its performance and robustness for affine registration.Source: §4.1.2 · pp. 23–24

4.2. Discrete

• Discrete registration can be formulated as MRF labeling and optimized through polyhedral or combinatorial methods, message passing, or max-flow and move-making algorithms.Applications include rigid registration by Porchetto et al. (2016) and deformable slice-to-volume models by Ferrante and Paragios (2013) and Ferrante et al. (2015a,b).Source: §4.2 · pp. 24–25
• Discrete methods avoid differentiability, can better explore non-convex objectives, support parallel implementations and explicit resolution control, but their accuracy is bounded by discretization.Ferrante et al. (2015b) reduce label-space dimensionality with a decoupled pairwise model, whereas Ferrante et al. (2015a) use higher-order cliques for stronger global deformation constraints at more challenging inference cost.Source: §4.2 · p. 26

4.3. Miscellaneous

• Heuristics and metaheuristics provide acceptable solutions without optimality guarantees for nonlinear, non-convex, or black-box problems, including genetic algorithms, CMA-ES, and simulated annealing.CMA-ES was combined with gradient descent across transformation parameters, but diverged on rotations when optimizing a 6-DOF prostate MR search space (Tadayyon et al., 2010, 2011).Source: §4.3 · pp. 26–27

Number of Slices

• Multi-slice registration maps several 2D slices to a 3D volume, whereas single-slice registration maps only one slice.Source: §5 · p. 27
• Registering each slice independently enables parallelization but loses contextual relationships among slices (Fei et al., 2003a; Ferrante et al., 2015a; Osechinskiy and Kruggel, 2011b; Yavariabdi et al., 2015).Source: §5 · p. 27
• Sparse multi-slice registration can produce results close to volume-to-volume registration while lowering computation by processing only a few slices.The reviewed advantages also include avoiding volume reconstruction, adapting similarity measures more easily, and enabling parallelization.Source: §5 · p. 28
• Multi-slice registration improves matching robustness by increasing image support, particularly when relative slice positions are known.Configurations include parallel, orthogonal, tracked arbitrary, and temporal slice sets.Source: §5 · p. 28
• Key-slice selection reduces redundant computation in freehand ultrasound by choosing informative slices using image entropy, including within a specified region of interest (Wein et al., 2008a; Olesch et al., 2011; Olesch and Fischer, 2011).Source: §5 · pp. 28–29
• Specialized multi-slice methods include iterative SVR reconstruction, intersection-based SIMC, and particle filtering for multimodal or motion-correction applications (Jiang et al., 2007b; Kim et al., 2008b, 2010a; Nir and Tannenbaum, 2011; Chen et al., 2016).Kainz et al. (2015) reported a fully parallel multi-GPU SVR framework five to ten times faster than the fastest available multi-CPU frameworks.Source: §5 · pp. 29–30

6. Applications

• Applications divide into image fusion and image-guided interventions, and motion correction for volume reconstruction.The latter corrects inter-slice misalignment by mapping individual slices to a reference or iteratively reconstructed volume.Source: §6 · p. 30, §6.2 · p. 32

6.1. Image Fusion and Image Guided Interventions (IGI)

• In image-guided procedures, registration aligns lower-quality intra-operative 2D images with annotated pre-operative CT or MRI, augmenting navigation during surgery.Applications include surgeries, therapies, biopsies, radio frequency ablation, organ tracking, and minimally invasive procedures; tissue shift, breathing, and heart motion make alignment challenging.Source: §6.1 · p. 30, Figure 4 · p. 31
• Automatic slice-to-volume registration methods outperform manual results, demonstrating the importance of automatic registration for image fusion and image-guided interventions (Fr¨uhwald et al., 2009).Applications include fusing pre-registered SPECT with MRI and live-time iMRI for tumor targeting, and fusing 2D fluoroCT with volumetric CT for lesion localization during biopsies.Source: §6.1 · p. 31

6.2. Motion Correction and Volume Reconstruction

• Motion correction reconstructs volumes by iteratively registering slices to anatomical references or volumes updated from current slice-motion estimates.Applications span fMRI, cardiac MRI, fetal imaging, and diffusion tensor imaging, typically assuming each slice is internally coherent while inconsistency occurs between slices.Source: §6.2 · p. 32
• The MSV method estimates independent 6-DOF rigid slice transformations for fMRI motion correction, while later methods jointly regularize motion or use deformable and field-corrected reconstruction extensions.For fetal MRI without an anatomical reference, iterative slice-to-volume registration with scattered-data interpolation or maximum-likelihood minimization reconstructs moving brains, with the latter guaranteeing convergence to a motion-corrected solution.Source: §6.2 · pp. 33–34

7. Validation of Slice-to-Volume Registration Methods

• Dedicated public benchmarks with anatomical landmarks, structure segmentations, and gold-standard 2D-to-3D transformations are needed to enable fair comparison of slice-to-volume registration methods.The MNI BITE dataset is widely used but was not designed for this task, and its landmarks come from reconstructed volumes rather than independent slices.Source: §7 · p. 34

7.1. Type of images

• Clinical images provide realistic validation, whereas phantoms enable controlled measurement; combining both addresses clinical realism and the annotation scarcity of patient data.Digitally simulated phantoms can additionally provide segmentation masks or probability maps, while clinical datasets usually require specialist annotations.Source: §7.1 · p. 35

7.2.1. Distance Between Transformation Parameters

• Transformation-parameter distances quantify registration error when the exact ground-truth mapping is known, but synthetic or bronze-standard transformations are often required because such mappings are rarely available.Synthetic ground truth perturbs known transformations, whereas bronze standards derive transformations from more reliable registration methods or landmarks.Source: §7.2.1 · pp. 35–36

7.2.2. Point-based Registration Error

• Landmark-based target registration error measures corresponding-point distances, but sparse slices can make landmark annotation impossible; Pardasani et al. (2016) retained only slices within 0.3mm of expert landmarks.Voxel-distance alternatives still require correspondences between the slice and volume.Source: §7.2.2 · pp. 36–37

7.2.3. Segmentation-based Metrics

• Segmentation metrics such as Dice quantify overlap and support deformable-registration evaluation, but broad or poorly localized structures may obscure errors; combining overlap with contour distances is recommended.Dice ranges from 0 for no overlap to 1 for complete overlap, while Hausdorff distance and contour mean distance assess contour agreement.Source: §7.2.3 · pp. 37–38

7.2.4. Appearance-based metrics

• Appearance-based validation uses reconstruction RMSD, SNR, sharpness, or modality-dependent similarity measures; sharpness avoids requiring ground-truth reconstructed images.Monomodal cases can use SAD, SSD, or CC, whereas multimodal cases may require MI, NMI, or modality-specific measures such as LC2.Source: §7.2.4 · p. 38
• Quantified expert visual inspection adds complementary knowledge that objective metrics cannot capture, but it is time-consuming and requires human intervention.It can be combined with the other validation strategies.Source: §7.2.4 · pp. 38–39

• The survey classifies prior work by matching criterion, transformation model, optimization method, slice count, application, and validation strategy.Source: §8 · p. 39
• Most methods use rigid registration with iconic matching criteria and continuous optimization, as summarized in Table 1.Source: §8 · p. 39
• Rigid models suit simple slice mapping, but realistic deformation models are needed for organ motion, tissue shifts, and clinically trustworthy non-rigid solutions.Non-rigid models may be geometrically correct yet anatomically unrealistic, motivating physically grounded deformation models.Source: §8 · p. 39
• Multiple slices increase image support, while combining iconic criteria with geometric or sensor-based strategies improves robustness and refines rigid estimates.These combinations can preserve iconic accuracy while handling image changes, discontinuities, and limited support.Source: §8 · pp. 39–40
• Discrete optimization remains underused but may help with many parameters, non-convex energies, or wide search spaces, including deformable and poorly initialized problems.Continuous methods remain effective for basic registration, whereas discrete strategies have shown potential in related biomedical image-analysis problems.Source: §8 · p. 40
• Deep convolutional networks offer a largely unexplored paradigm that may produce faster and more accurate slice-to-volume registration predictions (Hou et al., 2017).Related CNN work learns similarity measures and registration processes (Fischer et al., 2015; Miao et al., 2016; Yang et al., 2016).Source: §8 · p. 40