PaperlayerView exact revision on arXiv

Source-linked AI summary

Recursive Refinement Network for Deformable Lung Registration between Exhale and Inhale CT Scans

Xinzi He, Jia Guo, Xuzhe Zhang, Hanwen Bi, Sarah Gerard, David Kaczka, Amin Motahari, Eric Hoffman, Joseph Reinhardt, R. Graham Barr, Elsa Angelini, Andrew Laine

arXiv:2106.07608v1eess.IVcs.AIcs.CVcs.LG

TL;DR

Lung registration is challenging because of nonlinear deformation over large volume excursions and difficult optimization. RRN uses multiscale features, normalized local cost correlations, and recursive deformation-field refinement, achieving 0.83 mm average TRE on DirLab COPDGene.

  • Problem

    Lung registration is challenging because of assumptions about tissue elasticity, nonlinear deformation over large volume excursions, computational complexity, and non-convex optimization.

    Source: §1 · p. 2

  • Method

    RRN extracts multiscale features, constructs normalized local cost correlation volumes, and recursively generates or refines deformation fields across pyramid levels.

    Source: §1 · pp. 2–3

  • Results

    0.83 mm average TRE on DirLab COPDGene; RRN reports 13% error reduction from the leaderboard’s best result and 89% versus VoxelMorph.

    Source: §4 · p. 8

  • Takeaways & Limitations

    RRN is presented as a general framework that could extend beyond lung scans to brain MRI or liver CT registration.

    Source: §4 · p. 9

Abstract

from arXiv · show

Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, construct normalized local cost correlation volume and recursively refine volumetric deformation vector fields. RRN achieves state of the art performance for 3D registration of expiratory-inspiratory pairs of CT lung scans. On DirLab COPDGene dataset, RRN returns an average Target Registration Error (TRE) of 0.83 mm, which corresponds to a 13% error reduction from the best result presented in the leaderboard. In addition to comparison with conventional methods, RRN leads to 89% error reduction compared to deep-learning-based peer approaches.

1 Introduction

Lung registration is challenging because it involves nonlinear deformation over large volume excursions and difficult optimization. RRN combines multiscale features, local cost correlations, and recursive deformation-field refinement.

1 Introduction

  • Lung deformation fields can support local biomechanical measurements, and registration may have clinical utility for treatment planning.Source: §1 · p. 1
  • Lung registration is challenged by assumptions about tissue elasticity, nonlinear deformation over large volume excursions, computational complexity, and non-convex optimization.Source: §1 · p. 2
  • RCN recursively predicts velocity fields with stacked CNNs, but its parameter count grows linearly and it requires subnetwork pretraining.Source: §1 · p. 2
  • RRN combines pyramidal features, normalized local cost correlation volumes, and lightweight estimators that generate or refine deformation fields across pyramid levels.The introduction presents these as the network’s three main components.Source: §1 · pp. 2–3

2 Methods

RRN replaces image pyramids with CNN-generated feature pyramids and recursively refines deformation fields across scales. Its moving- and fixed-image pyramids use siamese networks with shared weights.

2.1 Recursive Refinement Network (RRN)

  • RRN uses feature pyramids, normalized local cost correlations, and recursive refinement to produce more accurate high-resolution deformation fields.Source: §2.1 · p. 3

Feature Pyramid Extractor

  • The moving and fixed images each produce a four-level feature pyramid through siamese networks with shared weights.Source: Feature Pyramid Extractor · p. 3
  • Each pyramid level downsamples features by a factor of two using a stride-2 convolution followed by two stride-1 convolutions; channel counts are 16, 32, 64, and 96.Leaky ReLU is used because gradients may be sparse.Source: Feature Pyramid Extractor · p. 3

Feature Spatial Transformation

At each non-bottom pyramid level, the moving features are warped using the upsampled deformation field from the next coarser level. Local cost correlation compares warped-moving and fixed features within a neighborhood.

Feature Spatial Transformation

  • At each non-bottom level, the moving features are warped using the 2× upsampled deformation field predicted at the next coarser level.Trilinear interpolation allows gradients to flow to both the current-level features and the deformation field.Source: Feature Spatial Transformation · p. 4

Local Cost Correlation Volume

  • A local cost correlation volume stores similarity measures between warped-moving and fixed-image features within a radius r, avoiding a prohibitive all-pairs volume.The paper describes these volumes as more discriminating for deformation-field estimation than concatenated warped features.Source: Local Cost Correlation Volume · p. 4

Cost Correlation Volume Normalization

The section motivates cost-volume normalization by reported feature vanishing at higher levels when intermediate deformation fields are unsupervised. It normalizes correlations at each feature level using the mean and standard deviation.

Cost Correlation Volume Normalization

Initial and Intermediate Field Estimator

The field estimators predict or refine deformation vector fields across pyramid levels, passing contextual features between levels.

Initial and Intermediate Field Estimator

Final Field Estimator

A separate large-receptive-field network predicts the final high-resolution deformation vector field.

Final Field Estimator

  • A separate network with a large receptive field predicts the final high-resolution deformation vector field instead of interpolating a low-resolution field with B-splines.Source: Final Field Estimator · p. 5
  • The final estimator is a seven-layer dilated convolutional network with 3 × 3 × 3 kernels and dilation rates of 1, 2, 4, 8, 16, 1, and 1.Source: Final Field Estimator · p. 5

2.2 Loss Function

The unsupervised loss combines image similarity and deformation regularization, with local cross correlation measuring similarity over voxel-centered patches.

2.2 Loss Function

  • The loss combines normalized local cross correlation for image similarity with total variation regularization favoring piecewise-smooth deformation fields, weighted by λ.Source: §2.2 · p. 5

Local Cross Correlation

  • Local cross correlation sums weighted cross-correlation coefficients computed over patches centered at each voxel.The correlation uses local means of the fixed and warped images.Source: Local Cross Correlation · p. 5

Total Variation

Total variation regularization is preferred to an L2 smoothness penalty because lung-lobe sliding can produce nonsmooth transitions in the deformation field.

Total Variation

  • Total variation is preferred to L2 regularization because sliding between lung lobes can produce nonsmooth transitions in the deformation field.Source: Total Variation · p. 5
  • The equation uses e_i to denote standard basis vectors of R.Source: Total Variation · p. 5

3 Experiments & Results

Experiments evaluate expiration-to-inspiration lung CT registration on DirLab COPDGene and compare RRN with classic and learning-based approaches. The ablation examines feature pyramids and cost correlation volumes against VoxelMorph.

3.1 Data & Preprocessing

  • The ten DirLab COPDGene cases contain expiration–inspiration CT pairs and 300 annotated landmarks each; scans were cropped, resampled, and intensity-clipped before training.The scans were resampled to 256 × 256 × 256 voxels and clipped to [-1000 -100] HU.Source: §3.1 · p. 6

3.2 Results using TRE

  • RRN is best on 6 of 10 cases, otherwise beaten by pTV; RRN and pTV do not use lung masking, unlike the other listed methods.The authors also report better standard deviation and high accuracy for RRN, which was optimized over the whole dataset.Source: §3.2 · pp. 6–7, Table 1 · p. 7

3.3 Ablation Experiments

  • Compared with VoxelMorph, registration performance improves with feature pyramids and then cost volume, while RRN uses about 12 GB GPU memory versus 22 GB.The ablation replaces the cost correlation volume with concatenated warped and fixed features when testing its impact.Source: §3.3 · p. 7, Table 2 · p. 8

4 Conclusion

The conclusion introduces RRN for unsupervised lung CT registration using multiscale pyramidal decomposition and cost volume metrics. On DirLab COPDGene, it reports 0.83 mm average TRE and lower error than the stated leaderboard and VoxelMorph comparisons.

4 Conclusion

  • RRN achieves 0.83 mm average TRE on DirLab COPDGene, with 13% error reduction from the leaderboard’s best result and 89% error reduction versus VoxelMorph.The authors also report the least TRE among recent deep-learning frameworks on the same dataset and lower memory requirements than VoxelMorph.Source: §4 · p. 8
  • RRN performs unsupervised lung CT registration using multiscale pyramidal decomposition and cost volume metrics.The approach is described as similar to recent optical-flow methods.Source: §4 · p. 8
  • The authors describe RRN as not limited to one image type or anatomy and identify brain MRI and liver CT registration as possible extensions.These applications are proposed possibilities, not reported evaluations in this conclusion.Source: §4 · p. 9

Related papers

Loading 2106.07608v1…