PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Image Registration with Application for MR and TRUS Image Fusion

Pingkun Yan, Sheng Xu, Ardeshir R. Rastinehad, Brad J. Wood

arXiv:1804.11024v2cs.CV

TL;DR

MR-TRUS registration is difficult because multimodal images differ greatly in appearance and correspondence, despite its importance for fusion-guided prostate interventions. AIR-net jointly trains a registration generator and quality-evaluating discriminator, achieving reduced registration error and improved D-scores on clinical data. The method is currently evaluated with randomly generated 2D transformations.

  • Problem

    MR-TRUS registration is challenging because multimodal appearance, noisy ultrasound, differing resolutions, and correspondence variation complicate robust alignment for fusion-guided interventions.

    Source: §1 · p. 1

  • Method

    AIR-net jointly trains a generator that estimates transformations and a discriminator that evaluates whether MR-TRUS image pairs are aligned.

    Source: §1 · pp. 2–3

  • Results

    Mean TRE decreased from 6.11mm to 3.48mm, while D-scores significantly improved after registration and correlated well with TRE.

    Source: §3.2 · p. 7

  • Takeaways & Limitations

    AIR-net provides both registration estimation and quality evaluation, supporting quality checks that can detect potential registration failures in image-fusion guided procedures.

    Source: §1 · p. 3, §4 · p. 7

    The current experiment restricts randomly generated transformations to 2D axial-view rotations and translations with three degrees of freedom.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Robust and accurate alignment of multimodal medical images is a very challenging task, which however is very useful for many clinical applications. For example, magnetic resonance (MR) and transrectal ultrasound (TRUS) image registration is a critical component in MR-TRUS fusion guided prostate interventions. However, due to the huge difference between the image appearances and the large variation in image correspondence, MR-TRUS image registration is a very challenging problem. In this paper, an adversarial image registration (AIR) framework is proposed. By training two deep neural networks simultaneously, one being a generator and the other being a discriminator, we can obtain not only a network for image registration, but also a metric network which can help evaluate the quality of image registration. The developed AIR-net is then evaluated using clinical datasets acquired through image-fusion guided prostate biopsy procedures and promising results are demonstrated.

1 Introduction

MR-TRUS registration is clinically important but difficult because multimodal appearance, noise, resolution, and correspondence vary substantially. AIR-net addresses this challenge with jointly trained networks that estimate transformations and evaluate registration quality.

  • MR-TRUS registration supports fusion-guided prostate interventions, but noisy ultrasound, differing resolutions, and multimodal appearance make robust similarity metrics difficult to define.Source: §1 · p. 1
  • Deep-learning registration methods improved performance, but several approaches lacked direct feedback about registration quality for image-fusion interventions.Source: §1 · pp. 1–2
  • AIR-net uses adversarially trained CNNs to estimate transformation parameters and evaluate registration quality simultaneously.Source: §1 · pp. 2–3
  • The generator directly predicts transformations, while discriminator feedback improves the generator toward transformations close to ground truth.Source: §1 · p. 3
  • The end-to-end framework provides both networks after training, enabling efficient registration estimation and registration-quality evaluation.Source: §1 · p. 3

2 Adversarial Image Registration (AIR)

AIR-net represents paired MR and TRUS volumes as multi-channel 2D images and uses CNN-based generator and discriminator networks. WGAN-style adversarial training combines registration discrimination with a transformation-distance term to improve alignment.

2.1 Generator and Discriminator Networks

  • 2.1 Generator and Discriminator Networks: 3D MR and TRUS volumes are treated as multi-channel 2D images so deeper CNNs can be trained on a single GPU.The authors report that deeper 2D CNNs performed better than shallower 3D CNNs in their experiments.Source: §2.1 · p. 3
  • 2.1 Generator and Discriminator Networks: The generator predicts transformation parameters, while the discriminator uses a sigmoid output to evaluate registration quality.Source: §2.1 · pp. 3–4

2.2 Adversarial Training

  • 2.2 Adversarial Training: WGAN training distinguishes well-aligned image pairs from pairs produced with randomly perturbed transformations.Source: §2.2 · p. 4
  • 2.2 Adversarial Training: The generator loss combines discriminator feedback with Euclidean distance between estimated and randomly created transformations, weighted by α.Source: §2.2 · p. 4
  • 2.2 Adversarial Training: Removing the transformation-distance term remains possible but makes training longer and requires careful parameter tuning.Source: §2.2 · p. 4

3 Experiments

Experiments evaluated AIR-net on clinical MR-TRUS biopsy data using direct transformation estimation, discriminator scoring, and iterative registration from perturbed alignments. Registration substantially improved alignment and reduced TRE, while discriminator scores increased with registration quality.

3.1 Materials and Training

  • AIR-net was trained and validated on 763 clinical MR-TRUS biopsy cases, split into 636 training and 127 validation cases.The data came from NIH and Mount Sinai Hospital and were acquired using an FDA-approved UroNav fusion-guided biopsy system.Source: §3.1 · p. 5
  • The network required about 8 hours of training but estimated a transformation in less than 100ms per image pair during testing.Both generator and discriminator networks could then be used efficiently for iterative registration updates.Source: §3.1 · p. 5

3.2 Experimental Results

  • Evaluation started from randomly perturbed registrations, after which the generator produced new transformations and the discriminator scored the resulting aligned image pairs.TRE was computed before and after registration, and the resampled TRUS volume was paired with MR for discriminator evaluation.Source: §3.2 · p. 5
  • The experiment restricted randomly generated transformations to 2D axial rotation and translation with three degrees of freedom.The authors state that they are extending the method to more general scenarios.Source: §3.2 · p. 5
  • Examples showed AIR-net bringing perturbed MR-TRUS alignments close to expert manual registrations, with improved alignment reflected by higher D-scores.Figure 2 compares perturbed, AIR-net-registered, and expert-manually registered images across three cases.Source: §3.2 · pp. 5–6, Figure 2 · p. 6
  • Mean TRE decreased from 6.11mm to 3.48mm after registration (p <0.01), while D-scores also improved significantly (p <0.01).The quantitative evaluation used TRE and discriminator scores to assess registration improvement and quality evaluation.Source: §3.2 · pp. 6–7, Figure 3 · p. 7

4 Conclusions

The paper presents AIR-net as a GAN-based multimodality registration method that jointly provides registration and quality evaluation. Its clinical relevance is demonstrated by improved registration results, while broader comparison with state-of-the-art methods remains future work.

4 Conclusions

  • AIR-net combines a registration estimator with a quality evaluator to detect potential registration failures in image-fusion guided procedures.The authors identify this joint functionality as useful for warning physicians about potential problems during clinical interventions.Source: §4 · p. 7
  • The authors report that AIR-net effectively performs multimodality registration, but further evaluation against other state-of-the-art methods is planned.The stated future work concerns comparisons of registration performance.Source: §4 · p. 7
Loading 1804.11024v2…