Source-linked AI summary
BIRNet: Brain Image Registration Using Dual-Supervised Fully Convolutional Networks
Jingfan Fan, Xiaohuan Cao, Pew-Thian Yap, Dinggang Shen
TL;DR
Training image-registration networks is challenging because ground-truth deformation fields are lacking and registration can be computationally expensive in high dimensions. BIRNet uses dual guidance during fully convolutional training and reports state-of-the-art performance without parameter tuning.
Problem
Ground-truth deformation fields are difficult to obtain, while image registration can involve computationally expensive high-dimensional processing.
Method
BIRNet is a dual-supervised fully convolutional network using deformation fields and image similarity or difference metrics to guide training.
Results
BIRNet achieves state-of-the-art registration performance without parameter tuning and is reported to have accurate and robust performance.
Takeaways & Limitations
The proposed method is presented as general, fast, accurate, and easy to use.
Takeaways & Limitations
Registration to a fixed template remains an issue identified for future optimization.
Abstract
from arXiv · showhide
In this paper, we propose a deep learning approach for image registration by predicting deformation from image appearance. Since obtaining ground-truth deformation fields for training can be challenging, we design a fully convolutional network that is subject to dual-guidance: (1) Coarse guidance using deformation fields obtained by an existing registration method; and (2) Fine guidance using image similarity. The latter guidance helps avoid overly relying on the supervision from the training deformation fields, which could be inaccurate. For effective training, we further improve the deep convolutional network with gap filling, hierarchical loss, and multi-source strategies. Experiments on a variety of datasets show promising registration accuracy and efficiency compared with state-of-the-art methods.
1. Introduction
Brain deformable registration is difficult because it requires expensive high-dimensional optimization, task-dependent parameter tuning, and lacks ideal ground-truth deformations. BIRNet addresses these challenges with dual supervision and one-pass deformation prediction, achieving reported accuracy and robustness.
- Deformable registration remains challenging because it involves computationally expensive high-dimensional optimization and task-dependent parameter tuning.
- Deep learning does not directly resolve registration because ideal ground-truth deformations are difficult to obtain and manually annotate.
- BIRNet is a fully convolutional network that predicts deformation fields from image appearance without requiring ideal ground-truth deformation labels.
- Its dual guidance combines coarse supervision from existing registration fields with fine supervision from similarity between the template and warped subject image.Fine guidance reduces reliance on potentially inaccurate training deformation fields.
- The end-to-end framework predicts deformation in one pass without parameter tuning and is validated across varied datasets and registration tasks.The authors report accuracy and robustness for the proposed method.
2. Related Works
Prior deformable registration methods commonly use intensity- or feature-based optimization, while learning-based methods predict deformation parameters from image-related representations. These approaches remain constrained by high-dimensional iterative optimization and the difficulty of constructing reference deformations for training.
- Deformable registration methods are commonly categorized as intensity-based or feature-based optimization approaches.
- Unlike linear registration, deformable registration is an often ill-posed, high-dimensional optimization problem.Linear registration globally aligns images, while deformable registration corrects local deformations.
- Most optimization-based methods involve time-consuming iterative optimization, limiting computational efficiency.
- GPU acceleration can substantially speed voxel- or patch-level computation, but iterative optimization and CPU-GPU memory swapping are not fully accelerated.The cited work reports speed increases of more than 10 for some parallelizable operations.
- Learning-based registration predicts deformation parameters using statistical or machine-learning models, including PCA, support vector regression, sparse representation, dictionaries, and boosted trees.
- Recent CNN-based methods still require carefully built reference deformations because ideal ground-truth deformation fields are unavailable.
3. Method
BIRNet predicts brain deformation fields with a hierarchical dual-supervised fully convolutional network. Its method combines deformation-field and image-difference guidance with gap filling, multi-channel inputs, hierarchical losses, and data augmentation.
- Overview: BIRNet uses a U-Net-based regression model to predict deformation fields directly from image appearance.The network processes local 3D image patches and outputs displacement vectors.
- Hierarchical Dual-Supervision: Dual supervision combines deformation-field loss with image-difference loss to reduce reliance on potentially inaccurate training deformation fields.The deformation-field term provides rough guidance, while image-difference guidance refines registration results.
- Hierarchical Dual-Supervision: Hierarchical losses supervise multiple upsampling layers, directly training earlier convolutional layers to improve convergence and reduce over-fitting.The conventional U-Net applies loss only at the final layer, leaving the first half less strongly updated.
- Gap filling: Gap filling inserts additional convolutional layers between contracting and expansion phases to connect low-level and high-level features.The added path synchronizes feature-map representations, making intermediate map C more similar to deformation-related map B and improving registration accuracy.
- Multi-Channel Inputs: Multi-channel inputs concatenate image intensity with difference and gradient maps to provide additional registration information.The paper reports that these feature maps can improve alignment beyond matching similarity in the original intensity image alone.
- Data Augmentation: Data augmentation warps images with different degrees and iteratively includes predicted images during training to overcome over-fitting.The dataset is augmented because it is too small for effective training, with each warped image paired with its respective deformation field target.
4. Experiments
Experiments compare BIRNet with established deformable-registration methods across LPBA40 and four additional brain-image datasets. Results indicate that dual guidance improves accuracy, generalizes without parameter tuning, and produces registrations visually close to the template.
- Experimental setup: BIRNet is compared with Diffeomorphic Demons, FNIRT, and SyN on LPBA40 and four additional brain-image datasets.The additional datasets are IBSR18, CUMC12, MGH10, and IXI30.
- Qualitative and ablation results: BIRNet produces registrations most similar to the template in the highlighted regions, and image-similarity guidance provides useful training guidance beyond deformation-field supervision.The experiments also report better performance for BIRNet than the original U-Net and BIRNet_WOS.
- LPBA40 accuracy: BIRNet outperforms or matches established methods across most LPBA40 ROIs, while BIRNet_WOS performs slightly worse than Diffeomorphic Demons and SyN.On the 30 training subjects, BIRNet performs better on 35 of 54 ROIs and comparably on the other 19.
- LPBA40 validation: 29 of 54 validation ROIs have higher DSC with BIRNet, while the remaining 25 have very similar values to Diffeomorphic Demons and SyN.BIRNet_WOS shows a less than 1.5% average performance drop relative to Diffeomorphic Demons and SyN.
- Cross-dataset generalization: BIRNet performs comparably to fine-tuned SyN and FNIRT on GM and WM DSCs across IBSR18, CUMC12, MGH10, and IXI30 without additional parameter tuning.The model trained on LPBA40 is applied directly to the four datasets.
BIRNet
BIRNet is implemented on a single GPU and compared with CPU and GPU implementations of competing registration algorithms. Its non-iterative inference requires the least computational time for a typical 3D brain image.
- Implementation and speed: BIRNet is implemented in Caffe on a single Nvidia TitanX Pascal GPU.Speed is compared with CPU and GPU implementations of the other methods, with no GPU implementation available for FNIRT.
- Implementation and speed: For a typical 220×220×184 3D brain image, BIRNet requires the least computation time among seven deformable-registration algorithms.The comparison includes CPU and GPU variants of Diffeomorphic Demons and SyN, plus CPU FNIRT and BIRNet.
5. Conclusion
BIRNet addresses the lack of ground-truth deformation fields by combining pre-registered deformation supervision with image-similarity guidance and complementary training strategies. It achieves state-of-the-art registration accuracy without parameter tuning, while remaining fast and practical, though future work must address template transfer and deformation smoothness.
- BIRNet uses pre-registered deformation fields and image similarity or difference metrics to guide training and refine registration results.This dual guidance addresses the lack of ground-truth deformation fields.
- Gap filling, hierarchical supervision, multi-channel inputs, and data augmentation are used to improve registration accuracy.
- BIRNet achieves state-of-the-art registration accuracy without parameter tuning.
- The method is described as general, fast, accurate, and easy to use for practical brain image registration tasks.
- Future work must refine the model for new template images and add a diffeomorphic constraint to improve predicted deformation-field smoothness.The current model registers subjects to a fixed template, and deformation smoothness is supervised by diffeomorphic training samples.