Source-linked AI summary
XMorpher: Full Transformer for Deformable Medical Image Registration via Cross Attention
Jiacheng Shi, Yuting He, Youyong Kong, Jean-Louis Coatrieux, Huazhong Shu, Guanyu Yang, Shuo Li
TL;DR
DMIR requires backbones that represent correspondences between paired moving and fixed images, but existing networks and transformers primarily focus on single-image features. XMorpher uses dual parallel U-shaped transformers with CAT-based cross-attention and local multi-size windows, achieving a reported 2.8% DSC improvement over Voxelmorph.
Problem
Existing single-image networks and transformers are limited for DMIR because they do not directly represent moving-fixed correspondences between paired images.
Method
XMorpher uses dual parallel feature-extraction networks that exchange multi-level information through CAT cross-attention and local multi-size windowed correspondence computation.
Results
2.8% higher DSC and 0.87% lower Jacobian matrix were reported for VM-XMorpher than Voxelmorph.
Takeaways & Limitations
XMorpher provides an effective representation of paired-image correspondences for fine registration and supports refined details and topology preservation relative to fusion-first networks.
Abstract
from arXiv · showhide
An effective backbone network is important to deep learning-based Deformable Medical Image Registration (DMIR), because it extracts and matches the features between two images to discover the mutual correspondence for fine registration. However, the existing deep networks focus on single image situation and are limited in registration task which is performed on paired images. Therefore, we advance a novel backbone network, XMorpher, for the effective corresponding feature representation in DMIR. 1) It proposes a novel full transformer architecture including dual parallel feature extraction networks which exchange information through cross attention, thus discovering multi-level semantic correspondence while extracting respective features gradually for final effective registration. 2) It advances the Cross Attention Transformer (CAT) blocks to establish the attention mechanism between images which is able to find the correspondence automatically and prompts the features to fuse efficiently in the network. 3) It constrains the attention computation between base windows and searching windows with different sizes, and thus focuses on the local transformation of deformable registration and enhances the computing efficiency at the same time. Without any bells and whistles, our XMorpher gives Voxelmorph 2.8% improvement on DSC , demonstrating its effective representation of the features from the paired images in DMIR. We believe that our XMorpher has great application potential in more paired medical images. Our XMorpher is open on https://github.com/Solemoon/XMorpher
1 Introduction
Existing DMIR backbones and transformers mainly represent single images, limiting their ability to extract and match moving-fixed correspondences. XMorpher addresses this with dual feature extraction and cross-attention-based fusion for multi-level alignment.
- Motivation: DMIR backbones extract moving and fixed features and match them to obtain correspondences for fine registration.Registration places moving images in the fixed-image coordinate system, facilitating image comparison for diagnosis.
- Limitations of existing methods: Fusion-first methods mix feature extraction and matching, distorting mixed regions and weakening one-to-one correspondence identification.These methods adapt paired inputs to single-image networks, which contributes to weak alignment and inefficient feature representation.
- Limitations of existing methods: Existing transformers use self-attention within one image and lack designs for moving-fixed correspondence between paired images.Their attention mechanism emphasizes internal image relevance while ignoring cross-image correspondences needed for registration.
- XMorpher: XMorpher uses dual parallel feature-extraction subnetworks whose features are progressively fused and matched through cross attention.This design targets effective moving-fixed correspondence representation and fine-grained multi-level semantic information.
- XMorpher: XMorpher introduces Cross Attention Transformer blocks to compute mutual relevance between moving and fixed features for efficient fusion.The architecture is presented as a full transformer backbone for paired-image DMIR.
2 Methodology
XMorpher is a full transformer registration backbone built from dual U-shaped networks that exchange features through cross-attention fusion. Its architecture includes CAT-based fusion and multi-size window partitioning for local correspondence.
- Architecture: XMorpher extracts and matches moving-fixed features using dual U-shaped subnetworks, CAT blocks, and multi-size window partitioning.The representation produces a deformation vector field and a finely warped image through spatial transformation.
- Architecture: The overall design forms an X-shaped network by exchanging information between parallel feature-extraction paths through cross-attention fusion modules.The supplied architecture description identifies dual U-shape networks and cross-attention-based feature fusion as its central components.
- Feature fusion: CAT blocks fuse two input feature tokens from different subnetworks while incorporating attention information.The feature-fusion module contains two CAT blocks that share parameters for mutual correspondences.
I. X-shape architecture with parallel communicating feature extrac-
XMorpher communicates between parallel U-shaped feature extractors through repeated cross-attention fusion. Its CAT blocks use local multi-size windows and W-MCA to learn efficient correspondences between base and searching features.
- X-shape architecture: Parallel U-shaped networks exchange cross-image information through repeated feature-fusion modules while extracting features at multiple semantic levels.The networks communicate k times within a fusion module, supporting frequent exchange before deeper processing.
- Local correspondence: CAT partitions base and searching features into window sets of equal count but different window sizes before attention calculation.Base windows provide queries, while searching windows provide keys and values for local correspondence computation.
- Local correspondence: Window-based cross attention limits computation to local regions, avoiding large-span searches for precise deformable correspondences.The design reflects deformable registration’s focus on local voxel displacement and improves computational efficiency.
- Local correspondence: Searching-window dimensions are scaled by α, β, and γ relative to base-window dimensions.The relation is hse=α·h, wse=β·w, and dse=γ·d; sliding windows preserve equal window-set counts.
- W-MCA: W-MCA maps base-window queries and searching-window key-value pairs to outputs using compatibility weights over values.It uses multi-head attention, dot products, and softmax to compute mutual correspondences within the paired windows.
3 Experiment
Experiments on heart registration evaluated XMorpher under unsupervised and semi-supervised strategies, showing strong quantitative, visual, and ablation results.
- Experimental setup: The experiments used MM-WHS 2017 and ASOCA heart-registration data to construct unsupervised and semi-supervised training pairs.The training set included 500 labeled-unlabeled pairs and 9,900 unlabeled-unlabeled pairs.
- Quantitative comparison: 83.0% DSC was achieved by VM-XMorpher, with the top-ranked Jacobian-matrix performance in the comparison.VM-XMorpher was 2.8% higher on DSC and 0.87% lower on the Jacobian matrix than Voxelmorph.
- Quantitative comparison: XMorphers achieved state-of-the-art DSC under both unsupervised and semi-supervised strategies and top-ranked Jacobian-matrix performance.The Jacobian metric is reported as |Jφ| ≤0 (%).
- Visual comparison: Visual comparisons reported advantages for XMorpher in both unsupervised and semi-supervised strategies.The unsupervised results specifically reported better boundary recognition, neighboring-region resolution, and smoother deformation grids.
- Ablation study: Removing cross attention reduced DSC by 1.5 percent, while window-size ablations showed an inverse variation between DSC and Jacobian matrix.The ablation also reported better Jacobian-matrix performance for VM-XMorpher than the other models.
4 Conclusion
XMorpher is presented as a full transformer network for representing multi-level correspondence between moving and fixed images efficiently in deformable registration.
- Conclusion: XMorpher represents multi-level correspondence between moving and fixed images with a full transformer network and window-based CAT blocks.The authors report outstanding DMIR performance under different training strategies and application prospect in diagnosis and treatment.
- Conclusion: The work concludes that XMorpher has application potential in diagnosis and treatment.