Source-linked AI summary
Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Network
Chun-Mei Feng, Huazhu Fu, Shuhao Yuan, Yong Xu
TL;DR
High-resolution MRI can be difficult to obtain, while existing approaches do not fully explore how multi-contrast features interact across stages. MINet integrates hierarchical features from different contrasts to guide target-image super-resolution, and the paper reports superior results to state-of-the-art multi-contrast methods across fastMRI and clinical datasets.
Problem
Obtaining high-resolution MRI is difficult, and existing methods do not explore interactions between different contrasts at multiple stages despite their complementary information.
Source: §1 · pp. 1–2Method
MINet learns multi-stage features for different contrasts and integrates them by similarity to form enriched representations that guide target-contrast restoration.
Source: §1 · p. 2, §2.1 · p. 3, Multi-Stage Feature Representation · p. 4Results
MINet outperforms current state-of-the-art methods in experiments on fastMRI and clinical datasets.
Source: §1 · p. 3Takeaways & Limitations
Abstract
from arXiv · showhide
Super-resolution (SR) plays a crucial role in improving the image quality of magnetic resonance imaging (MRI). MRI produces multi-contrast images and can provide a clear display of soft tissues. However, current super-resolution methods only employ a single contrast, or use a simple multi-contrast fusion mechanism, ignoring the rich relations among different contrasts, which are valuable for improving SR. In this work, we propose a multi-stage integration network (i.e., MINet) for multi-contrast MRI SR, which explicitly models the dependencies between multi-contrast images at different stages to guide image SR. In particular, our MINet first learns a hierarchical feature representation from multiple convolutional stages for each of different-contrast image. Subsequently, we introduce a multi-stage integration module to mine the comprehensive relations between the representations of the multi-contrast images. Specifically, the module matches each representation with all other features, which are integrated in terms of their similarities to obtain an enriched representation. Extensive experiments on fastMRI and real-world clinical datasets demonstrate that 1) our MINet outperforms state-of-the-art multi-contrast SR methods in terms of various metrics and 2) our multi-stage integration module is able to excavate complex interactions among multi-contrast features at different stages, leading to improved target-image quality.
1 Introduction
MRI can be difficult to acquire at high resolution, and existing SR methods leave complementary multi-contrast information and its stage-wise interactions underexplored. MINet uses those interactions to guide target-contrast restoration, with experiments reporting superior results to current state-of-the-art methods.
1 Introduction
- MRI's complex acquisition process makes obtaining high-resolution images difficult, motivating super-resolution as a post-processing approach.Earlier SR approaches include interpolation, iterative deblurring, low-rank, dictionary-learning, and deep-learning methods.Source: §1 · pp. 1–2
- Existing multi-contrast SR methods do not explore interactions between contrasts at different stages, a gap identified as critical for target-contrast restoration.Different contrasts provide complementary information, and some are easier to acquire than others.Source: §1 · p. 2
- MINet models correlations among hierarchical stages of different contrasts, weighting and fusing complementary features with target features to guide their learning.Its contributions include exploring dependencies among fused features and improving their representation ability.Source: §1 · pp. 2–3
- Experiments on fastMRI and clinical datasets report that MINet obtains superior results to current state-of-the-art methods.The supplied introduction gives no metric values for this comparison.Source: §1 · p. 3
2 Methodology
MINet uses HR T1WI to guide SR of LR T2WI through features learned at multiple stages. Its integration module models relationships across contrasts and stages to form enriched features for reconstruction.
2.1 Overall Architecture
- HR T1WI guides LR T2WI super-resolution through independent convolutional branches, with residual groups learning features across stages and progressive fusion of T1WI features.The LR T2WI representation is upscaled by sub-pixel convolution; the model retains intermediate features rather than using only the final stage.Source: §2.1 · p. 3, Multi-Stage Feature Representation · p. 4
- MINet concatenates intermediate features from all stages, then applies a multi-stage integration module to model relations among them and produce enriched features.The module is introduced because concatenating features alone treats them independently and does not fully explore their relations.Source: Multi-Stage Feature Representation · p. 4
- A channel-spatial attention module uses a 3D convolution to generate an attention map that modulates target features, while a residual layer preserves original feature information.The described attention uses joint channel and spatial features.Source: Multi-Contrast Feature Enhancement · p. 4
- A convolutional reconstruction layer produces the restored target image; auxiliary T1WI reconstruction also contributes to the L1 loss, weighted alongside T2WI reconstruction.The T1WI reconstruction procedure is used only as an auxiliary.Source: Image Reconstruction · p. 5, Loss Function · p. 5
2.2 Multi-Stage Integration Module
- The integration module computes pairwise similarities across contrast and stage features, uses normalized affinities to summarize them, and forms a holistic representation.Its attention weights emphasize informative T2WI stages and T1WI stages that can guide T2WI SR.Source: §2.2 · pp. 5–6
3 Experiments
Experiments compare MINet with single- and multi-contrast SR methods across fastMRI and two clinical datasets, using SSIM and PSNR at multiple enlargement scales. The reported quantitative and visual results favor MINet, while ablations examine the roles of auxiliary contrast and feature interactions.
Datasets and Baselines
- Evaluation uses fastMRI PDWI to guide FS-PDWI SR and SMS and uMR T1WI to guide T2WI SR, with clinical patients split 7:1:2 and all baselines retrained using default settings.MINet is trained for 50 epochs with Adam at a 1e-5 learning rate; the clinical cohorts use patient-wise splits.Source: Datasets and Baselines · p. 6, Experimental Setup · p. 6
Quantitative Results
- 31.769 dB PSNR and 0.709 SSIM were reached on fastMRI at 2×, up from 29.484 dB and 0.682 for Lyu et al.; MINet reports the best results across all datasets.The quantitative comparison covers three datasets and 2× and 4× enlargement.Source: Table 1 · p. 7, Quantitative Results · p. 7
Qualitative Evaluation
- On fastMRI and SMS at 2× and 4×, MINet shows fewer chessboard effects and structural losses, and error-map texture indicates worse restoration when more obvious.The authors report robustness across enlargement scales and datasets.Source: Qualitative Evaluation · p. 7, Figure 3 · p. 8
Ablation Study
- The full MINet performs best in the SMS 2× ablation; removing auxiliary contrast performs worst, and removing FInt is less optimal than removing FAtt.The comparison attributes value to both single-contrast feature enhancement by FAtt and fused-feature enhancement by FInt.Source: Ablation Study · pp. 7–8, Table 2 · p. 8
4 Conclusion
MINet uses complementary-contrast guidance for target-contrast MRI super-resolution, exploring modality-specific properties and interactions between contrasts. Its multilevel feature modules are presented as a potential guide for further research on these interactions.
4 Conclusion
- MINet restores a target-contrast image with guidance from a complementary contrast.Source: §4 · p. 9
- The network explores modality-specific properties alongside interactions between multi-contrast images.Source: §4 · p. 9
- FInt and FAtt are applied collaboratively to multilevel and multi-contrast features to capture more informative target-image features.Source: §4 · p. 9
- The work presents its approach as a potential guide for further research into interactions between multi-contrast MRI images.Source: §4 · p. 9