Source-linked AI summary
End-to-End Variational Networks for Accelerated MRI Reconstruction
Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C. Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, Patricia Johnson
TL;DR
The paper addresses reconstruction of undersampled multi-coil MRI, where conventional sensitivity-map estimation limits accelerated imaging. It extends variational networks by learning sensitivity maps and reconstruction fully end-to-end, achieving higher-fidelity and state-of-the-art results on brain and knee fastMRI data.
Problem
Undersampled multi-coil MRI reconstruction remains challenging because traditional sensitivity-map estimation can be inaccurate, especially with limited ACS data and high acceleration.
Method
The method extends variational networks by learning sensitivity maps and the reconstruction process jointly from data in a fully end-to-end model.
Results
The model yields higher-fidelity reconstructions and new state-of-the-art results for both brain and knee MRIs on fastMRI.
Takeaways & Limitations
Learning sensitivity maps within the reconstruction network addresses a limitation of conventional multi-coil reconstruction and supports strong results across both anatomies.
Takeaways & Limitations
Clinical validation remains necessary because the quantitative measures provide only a rough estimate and may miss subtle, clinically important details.
Abstract
from arXiv · showhide
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (parallel imaging) and acquiring fewer samples than necessary for traditional signal processing methods (compressed sensing). While the combination of these methods has the potential to allow much faster scan times, reconstruction from such undersampled multi-coil data has remained an open problem. In this paper, we present a new approach to this problem that extends previously proposed variational methods by learning fully end-to-end. Our method obtains new state-of-the-art results on the fastMRI dataset for both brain and knee MRIs.
1 Introduction
MRI is diagnostically valuable but slow, motivating parallel imaging and compressed sensing; this paper learns the reconstruction process end-to-end and reports higher-fidelity, state-of-the-art results.
- MRI’s slow speed limits its utility, while shorter scans could reduce costs and accommodate patients unable to remain still.
- Parallel imaging combines multiple receiver-coil views, while compressed sensing reconstructs images from undersampled measurements.
- Existing optimization-inspired neural networks assume the forward process is fully known, which is unrealistic for multi-coil reconstruction.
- The proposed method learns the forward process jointly with reconstruction, avoiding the fully known-forward-process assumption.
- Experiments report higher-fidelity reconstructions and new state-of-the-art results for both knee and brain MRI.
- The authors extend variational networks to learn completely end-to-end, explore architectures and representations, and evaluate on fastMRI brain and knee datasets.
2 Background and Related Work
Accelerated multi-coil MRI acquires undersampled k-space, but reconstruction must address aliasing and imperfect sensitivity-map estimation. Variational networks unroll optimization steps while learning update parameters, and this work learns sensitivity maps within the network.
- 2.1 Accelerated MRI acquisition: MRI forms images by applying an inverse Fourier transform to frequency-domain k-space measurements acquired by receiver coils.
- 2.1 Accelerated MRI acquisition: Each receiver coil measures Fourier-transformed anatomy modulated by its position-dependent sensitivity map, with measurement noise.
- 2.1 Accelerated MRI acquisition: Undersampling k-space accelerates acquisition but causes aliasing artifacts when reconstructed naively with an inverse Fourier transform.
- 2.1 Accelerated MRI acquisition: Parallel imaging estimates coil sensitivity maps from fully sampled low-frequency ACS data, but insufficient ACS size limits maximum acceleration.
- 2.2 Compressed Sensing for Parallel MRI Reconstruction: Compressed sensing reconstructs images from fewer k-space measurements by enforcing sparsity through an optimization problem solved with iterative gradient updates.
- 2.3 Deep Learning for Parallel MRI Reconstruction: VarNet models each layer after a gradient-update step, using a CNN and learning both its parameters and the update coefficients from data.
- 2.3 Deep Learning for Parallel MRI Reconstruction: Traditional sensitivity maps degrade VarNet performance at high acceleration, so the proposed model predicts them from data as part of the network.
3 End-to-End Variational Network
E2E-VarNet reconstructs undersampled multi-coil MRI by refining intermediate k-space representations through repeated cascades and learning sensitivity maps within the network. The final k-space is transformed into an image using inverse Fourier transformation and root-sum-squares reduction.
- E2E-VarNet: E2E-VarNet takes masked multi-coil k-space as input and applies repeated refinement cascades in k-space.The method uses k-space intermediate quantities rather than image-space quantities, distinguishing it from the original variational network formulation.
- Cascades: Each cascade performs a refinement step analogous to gradient descent, with a learned refinement module operating on the intermediate k-space.The refinement module is defined through the k-space update obtained by applying the forward encoding operations to the image-space update.
- Cascades: The refinement module combines multi-coil data into one complex image, applies a CNN, and maps the result back to multi-coil k-space.In the experiments, the CNN is implemented as a U-Net; applying it after coil combination allows the same network to handle different numbers of coils.
- Learned sensitivity maps: The SME module estimates sensitivity maps as part of the reconstruction network rather than receiving them as externally computed inputs.It processes ACS-centered coil data with a CNN and normalizes the estimates so the required sensitivity-map property is satisfied.
- Image reconstruction and training: The final k-space representation is converted to an image with an inverse Fourier transform followed by root-sum-squares reduction for each pixel.All network parameters, including the SME and cascade CNN parameters and the η_t values, are learned by minimizing negative SSIM against the ground-truth image.
4 Experiments
The experiments compare progressively stronger variational-network designs on fastMRI knee data, showing benefits from U-Nets, k-space intermediates, and learned sensitivity maps across sampling masks. E2E-VN also produces good reconstructions with few ACS lines and outperforms published fastMRI leaderboard models for brain and knee MRI.
- Experimental setup: The study evaluates VN, VNU, VNU-K, and E2E-VN on the multicoil fastMRI knee dataset.The models validate architectural and representation choices under undersampling.
- Experimental setup: E2E-VN contains about 30M parameters, including an additional 0.5M-parameter sensitivity-map estimation module.The other non-baseline models use 12 cascades and about 29.5M parameters.
- Results: At 8× acceleration, VN images contain severe artifacts in qualitative comparisons with ground truth and E2E-VN.Figure 2 compares GT, VN, and E2E-VN reconstructions.
- Results: VNU, VNU-K, and E2E-VN successively improve reconstruction quality over VN, with gains attributed to U-Nets, k-space intermediates, and learned sensitivity maps.The reported ordering is consistent across mask types and mask parameters.
- Significance of learning sensitivity maps: Learning sensitivity maps improves SSIM across equispaced mask settings, with larger gains when fewer low-frequency lines are sampled.The experiments examine reconstruction quality at 4× and 6× acceleration.
- Significance of learning sensitivity maps: E2E-VN produces good reconstructions even with a small number of ACS lines, overcoming degradation associated with poor sensitivity-map estimates.The result addresses a limitation of classical sensitivity-map estimation at high acceleration.
- Results: On fastMRI test datasets, E2E-VN outperforms all other models published on the leaderboard for both brain and knee MRI.These results use training on the training and validation sets for 100 epochs.
5 Conclusion
The paper introduces End-to-End Variational Networks for multi-coil MRI reconstruction by learning sensitivity maps and the reconstruction process jointly. Architecture exploration supports new state-of-the-art results on brain and knee MRI, while clinical validation remains necessary because quantitative measures only roughly estimate reconstruction quality.
- Conclusion: E2E-VarNet learns sensitivity maps within the network to address the incompletely known forward process in multi-coil MRI.The method extends variational networks with fully end-to-end learning.
- Conclusion: Architecture and intermediate-representation exploration enabled new state-of-the-art results on both brain and knee MRIs.The conclusion links these design choices with the reported outcome.
- Limitations: Quantitative reconstruction measures provide only rough quality estimates, so rigorous clinical validation is required before clinical use.Subtle details in small image regions may be clinically important.
6 Supplementary Materials
The paper examines dithering as post-processing for MRI reconstructions, adding brightness-adaptive Gaussian noise to improve perceived naturalness without obscuring dark regions. Examples across brain and knee images show that dithered reconstructions look more natural, particularly at higher accelerations.
- 6.1 Dithering as post-processing: The SSIM loss and other common losses can produce overly smooth reconstructions despite preserving diagnostic content.The authors motivate dithering as a way to enhance perceived sharpness rather than restore diagnostic information.
- 6.1 Dithering as post-processing: Dithering adds zero-mean Gaussian noise whose local standard deviation scales with the square root of median brightness in an 11 × 11-pixel patch.The image is first normalized by maximum pixel intensity, and the noise scale is controlled by σ.
- 6.1 Dithering as post-processing: The dithered images look more natural, especially at higher accelerations.Examples with and without noise are shown in Figs. 4–7.
- 6.1 Dithering as post-processing: At 4× acceleration, brain examples compare ground truth, reconstruction, and reconstruction with added noise.The figure presents the three image conditions from left to right.
- 6.1 Dithering as post-processing: At 8× acceleration, brain examples compare ground truth, reconstruction, and reconstruction with added noise.The figure uses the same left-to-right comparison at a higher acceleration factor.
- 6.1 Dithering as post-processing: Knee examples similarly show ground truth, reconstruction, and reconstruction with added noise at 4× and 8× acceleration.These examples extend the visual comparison beyond brain MRI.