Source-linked AI summary
Learning a Variational Network for Reconstruction of Accelerated MRI Data
Kerstin Hammernik, Teresa Klatzer, Erich Kobler, Michael P Recht, Daniel K Sodickson, Thomas Pock, Florian Knoll
TL;DR
Accelerated clinical multi-coil MRI reconstruction must recover images from undersampled data while avoiding artifacts, unnatural appearance, costly iterative computation, and manual parameter tuning. The paper addresses this with a variational network that embeds a generalized compressed-sensing model in an unrolled gradient-descent scheme and learns its parameters offline. The resulting reconstructions outperform traditional methods across tested acceleration factors and sampling patterns, preserve untrained pathologies, and reconstruct a slice in 193 ms.
Problem
Accelerated multi-coil MRI reconstruction must address undersampling artifacts, limited image priors, long iterative computation, and hyper-parameter selection.
Method
A variational network embeds a generalized variational compressed-sensing model in an unrolled gradient-descent scheme, learning filters, activation functions, and data-term weights offline.
Results
Variational network reconstructions outperform traditional reconstructions across a wide range of pathologies and tested acceleration factors and sampling patterns.
Takeaways & Limitations
The approach preserves important features and pathologies absent from training data and offers reconstruction speed suited to clinical workflow integration.
Takeaways & Limitations
Total variation's piecewise-constant approximation is unsuitable for the complex structure of MR images, motivating a more general learned regularizer.
Abstract
from arXiv · showhide
Purpose: To allow fast and high-quality reconstruction of clinical accelerated multi-coil MR data by learning a variational network that combines the mathematical structure of variational models with deep learning. Theory and Methods: Generalized compressed sensing reconstruction formulated as a variational model is embedded in an unrolled gradient descent scheme. All parameters of this formulation, including the prior model defined by filter kernels and activation functions as well as the data term weights, are learned during an offline training procedure. The learned model can then be applied online to previously unseen data. Results: The variational network approach is evaluated on a clinical knee imaging protocol. The variational network reconstructions outperform standard reconstruction algorithms in terms of image quality and residual artifacts for all tested acceleration factors and sampling patterns. Conclusion: Variational network reconstructions preserve the natural appearance of MR images as well as pathologies that were not included in the training data set. Due to its high computational performance, i.e., reconstruction time of 193 ms on a single graphics card, and the omission of parameter tuning once the network is trained, this new approach to image reconstruction can easily be integrated into clinical workflow.
INTRODUCTION
This work applies deep learning to MRI image formation, targeting accelerated reconstruction that addresses limitations of conventional compressed sensing and iterative methods. It introduces a trainable variational network that learns reconstruction parameters offline for efficient application to unseen multi-coil data.
- Deep learning is extended from image processing and interpretation to the earlier stage of medical image formation.
- Clinical compressed sensing faces challenges from Cartesian sampling, inadequate sparsifying transforms, long iterative reconstruction times, and hyper-parameter tuning.
- Current iterative reconstructions treat each examination as a new optimization problem without explicitly using expected anatomy or known undersampling-artifact structure.
- The proposed approach shifts optimization from online reconstruction to offline learning of inverse-transform parameters applicable to new data.
- The variational network learns a complete reconstruction procedure for complex-valued multichannel MR data, including parameters otherwise set empirically.
THEORY
The theory formulates accelerated multi-coil MRI reconstruction as a regularized inverse problem and unfolds a learned variational model within gradient descent. The model replaces limited image priors with learned convolutional filters and nonlinear functions while using measured data and coil sensitivities as inputs.
- The reconstruction seeks an image satisfying an undersampled multi-coil MRI forward model, with missing k-space data zero-padded and the problem treated as ill-posed.
- Noisy undersampled data are handled through gradient descent on a least-squares problem, with early stopping used to limit over-fitting.
- A regularization term balances data fidelity against an image prior, generalizing total variation through multiple convolutional filter terms and nonlinear potential functions.
- Total variation supports sharp edges and efficient convex optimization but favors piecewise-constant solutions unsuitable for complex MR image structure.
- Filters, activation functions, and data-term weights vary across gradient-descent steps and are learned from data through back-propagation-based optimization.
- The variational network unfolds iterative reconstruction steps, using raw data, precomputed coil sensitivity maps, and zero-filled initializations as inputs.
METHODS
The variational network unfolds iterative reconstruction into gradient-descent steps whose filters, activation functions, and data-term weights are learned offline. It uses complex-valued multichannel MRI inputs, reference-based training, and fixed trained parameters for efficient reconstruction of unseen data.
- Variational Network Parameters: The variational network unfolds iterative reconstruction into T gradient-descent steps, with each iteration corresponding to one network step.The network receives undersampled k-space data, coil sensitivity maps, and a zero-filled initialization; its inputs and outputs are complex-valued.
- Variational Network Parameters: The learned model includes filter kernels, activation functions, and non-negative data-term weights that vary across gradient-descent steps.Filters are constrained to be zero-mean and unit norm, while activation functions are parameterized by weighted Gaussian radial basis functions.
- Variational Network Training: Training minimizes the similarity loss between VN reconstructions and clean, artifact-free reference images, with parameter gradients computed by back-propagation.The training problem uses an MSE loss on epsilon-smoothed magnitude images and the IIPG optimizer; parameter constraints are enforced by projected gradient methods.
- Variational Network Training: After training, the parameters are fixed and previously unseen k-space data are reconstructed by forward propagation through the VN.This shifts optimization from online reconstruction to offline training.
- Experimental Setup: The clinical knee protocol comprised five 2D turbo spin echo sequences spanning different contrasts, orientations, matrix sizes, and signal-to-noise ratios.The listed sequences included coronal PD, coronal fat-saturated PD, axial fat-saturated T2, sagittal fat-saturated T2, and sagittal PD.
- Experimental Setup: Each experiment used a 10-step VN with 48 real/imaginary filter pairs per iteration, 31 RBFs per activation function, and 131,050 total parameters.The initial reconstruction was zero-filled, and the architecture was kept fixed across experiments.
- Experimental Setup: Training used 200 images from 10 patients for each contrast, sampling pattern, and acceleration factor, while testing used ten separate clinical patients with unrepresented pathology.Fully sampled data were retrospectively undersampled, and gold-standard references were coil-sensitivity-combined fully sampled reconstructions.
- Experimental Setup: The VN was compared with CG SENSE and PI-CS TGV using consistent forward and adjoint operators, while comparator hyperparameters were individually selected by grid search.Experiments varied contrast, orientation, acceleration factor, and sampling pattern, including regular and variable-density random sampling.
RESULTS
Across acceleration factors, sampling patterns, contrasts, and clinical knee cases, the learned VN produced sharper, more natural images with fewer residual artifacts than CG SENSE and PI-CS TGV. Quantitative evaluation supported superior MSE and SSIM performance across the tested cases.
- Acceleration and sampling: The learned VN suppressed undersampling artifacts while retaining sharper and more natural-looking images than CG SENSE and PI-CS TGV at R = 3 and R = 4.At R = 4, PI-CS TGV could appear cartoonish in some regions, whereas the VN maintained more natural image appearance.
- Acceleration and sampling: For R = 3, regular sampling generally produced better intensity homogeneity and sharpness than variable-density random sampling for PI-CS TGV and VN.For R = 4, random sampling improved reconstruction results and, in the displayed cases, reduced residual artifacts or increased sharpness.
- Contrast and SNR: Lower SNR in fat-saturated scans produced more blocky, unnatural PI-CS TGV reconstructions, while the VN still suppressed artifacts and improved image quality.The comparison covered fat-saturated PD-weighted scans at R = 3 and R = 4.
- Quantitative evaluation: The learned VN showed superior performance in MSE and SSIM for all evaluated clinical knee sequences and acceleration factors.Table 1 reports results for five clinical knee datasets, R ∈ {3, 4}, and both regular and variable-density random sampling.
- Clinical protocol: Across a complete R = 4 knee protocol including pathological and pediatric cases, residual artifacts visible in reference reconstructions were not visible in the learned VN images.The cases spanned multiple sequences, patients, ages, genders, and anatomical variations.
DISCUSSION
The VN combines variational methods with deep learning to learn interpretable reconstruction parameters for complex multi-coil MRI, while shifting expensive optimization offline. Its learned filters and potential functions differ from fixed classical priors and support efficient reconstruction across sampling conditions.
- Model and motivation: The VN embeds a variational reconstruction model in a gradient-descent scheme, combining variational methods with deep learning for clinical multi-coil MRI.The architecture addresses complex-valued data, coil sensitivities, and sampling-pattern-specific artifacts.
- Interpretability: Learned filter kernels, activation functions, and potential functions provide an inspectable prior rather than a purely black-box reconstruction.Potential functions can be linked to regularization norms used in traditional compressed sensing.
- Learned priors: The learned potential functions include both functions close to the convex l1 norm and substantial deviations from it.Some learned functions exhibit student-t characteristics or have been associated with image sharpening.
- Learned priors: Larger 11 × 11 filters can capture characteristic backfolding artifacts of Cartesian undersampling more efficiently than simpler hand-designed priors.The discussion contrasts learned filters with the piecewise-constant and piecewise-affine priors of TV and TGV.
- Clinical efficiency: 193 ms per slice is the reported VN reconstruction time, compared with 75 ms for CG SENSE and 11.73 s for PI-CS TGV.Training took approximately four days on a single graphics card, but new data require no additional optimization or parameter selection.
- Sampling behavior: For acceleration R = 3, random sampling did not improve nonlinear reconstructions because coil sensitivities could already fill missing k-space robustly; at R = 4, randomness improved results.The discussion attributes the difference to the balance between parallel-imaging performance and regularization.
CONCLUSION
The paper presents a variational network that learns the complete reconstruction procedure for complex multi-coil MR data from clinical datasets. The approach preserves important features absent from training data, outperforms traditional reconstructions across pathologies, and offers reconstruction speed suitable for clinical workflow.
- Contribution: The VN learns the complete reconstruction procedure and associated model parameters during offline training on clinical patient datasets.The approach is designed for efficient reconstruction of complex multi-coil MR data.
- Clinical performance: The VN preserves important features not represented in the training data and outperforms traditional reconstructions across a wide range of pathologies.The conclusion presents this as evidence of broad reconstruction performance across clinical variation.
- Clinical performance: High reconstruction speed supports integration of the VN approach into clinical workflow.The paper describes this speed as substantial for workflow integration.
APPENDIX A
Appendix A describes the projected-gradient-based IIPG optimizer used to train the highly non-convex network objective. It updates parameters on mini batches with over-relaxation and independently projects constrained parameters.
- Optimizer: The IIPG optimizer solves the highly non-convex network training problem as a variant of projected gradient descent.It is related to the Inertial Proximal Alternating Linearized Minimization algorithm.
- Constraint handling: Independent projections enforce the parameter constraints because the constraints do not depend on one another.The data-term-weight update is clamped at zero to impose non-negativity.
- Constraint handling: Filter-kernel projection begins by subtracting the kernel means before projecting onto the unit-norm constraint.This projection step is part of the constrained parameter update.
- Optimization procedure: Training splits the set S into NB mini batches and updates parameters using a selected step size, epoch count, and initial parameters.The algorithm initializes θ and iterates over epochs and mini batches.
- Optimization procedure: Each iteration first applies epoch-dependent over-relaxation before computing the gradient on the current mini batch.The over-relaxation constant βe provides moderate acceleration.
APPENDIX B
Appendix B derives gradients for the VN loss and its learned parameters through the unrolled reconstruction steps. The derivation covers potential functions, intermediate reconstructions, filter kernels, and data-term weights using back-propagation.
- Gradient derivation: Back-propagation obtains derivatives with respect to the parameter set θt at every gradient step.The parameter set includes learned weights and filter kernels.
- Gradient derivation: The appendix defines the gradients of the loss with respect to the final reconstruction and propagates them through the network's gradient steps.The reconstruction error of each gradient step contributes to the derivative calculation.
- Data-term weights: The appendix explicitly includes derivatives with respect to the data-term weights λt for each gradient step.These derivatives are part of the full parameter-gradient calculation.
- Potential functions: Potential-function derivatives are represented using weighted Gaussian radial basis functions and their derivatives.The appendix also expresses the corresponding matrix-vector relationships.
- Filter kernels: Filter-kernel derivatives are derived by rewriting convolution as matrix-vector multiplication and accounting for rotated kernels induced by transposition.A rotation matrix represents the transpose operation when differentiating with respect to the unrotated kernel.
SUPPLEMENTARY MATERIAL1
The supplementary material presents reconstruction videos for complete imaged volumes from clinical knee MRI sequences under different sampling settings.
- Video 1 shows a complete coronal PD-weighted volume for a 50-year-old male with regular sampling at acceleration R = 4.
- Video 2 shows the same patient and sequence reconstructed with variable-density random sampling at acceleration R = 4.
- Video 3 shows a complete axial fat-saturated T2-weighted volume from a 45-year-old female.