PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning for fast MR imaging: a review for learning reconstruction from incomplete k-space data

Shanshan Wang, Taohui Xiao, Qiegen Liu, Hairong Zheng

arXiv:2012.08931v1eess.IV

TL;DR

Reconstructing MR images from incomplete k-space data is the focus of this survey of deep learning work published from 2016 through June 2020. It reviews data-driven and unrolled approaches, which show promising reconstruction performance and higher acceleration factors, while identifying open challenges.

  • Problem

    The survey addresses reconstruction of MR images from incomplete k-space data.

    Source: §VII · p. 8

  • Method

    The survey reviews deep learning reconstruction approaches, including data-driven mappings from undersampled to fully sampled data and unrolled methods combining physics models with deep learning.

    Source: §VII · p. 8

  • Results

    The surveyed approaches show promising performance; data-driven methods learn mappings from undersampled to fully sampled data, while unrolled methods generally achieve better reconstruction results and higher acceleration factors.

    Source: §VII · p. 8

  • Takeaways & Limitations

    The surveyed methods show big potential for the next generation of fast MR imaging techniques.

    Source: §VII · p. 8

    Open questions include how much training data is needed, optimal network architectures and depth, application-specific configurations, and theoretical support.

    Source: §VI · p. 8

Abstract

from arXiv · show

Magnetic resonance imaging is a powerful imaging modality that can provide versatile information but it has a bottleneck problem "slow imaging speed". Reducing the scanned measurements can accelerate MR imaging with the aid of powerful reconstruction methods, which have evolved from linear analytic models to nonlinear iterative ones. The emerging trend in this area is replacing human-defined signal models with that learned from data. Specifically, from 2016, deep learning has been incorporated into the fast MR imaging task, which draws valuable prior knowledge from big datasets to facilitate accurate MR image reconstruction from limited measurements. This survey aims to review deep learning based MR image reconstruction works from 2016- June 2020 and will discuss merits, limitations and challenges associated with such methods. Last but not least, this paper will provide a starting point for researchers interested in contributing to this field by pointing out good tutorial resources, state-of-the-art open-source codes and meaningful data sources.

II. DEEP LEARNING BASICS

Deep learning uses multilayer neural networks to learn complex patterns from data. Its basic components include feature-extracting convolution, activation, pooling, and fully connected layers.

A. Convolutional layer

  • Convolutional layers apply shared, parameterized filters across inputs to produce feature maps, reducing the number of parameters needed.Weight sharing is motivated by similar structures and features occurring across an image.Source: A. Convolutional layer · p. 2

B. Activation layer

  • Activation functions add nonlinearity to extracted features, helping networks learn complex input-output mappings.Examples include sigmoid, tanh, and ReLU.Source: B. Activation layer · p. 2

C. Pooling

  • Pooling summarizes each input-map rectangle with a value, typically using average, maximum, or sum, and downsamples the map.Because small image variations cause small changes in activation maps, pooling provides some translational invariance.Source: C. Pooling · p. 2

D. Fully-connected layer

  • Fully connected layers connect each neuron to every neuron in the preceding layer and offer a cheap way to add nonlinearity.They are common in multilayer perceptrons and are also used in current deep learning techniques.Source: D. Fully-connected layer · p. 2

E. Batch normalization

Batch normalization adjusts and scales network-layer activations, with benefits for learning speed, stability, and overfitting.

E. Batch normalization

  • Batch normalization adjusts and scales layer activations, normalizing input distributions to accelerate model learning.It can also reduce sensitivity to weight initialization and scale, simplifying tuning and stabilizing learning.Source: E. Batch normalization · p. 2
  • Batch normalization can reduce overfitting through its regularization effect.The passage also reports reduced sensitivity to weight initialization and scale.Source: E. Batch normalization · p. 2
  • By reducing sensitivity to weight initialization and scale, batch normalization simplifies tuning and makes network learning more stable.Source: E. Batch normalization · p. 2

F. Dropout regularization

Dropout regularization randomly removes neurons during training and can be viewed as averaging across sampled neural networks.

F. Dropout regularization

G. Capacity, overfitting and under-fitting

Model capacity, problem complexity, and data size and heterogeneity must be considered together when configuring a model for image reconstruction.

G. Capacity, overfitting and under-fitting

H. Backward propagation

Backward propagation updates network weights using gradients of the loss, distributing output error backward through the layers.

H. Backward propagation

III. MR IMAGE RECONSTRUCTION BASICS

MR reconstruction uses measured k-space data to recover an image; when measurements are subsampled, regularization can address the resulting under-determined problem.

III. MR IMAGE RECONSTRUCTION BASICS

  • The figure depicts image reconstruction from incomplete k-space data.Source: Figure 1
  • MR scanner measurements are modeled as y = Ex + n, with y as k-space data, x as the image, E as the encoding matrix, and n as noise.E can be MF for single-channel imaging or MFS for parallel imaging; M is the sampling mask, F the Fourier transform, and S sensitivity encoding.Source: §III · p. 3
  • If data meet the Nyquist sampling theorem, the image can be directly updated; fully sampled Cartesian data allow direct inverse Fourier transform.Source: §III · p. 3
  • Subsampled data create an under-determined reconstruction problem, for which sparsity or low-rankness constraints can be employed.Compressed sensing is identified as one approach to solving this problem.Source: §III · p. 3
  • Reconstruction objectives combine data fidelity with prior regularization, weighted by λ.The prior may use fixed transforms such as wavelets or total variation, or adaptive methods such as dictionary learning.Source: §III · p. 3

IV. DEEP LEARNING FOR MR RECONSTRUCTION

Deep learning reconstruction uses offline-trained models for online MR image reconstruction, within data-driven end-to-end or physics/model-driven unrolling approaches.

IV. DEEP LEARNING FOR MR RECONSTRUCTION

  • Offline training optimizes model parameters by minimizing a loss between reconstructed and reference images.Source: Figure 2 · p. 3
  • Online reconstruction uses the optimized model to produce a network prediction for regularization or direct reconstruction.Source: Figure 2 · p. 3
  • Deep learning MR reconstruction methods are categorized as data-driven end-to-end learning or physics/model-driven unrolling iterative methods.The former learns a nonlinear mapping from aliased images or incomplete k-space to artifact-free images or full k-space; the latter solves an inverse problem with traditional iterative algorithms.Source: §IV · p. 3

A. Data driven end-to-end deep learning MR image reconstruction

Data-driven end-to-end methods map undersampled inputs to reconstructed outputs, with image-, k-space-, and hybrid-domain approaches offering different reconstruction properties.

A. Data driven end-to-end deep learning MR image reconstruction

  • Image-domain learning targets noise and artifact removal, k-space learning preserves high-frequency detail, and hybrid learning balances these properties.The three subtypes learn image-to-image, k-space-to-image, or k-space-domain mappings.Source: A. Data driven end-to-end deep… · p. 3

1) U-Net

  • U-Net combines encoder and decoder paths with skip connections that merge low-level location and deep semantic information.Reported applications include iterative U-Net learning and a k-space-correction method that uses 29% of k-space data to generate high-quality MR images.Source: A. Data driven end-to-end deep… · p. 4

2) Generative adversarial network (GAN)

  • GAN methods use generator and discriminator networks trained adversarially; reported MRI variants include DAGAN and RefineGAN.DAGAN combines adversarial and content losses; RefineGAN uses cyclic data consistency and is reported superior to state-of-the-art CS-MRI in efficiency and image quality.Source: A. Data driven end-to-end deep… · p. 4

3) ResNet

4) Complex-valued neural network

  • Complex-valued MRI methods use phase information alongside amplitude or represent complex data through real and imaginary parts.Deepcomplex MRI is reported to outperform real-value networks, traditional CS-MRI, and other advanced deep-learning reconstruction algorithms.Source: A. Data driven end-to-end deep… · p. 4

5) Dual-domain neural network

  • Dual-domain networks process both frequency- and image-domain information; KIKI-net alternates k-space and image networks with data consistency operations.DIMENSION combines frequency- and spatial-domain networks and is reported to improve reconstruction results in less time for dynamic MRI.Source: A. Data driven end-to-end deep… · p. 5

7) Hybrid architecture models

  • Hybrid architectures combine structures such as U-Net, residual modules, dense blocks, attention, and GAN components.Examples include SANTIS, which combines adversarial and convolutional mapping with data fidelity and sampling-augmented training.Source: A. Data driven end-to-end deep… · p. 5

B. Physics/Model based unrolling iterative deep learning methods

Physics/model-driven unrolling methods embed MRI physics and regularized priors in learned iterative networks. The section reviews representative architectures and reports reconstruction advantages for several methods.

Variational Network (VN)

  • The variational network learns formulation parameters by unfolding gradient descent and was reported superior to traditional methods across pathologies with faster reconstruction.Its inputs include undersampled k-space data, coil sensitivity maps, and a zero-filling solution.Source: B. Physics/Model based unrolling… · p. 6

ADMM-Net

  • ADMM-Net unrolls ADMM to learn regularization parameters, and its authors reported high MR reconstruction accuracy.Generic-ADMM-CSNet later redesigned and unrolled ADMM for CS imaging and further improved this method type's reconstruction performance.Source: B. Physics/Model based unrolling… · p. 6

MoDL

  • MoDL combines a forward model, data consistency, and a learned CNN prior, with shared denoiser weights across iterations.Compared with direct inversion approaches, its smaller network uses fewer parameters and reduces training-data and training-time demand; weight sharing improves performance over pretrained-denoiser methods.Source: B. Physics/Model based unrolling… · p. 7

ISTA-Net

  • ISTA-Net turns ISTA into an unrolled network using nonlinear transformations for proximal mapping of a sparsity-inducing regularizer.ISTA-Net+ is described as an enhanced version that can further improve CS reconstruction performance.Source: B. Physics/Model based unrolling… · p. 7

PD-Net

  • PD-Net unrolls PDHG for accelerated MR imaging, replacing proximal operators with parameterized operators learned through offline training.Its iterations update primal and dual variables.Source: B. Physics/Model based unrolling… · p. 7

PD-Net and other unrolling optimization models

  • PD-Net achieved better reconstruction results than traditional CS reconstruction and other unfolding iterative methods.Other models include CRNN for cardiac sequences, adaptive de-aliasing for highly undersampled data without explicit sensitivity calculation, LANTERN, and VS-Net.Source: B. Physics/Model based unrolling… · pp. 7–8

V. OPEN SCIENCE AND REPRODUCIBLE WORKS

The survey points readers to publicly posted code and repository-hosted datasets as a starting point for entering deep-learning MR imaging research.

Open resources

  • Researchers can find implementations on GitHub or authors’ websites and datasets through repositories.The survey summarizes code websites and datasets in Table 1 as a starting point for beginners.Source: §V · p. 8
  • The article compiles surveyed fast-MRI studies with open-source code or data in Table 1.The table caption specifies that scope.Source: §V · p. 8
  • A practical entry route is to pair an implementation with the method described in its paper.The survey notes that this combination can form a starting point for beginners.Source: §V · p. 8

VI. CHALLENGES, LIMITATIONS AND FUTURE PERSPECTIVES

The survey contrasts tradeoffs across learning approaches and identifies supervision, workflow integration, and unresolved design questions as challenges and future directions.

Tradeoffs among learning approaches

  • Image-domain end-to-end learning can remove noise and artifacts but may leave images slightly blurry, while k-space learning preserves detail but may retain noise or artifacts.K-space-to-image learning is described as a better tradeoff between these goals.Source: §VI · p. 8
  • End-to-end methods generally need more training data and more complex networks, and their mapping process is less interpretable than physics-based unrolling.The survey also reports longer training time for end-to-end learning and relatively greater robustness of unrolling methods to adversarial samples or attacks.Source: §VI · p. 8

Dependence on fully sampled datasets

  • Most end-to-end MRI methods are supervised and require training labels, while self-supervised methods need no labels and remain a future research direction.The survey also describes semi-supervised and weakly supervised approaches.Source: §VI · p. 8

MRI workflow

  • Some methods already combine reconstruction with segmentation or produce segmentation directly from k-space, and the authors anticipate broader task-driven outputs.They name classification, detection, and diagnosis as possible future tasks.Source: §VI · p. 8

Challenges and future perspectives

  • The field still faces open questions about data needs, network architecture and depth, application-specific configurations, and theoretical support.Suggested directions include small datasets, weak or unsupervised learning, high-dimensional imaging, and multitasking.Source: §VI · p. 8

VII. CONCLUSIONS

The survey concludes that deep learning for reconstruction from incomplete k-space data has promising performance, while emphasizing data-related and theoretical limitations. It anticipates work on smaller datasets, broader imaging tasks, and stronger theoretical explanations.

Conclusions

  • Deep-learning reconstruction from incomplete k-space data has shown promising performance, but data bias and domain shift require attention.The surveyed approaches include data-driven mappings from undersampled to fully sampled data and unrolled methods combining physics models with deep learning.Source: §VII · p. 8
  • The survey identifies large training-data requirements, priors constrained by training-seen data or artifacts, and underdeveloped theoretical explanations as limitations.These limitations qualify the reported potential of the methods.Source: §VII · p. 8

Future perspectives

  • Future directions include small-data and weakly or unsupervised learning, high-dimensional imaging, and multitasking, alongside stronger theoretical explanations and robustness.The authors anticipate closer collaboration among mathematicians, physicians, and imaging scientists.Source: §VII · p. 9
Loading 2012.08931v1…