Source-linked AI summary
A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction
Jo Schlemper, Jose Caballero, Joseph V. Hajnal, Anthony Price, Daniel Rueckert
TL;DR
MRI acquisition is slow, and undersampling creates an ill-defined reconstruction problem requiring image priors. The paper addresses this with a deep cascade of CNNs incorporating data consistency for Cartesian undersampled 2D cardiac MR images. CNN reconstruction outperforms DLMRI across reported quality measures and reconstructs each slice in 23 ± 0.1 milliseconds, enabling real-time applications.
Problem
MRI acquisition is slow, while undersampled k-space measurements make image reconstruction underdetermined and ill-defined.
Method
A deep cascade of CNNs uses learned image reconstruction with interleaved data consistency stages for undersampled MR images.
Results
CNN reconstruction consistently outperformed DLMRI for 3-fold and 6-fold acceleration, with each slice reconstructed in 23 ± 0.1 milliseconds on a GeForce GTX 1080.
Takeaways & Limitations
The CNN approach reconstructs most anatomical structures more accurately at very aggressive undersampling rates and is fast enough for real-time applications.
Takeaways & Limitations
The evaluation simulated acquisition from SENSE-reconstructed data rather than using surface-coil sensitivity maps directly for parallel-imaging reconstruction.
Abstract
from arXiv · showhide
The acquisition of Magnetic Resonance Imaging (MRI) is inherently slow. Inspired by recent advances in deep learning, we propose a framework for reconstructing MR images from undersampled data using a deep cascade of convolutional neural networks to accelerate the data acquisition process. We show that for Cartesian undersampling of 2D cardiac MR images, the proposed method outperforms the state-of-the-art compressed sensing approaches, such as dictionary learning-based MRI (DLMRI) reconstruction, in terms of reconstruction error, perceptual quality and reconstruction speed for both 3-fold and 6-fold undersampling. Compared to DLMRI, the error produced by the method proposed is approximately twice as small, allowing to preserve anatomical structures more faithfully. Using our method, each image can be reconstructed in 23 ms, which is fast enough to enable real-time applications.
1 Introduction
MRI acquisition is slow because k-space is sampled sequentially, motivating undersampling and reconstruction methods that compensate for missing data. The paper explores deep CNN cascades for Cartesian undersampled 2D cardiac MR reconstruction.
- Motivation: MRI acquisition is slow because k-space samples are acquired sequentially and traversal speed is limited by MR physics.This creates demands on patients and makes MRI more expensive and less accessible.
- Motivation: Undersampling k-space can accelerate acquisition but produces aliasing artifacts because it violates the Nyquist-Shannon theorem.The reconstruction algorithm must compensate for missing data using prior knowledge about the image.
- Prior approaches: Compressed sensing reconstructs sub-Nyquist images by exploiting compressibility and incoherence between sampling and sparsity domains.CS-MRI includes nonlinear optimization or iterative algorithms, while dictionary learning provides adaptive sparse representations.
- Prior approaches: More aggressive undersampling can exploit spatio-temporal, inter-slice, multi-contrast, filter-response, or parallel-imaging redundancies.These explicit redundancies may complement sparse modeling of inherent redundancies.
- CNN approach: The paper investigates whether CNNs can exploit learned data representations for undersampled MR reconstruction.This extends CNN use beyond established imaging tasks and addresses preliminary prior research on CNN-based MR reconstruction.
- CNN approach: A deep cascade of CNNs is proposed for Cartesian undersampling of 2D static cardiac MR images.The cascade simulates iterative dictionary-learning reconstruction while enabling end-to-end optimization; the method is reported to produce high-quality reconstructions and reconstruct each image in about 23 ms.
2 Problem Formulation
The reconstruction problem uses undersampled Fourier measurements to recover an image from an underdetermined system. The formulation combines data fidelity with learned CNN reconstruction, treating the CNN as an image-domain de-aliasing operator trained against fully sampled targets.
- Problem setup: The task is to reconstruct an image x from k-space measurements y, where image pixels are represented as a column vector.The measurements are acquired through an undersampled encoding process.
- Problem setup: M << N undersampled measurements make the Fourier system underdetermined and the inversion ill-defined.Recovery therefore requires a-priori knowledge of image properties.
- Regularized formulation: Regularization terms R and parameter λ encode image priors and adjust data fidelity according to measurement noise.CS-based formulations commonly use ℓ0 or ℓ1 norms in a sparsifying domain.
- Dictionary-learning formulation: Dictionary-learning reconstruction approximates x using reconstructions from sparse codes of extracted image patches.Ri extracts a patch at location i, γi is its sparse code, and D is the dictionary.
- CNN formulation: The CNN mapping takes the zero-filled reconstruction xu as input and directly produces a reconstructed image.Because xu is affected by aliasing from sub-Nyquist sampling, the CNN performs image-domain de-aliasing.
- Data fidelity: Incorporating data fidelity during learning conditions the CNN output on known k-space information and discourages modification of acquired values.The final reconstruction is defined as xcnn = fcnn(xu|θ, λ, Ω).
- Training objective: The CNN is trained on input-target pairs by minimizing an objective based on an element-wise squared loss between fully sampled targets and CNN reconstructions.The training data consist of pairs (xu, xt).
3 Data Consistency Layer
The data consistency layer integrates CNN reconstruction with acquired k-space information through Fourier-domain operations. Its forward pass applies Fourier transformation, data consistency, and inverse transformation, while its backward pass uses the corresponding Jacobian.
- A closed-form k-space solution enables data fidelity to be incorporated directly into the network architecture.The formulation is derived for a fixed network parameter setting and is used to construct the data consistency operation.
- The reconstructed image is obtained by applying the inverse Fourier encoding matrix to the corrected k-space coefficients.In the noiseless limit, sampled coefficients are replaced by their original values, motivating the term data consistency step in k-space.
- The DC layer is differentiable because its forward operation and input Jacobian are specified for backpropagation.The backward derivation simplifies because the Fourier transform and data consistency operation are linear.
- The DC layer computes Fourier transformation, element-wise data consistency, and inverse Fourier transformation in sequence.The data consistency operation combines predicted and undersampled Fourier coefficients using λ.
- Complex-valued MR images are represented using real-valued operations in a space with twice as many dimensions.This approach replaces C^N with R^2N while retaining the stated derivations through Wirtinger calculus.
4 Cascading Network
The cascading network extends CNN reconstruction to iterative de-aliasing and data consistency by concatenating CNNs into a deep sequence. It can be viewed as an unfolded, trainable extension of the DLMRI optimization process.
- Architecture notation: The architecture depth and cascade depth are denoted by nd and nc, respectively.
- Motivation: The cascade addresses the limitation that a single CNN performs only one de-aliasing step and may overfit without extensive training data.The text also notes that training such networks may require lengthy training and careful fine-tuning.
- Cascading Network: A cascading network concatenates CNNs so each successive network reconstructs from the previous network’s output.This builds an extremely deep architecture that alternates intermediate de-aliasing with data consistency reconstruction.
- Relation to DLMRI: The cascade can be viewed as unfolding DLMRI’s optimization process into a directly trainable reconstruction pipeline.Each CNN is described as expressing a dictionary-learning reconstruction step, extending DLMRI through end-to-end training.
5 Architecture and Implementation
The implementation represents undersampled complex MR images with two real-valued channels and uses a deep convolutional architecture with residual connections. Training uses squared error, Adam optimization, weight decay, and GPU-memory-dependent minibatches.
- Input representation: The CNN input is a two-channeled image whose channels store the real and imaginary parts of the undersampled image.This representation accommodates complex-valued MR data using real-valued network operations.
- CNN architecture: Each CNN uses nd − 1 convolutional layers with 3 × 3 kernels, 64 filters, ReLU activations, and a final two-filter reconstruction layer.The final layer projects the extracted representation back into the image domain.
- CNN architecture: Residual connections are included in the CNN architecture.
- Optimization: Training minimizes pixel-wise squared error using Adam with α = 10^-4, β1 = 0.9, β2 = 0.999, and ℓ2 weight decay of 10^-7.The minibatch size is 10, but it is reduced for deeper models to fit within single-GPU memory.
6 Experimental Results
Experiments used a 10-scan cardiac MR dataset, evaluating CNN and DLMRI under 3-fold and 6-fold undersampling with quantitative and visual comparisons. CNN consistently achieved lower errors, better detail preservation, and dramatically faster reconstruction.
- Dataset: The dataset contained 10 fully sampled cardiac cine MR scans, yielding 300 independent 2D images after restricting experiments to individual time frames.Rigid transformations were crucial for preventing overfitting, and undersampling masks were generated on-the-fly to expose the network to diverse aliasing patterns.
- Evaluation: Evaluation used mean squared error for 3-fold and 6-fold acceleration, with fixed test masks to account for mask-dependent reconstruction variability.Visual inspection complemented the quantitative assessment.
- Quantitative results: CNN consistently outperformed DLMRI for both 3-fold and 6-fold acceleration, with smaller reconstruction-error standard deviations across subjects.The comparison aggregated reconstruction errors across 10 subjects.
- Qualitative results: CNN reconstructions were more homogeneous and preserved more anatomical details and texture than DLMRI, whose 6-fold results showed extreme block-like artefacts from over-smoothing.At 3-fold acceleration, both methods sometimes lost structures, but CNN recovered more details; at 6-fold acceleration, CNN preserved texture better.
- Reconstruction speed: 23 ± 0.1 milliseconds was the CNN reconstruction time per slice on a GeForce GTX 1080, compared with about 6.1 ± 1.3 hours per subject for DLMRI on CPU.The trained CNN’s fast GPU inference enables real-time applications, whereas DLMRI requires repeated dictionary-learning and sparse-coding iterations.
7 Discussion and Conclusion
The CNN reconstruction framework achieves accurate MR recovery with interleaved data-consistency stages and shows advantages over DLMRI, while several generalization and acquisition-scope questions remain.
- Interleaved data-consistency stages enable a CNN model to reconstruct images sufficiently well.
- Figure 3 compares original, 6x undersampled, DLMRI, and CNN images alongside 5x-scaled error maps.
- CNN reconstructions outperform DLMRI empirically, including more accurate recovery of most anatomical structures under aggressive undersampling.
- The study used SENSE-reconstructed data to simulate acquisition, rather than directly modeling surface-coil sensitivity maps for parallel imaging.
- The network learned de-aliasing from arbitrary Cartesian masks at a fixed sampling rate, but tolerance to other rates and radial or spiral trajectories remains to be investigated.
- Performance with pathology or more variable content remains undetermined, and generalization requires evaluation on a larger dataset.