PaperlayerView exact revision on arXiv

Source-linked AI summary

MR image reconstruction using deep density priors

Kerem C. Tezcan, Christian F. Baumgartner, Roger Luechinger, Klaas P. Pruessmann, Ender Konukoglu

arXiv:1711.11386v4cs.CVeess.IVstat.ML

TL;DR

Undersampled MR reconstruction needs priors to recover missing k-space information, while many deep-learning methods learn implicit mappings tied to paired data and acquisition settings. This paper learns an explicit VAE-based prior from fully sampled image patches and decouples it from encoding, producing high-quality reconstructions across several datasets and acquisition settings. The method still requires suitable high-quality training data and relies on modeling assumptions such as a unit-Gaussian latent prior.

  • Problem

    Existing deep-learning MR reconstruction methods learn implicit priors through mappings trained with paired undersampled and fully sampled images, motivating an alternative prior formulation.

  • Method

    DDP learns fully sampled MR-patch distributions with an unsupervised VAE and uses the resulting explicit likelihood as a Bayesian reconstruction prior decoupled from the encoding operation.

  • Results

    DDP produced visually high-quality and quantitatively accurate reconstructions on HCP, ADNI, multi-coil, and lesion-containing images, including accurate magnitude and phase recovery.

  • Takeaways & Limitations

    The learned prior reconstructed lesions without blurring them and can be applied across sampling schemes without retraining when images remain in the same domain.

  • Takeaways & Limitations

    DDP requires high-quality training data, and suitable training sets for functional or diffusion MRI remain a topic for future research.

Abstract

from arXiv · show

Algorithms for Magnetic Resonance (MR) image reconstruction from undersampled measurements exploit prior information to compensate for missing k-space data. Deep learning (DL) provides a powerful framework for extracting such information from existing image datasets, through learning, and then using it for reconstruction. Leveraging this, recent methods employed DL to learn mappings from undersampled to fully sampled images using paired datasets, including undersampled and corresponding fully sampled images, integrating prior knowledge implicitly. In this article, we propose an alternative approach that learns the probability distribution of fully sampled MR images using unsupervised DL, specifically Variational Autoencoders (VAE), and use this as an explicit prior term in reconstruction, completely decoupling the encoding operation from the prior. The resulting reconstruction algorithm enjoys a powerful image prior to compensate for missing k-space data without requiring paired datasets for training nor being prone to associated sensitivities, such as deviations in undersampling patterns used in training and test time or coil settings. We evaluated the proposed method with T1 weighted images from a publicly available dataset, multi-coil complex images acquired from healthy volunteers (N=8) and images with white matter lesions. The proposed algorithm, using the VAE prior, produced visually high quality reconstructions and achieved low RMSE values, outperforming most of the alternative methods on the same dataset. On multi-coil complex data, the algorithm yielded accurate magnitude and phase reconstruction results. In the experiments on images with white matter lesions, the method faithfully reconstructed the lesions. Keywords: Reconstruction, MRI, prior probability, machine learning, deep learning, unsupervised learning, density estimation

I. INTRODUCTION

MR reconstruction reduces acquisition time by recovering images from undersampled k-space, but existing deep-learning methods typically learn implicit priors tied to paired training data and acquisition settings. This work instead learns an explicit image prior with VAEs and decouples it from the encoding operation.

  • I. INTRODUCTION: Reducing k-space samples can shorten MR acquisition, while reconstruction methods compensate for the missing measurements using image or signal priors.Acquisition time is directly related to the number of k-space samples.
  • I. INTRODUCTION: Earlier deep-learning approaches learned undersampled-to-fully-sampled mappings or enhanced iterative algorithms, embedding prior information implicitly in trained network parameters.These approaches include feed-forward mappings and learned unrolled iterative methods.
  • I. INTRODUCTION: DDP learns an explicit prior from fully sampled MR images with VAEs and integrates it with data consistency in a Bayesian reconstruction model.The approach estimates the prior from image patches and formulates reconstruction as MAP estimation.
  • I. INTRODUCTION: Because DDP decouples the prior from data consistency, it does not require paired undersampled and fully sampled training data or retraining for changed sampling patterns.The supplied introduction identifies these as theoretical advantages of the decoupled formulation.
  • I. INTRODUCTION: The paper focuses on 2D imaging, while extending the same techniques to 3D imaging is left for future work.This is an explicit scope boundary of the presented method.

B. Learning the data distribution with VAEs

The VAE models the distribution of MR image patches through a latent-variable model trained without paired reconstruction targets. Its ELBO objective provides a tractable approximation to image log likelihood.

  • B. Learning the data distribution with VAEs: A VAE approximates the distribution of high-dimensional magnitude image patches using a latent variable z with a lower-dimensional representation than the patch.The latent prior can be a known distribution such as a unit Gaussian, while the decoder is parameterized by a neural network.
  • B. Learning the data distribution with VAEs: VAE training maximizes the evidence lower bound, which combines the expected data log likelihood with a tractable Kullback-Leibler divergence term.The ELBO is used as a lower bound and proxy for the intractable log likelihood.
  • B. Learning the data distribution with VAEs: The encoder and decoder parameters are jointly optimized over training samples, with the KLD term using the known latent prior to remain tractable.The training formulation distinguishes this tractable KLD from the divergence involving the true posterior.
  • B. Learning the data distribution with VAEs: The encoder maps a patch to an approximate latent posterior, while the decoder maps a latent vector to a conditional distribution of the data.The encoder and decoder are represented by separate neural networks.

C. Deep density prior (DDP) reconstruction model

DDP uses the trained VAE’s differentiable ELBO as an image-patch prior within a MAP reconstruction objective. The resulting algorithm alternates prior-based patch updates with data-consistency updates for the measured k-space data.

  • C. Deep density prior (DDP) reconstruction model: The trained VAE supplies a differentiable approximate log likelihood for magnitude patches, allowing its prior gradient to enter an iterative reconstruction algorithm.The ELBO approximates the true patch log likelihood, and differentiability enables computation of image-patch derivatives.
  • C. Deep density prior (DDP) reconstruction model: The ELBO is approximated with Monte Carlo samples from the encoder posterior, and the number of samples is denoted by J.Exact expectation evaluation is computationally infeasible, motivating the sampling approximation.
  • C. Deep density prior (DDP) reconstruction model: DDP combines the usual data term with a learned-prior regularization term in a MAP reconstruction objective over overlapping image patches.The model assumes independence between different patches when forming the prior over the image.
  • C. Deep density prior (DDP) reconstruction model: The reconstruction procedure initializes with a zero-filled image, repeatedly updates patches using the VAE prior, and enforces data consistency before returning the final image.The algorithm uses outer POCS iterations and inner prior-projection iterations.

D. Optimization using projection onto convex sets

DDP solves its reconstruction objective with projection onto convex sets, alternating a data-consistency projection and an approximate gradient-based prior projection. Phase handling is data-driven, with an optional additional phase projection for single-coil acquisitions.

  • D. Optimization using projection onto convex sets: POCS alternates projections onto the data-consistency and learned-prior constraints to iteratively minimize the DDP reconstruction objective.The data-consistency projection is exact in the stated formulation, whereas the prior projection is approximated by gradient ascent.
  • D. Optimization using projection onto convex sets: For multiple coils, the data-consistency projection implements SENSE reconstruction, while single-coil data can use an additional phase projection.The DDP prior does not explicitly provide phase information, so phase recovery is driven primarily by data consistency.
  • D. Optimization using projection onto convex sets: The prior projection extracts overlapping patches, computes their VAE-prior derivatives, averages overlapping updates, and repeats gradient steps.The resulting derivative image updates the reconstruction before the next data-consistency step.
  • D. Optimization using projection onto convex sets: The method applies a fixed number of POCS steps and returns the resulting reconstructed image.Algorithm 1 summarizes the iterative procedure.
  • D. Optimization using projection onto convex sets: The implementation ignores noise covariance, so it does not provide optimal signal-to-noise ratio for multi-coil data.The paper states that covariance correction is omitted for simplicity.

1) MR image data:

The study used three structural MRI sources to train the VAE prior and evaluate reconstruction across public datasets, lesion cases, and multi-coil acquisitions.

  • Three structural MRI sources supported prior learning and evaluation: HCP, ADNI, and newly acquired volunteer data.HCP supplied training images; ADNI tested images with white matter lesions; volunteer scans provided multi-coil complex data.
  • 790 T1-weighted HCP slices from 158 subjects trained the prior VAE.The images came from 3D MPRAGE scans and were normalized per slice before training.
  • ADNI images from subjects with Alzheimer’s disease and visible white matter lesions tested cross-domain reconstruction and lesion fidelity.
  • Eight healthy volunteers provided fully sampled complex k-space data for retrospective multi-coil undersampling experiments.A 16-element head coil array was used, with coil sensitivities measured by SENSE for six volunteers and also estimated using ESPIRiT.

2) Setup and evaluation:

Evaluation retrospectively undersampled held-out HCP, ADNI, and volunteer data under multiple Cartesian and radial patterns, comparing reconstructions with image-quality metrics and competing methods.

  • Held-out HCP, ADNI, and volunteer data were retrospectively undersampled and compared against fully sampled references.The evaluation used 17 separate HCP subjects, ADNI images, and data from 8 acquired subjects.
  • The study tested Cartesian undersampling ratios R=2, 3, 4, and 5, plus 4-fold radial undersampling with NUFFT.Cartesian patterns randomly sampled the phase-encoding direction while fully sampling the readout direction and included 15 central profiles.
  • The protocol generated a new random undersampling pattern for each test image and used fixed POCS, prior-projection, and step-size settings.POCS used 30 iterations for R=2–3, 60 for R=4–5, 10 prior-projection iterations, and step size α=1e-4.
  • Reconstructions were assessed using RMSE, CNR, and contrast difference at the gray–white matter boundary.FreeSurfer segmentations supplied the regions needed for CNR and contrast-difference measurements.
  • Figure 1 compares full-FOV, cropped-FOV, and zoomed reconstructions with clipped error maps at R=3.The figure uses a common undersampling pattern in the displayed comparisons, including the pattern used to train ADMM-Net.
  • The comparison included zero-filling, TV, DLMRI, ADMM-Net, BM3D-MRI, SIDWT, FDLCP, and PBDW.Some implementations, including the last three listed methods, were available only for cropped FOV images.

III. RESULTS

DDP produced accurate reconstructions across fully sampled benchmark images, measured multi-coil data, and lesion-containing images, while remaining applicable across acquisition variations. Its strongest reported advantages were contrast preservation, reconstruction of magnitude and phase, lesion fidelity, and reduced sensitivity to sampling-pattern changes.

  • Visual reconstruction quality: DDP, DLMRI, and FDLCP preserved visual quality better than TV, PBDW, SIDWT, and ADMM-Net in representative HCP reconstructions.TV and PBDW produced cartoon-like results, SIDWT left aliasing, and ADMM-Net missed a small gray-matter island.
  • Quantitative comparisons: DDP performed best in RMSE for full-FOV reconstructions except at R=2, and second best for cropped-FOV reconstructions; it matched or exceeded all methods in CNR.Table I summarizes mean and standard-deviation metrics for 17 test images, separated by full and cropped FOV.
  • Measured multi-coil data: On measured multi-coil data, DDP reconstructed magnitude and phase for both two-fold Cartesian and four-fold radial undersampling, with Cartesian R=2 RMSE of 6.97% using ESPIRiT maps.Using measured coil maps gave 6.92% RMSE for the same setting; magnitude images were smoother than fully sampled references.
  • Lesion reconstruction: DDP reconstructed white-matter lesions from ADNI images even though its VAE was trained only on healthy HCP subjects, without lesion-specific error increases.The reconstructions recovered gray- and white-matter structures and edges faithfully.
  • Sensitivity and generalization: Performance remained reasonable across modest domain differences, but substantially different contrast or resolution is expected to reduce reconstruction quality.The authors identify contrast- or resolution-specific priors and domain adaptation as possible remedies.
  • Sensitivity and generalization: Unlike sampling-specific feed-forward models, DDP remained usable across undersampling schemes and coil settings without retraining, because its learned prior is decoupled from data consistency.ADMM-Net performance decreased significantly when training and test undersampling patterns or ratios differed.

V. CONCLUSION

The paper proposes DDP, which uses a VAE-learned distribution of fully sampled MR patches as a probabilistic prior for undersampled k-space reconstruction. Evaluations on HCP, ADNI, and multi-channel data showed promising visual and quantitative results.

  • DDP uses a VAE to learn fully sampled MR-patch distributions and incorporates them as a probabilistic prior in Bayesian reconstruction.The method is designed for MR reconstruction from undersampled k-space acquisitions.
  • The learned prior removes sensitivity to the sampling pattern while supporting reconstruction across HCP, ADNI, and multi-channel k-space data.The conclusion reports promising visual quality and quantitative measures across these datasets and acquisition settings.
  • The supplementary document provides additional details and results supporting the main paper.

A. Details on VAE network architecture and training

The VAE uses convolutional encoder and decoder networks with fully connected layers to model posterior and likelihood distributions for image patches. The supplementary setup also specifies training, datasets, reconstruction baselines, and acquisition conditions.

  • A. Details on VAE network architecture and training: The encoder maps an image patch to posterior mean and covariance, while the decoder maps a latent vector to likelihood mean and covariance.Both networks are mostly convolutional and contain a single fully connected layer.
  • A. Details on VAE network architecture and training: The VAE architecture uses 3x3 convolutions, ReLU activations, and log-variance outputs, with Adam optimization at a 5e-4 learning rate.Weights use a truncated normal initializer with standard deviation 0.05.
  • A. Details on VAE network architecture and training: The base model uses 28×28 patches and a 60-dimensional latent space, with additional experiments varying patch size and latent dimensionality.
  • A. Details on VAE network architecture and training: HCP, ADNI, and volunteer datasets use different scanners, acquisition parameters, resolutions, and preprocessing conditions.The HCP, ADNI, and volunteer acquisition descriptions specify distinct devices and sequence settings.
  • A. Details on VAE network architecture and training: The evaluation compares DDP with zero-filling, TV, dictionary learning, ADMM-Net, SIDWT, FDLCP, PBDW, and BM3D-MRI.ADMM-Net uses the same 790 images for training as the VAE and matches training and test undersampling patterns except in sensitivity tests.

D. Performance Metrics

The evaluation uses normalized RMSE, CNR, and contrast-based measures to compare reconstructed and fully sampled images. CNR is computed from gray-matter and white-matter statistics, with white-matter measurements restricted to tissue boundaries.

  • D. Performance Metrics: Normalized RMSE compares reconstructed and fully sampled images over image voxels, with normalization by the fully sampled reference magnitude.Masked evaluations use only voxels within the mask.
  • D. Performance Metrics: CNR is computed from gray- and white-matter means divided by their standard deviations.The gray-matter region uses the whole segmentation, while white matter uses its boundary region.
  • D. Performance Metrics: The white-matter boundary is obtained by subtracting an eroded white-matter segmentation from the original segmentation.A 7x7 square structuring element is used for binary erosion, isolating white-matter voxels adjacent to gray matter.
  • D. Performance Metrics: Contrast difference is reported as a related measure without division by the standard deviation.

E. Complementary Results

Complementary experiments examine prior samples, convergence, parameter sensitivity, multi-coil reconstruction, anatomical generalization, and robustness across undersampling settings.

  • Prior samples: VAE-generated 28x28 patches look realistic, and their variance maps show higher uncertainty in sulci-like structures filled with cerebrospinal fluid.The samples were generated from random unit-Gaussian latent vectors without cherry-picking.
  • Convergence: For R=2, RMSE decreases and ELBO increases during DDP iterations, with both measures converging after approximately 15 iterations; higher R values require more iterations.Initialization with the zero-filled image explains the initially high RMSE.
  • Parameter sensitivity: Reconstruction accuracy is relatively insensitive to latent-space dimension, while smaller patch sizes reduce reconstruction error more substantially.The parameter-sensitivity experiments used five test subjects at R=3.
  • Multi-coil reconstruction: The DDP projection reduces RMSE relative to SENSE-only reconstruction, although both methods later show increasing error attributed to coil-sensitivity mismatch.The DDP projection is activated after 10 iterations, and the RMSE drops immediately afterward.
  • Anatomical generalization: DDP reconstructs superior brain slices fairly well but struggles more in inferior regions, especially the cerebellum, with RMSEs of 4.28% (0.68) and 6.90% (1.23) for R=2 and R=3.The reported values combine inferior and superior anatomical regions for N=17 slices.
  • Anatomical generalization: Inferior slices have RMSEs of 4.44% (0.71) and 7.41% (1.06) for R=2 and R=3, respectively, likely reflecting greater structural complexity.These values are reported for N=11 inferior slices, compared with N=6 for the other anatomical region.
Loading 1711.11386v4…