PaperlayerView exact revision on arXiv

Source-linked AI summary

Time-Dependent Deep Image Prior for Dynamic MRI

Jaejun Yoo, Kyong Hwan Jin, Harshit Gupta, Jerome Yerly, Matthias Stuber, Michael Unser

arXiv:1910.01684v2eess.IVcs.CVcs.LG

TL;DR

Dynamic MRI reconstruction must recover moving-organ dynamics from sparsely sampled data despite limitations in existing models and priors. This paper optimizes an untrained deep-image-prior network with a time-parameterized manifold, outperforming state-of-the-art compressed-sensing approaches by 3.8 dB while enabling continuous temporal control.

  • Problem

    Dynamic MRI requires reconstructing moving-organ dynamics from partially sampled k-space, while existing methods face restrictions involving SNR, coil design, priors, processing, and efficiency.

  • Method

    The method optimizes an untrained convolutional network with an explicit time-parameterized manifold to reconstruct dynamic MRI sequences directly from k-space measurements.

  • Results

    3.8 dB improvement over state-of-the-art compressed-sensing approaches was reported, with better reconstructions in retrospective and fetal cardiac datasets.

  • Takeaways & Limitations

    The approach reconstructs high-resolution dynamic MRI with sub-frame or continuous temporal control without requiring heartbeat marking or spoke reordering.

  • Takeaways & Limitations

    The forward model’s NuFFT implementation is a major runtime bottleneck, accounting for 47% of each iteration’s running time.

Abstract

from arXiv · show

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. Existing reconstruction methods suffer from restrictions either in the model design or in the absence of ground-truth data, resulting in low image quality. We introduce a generalized version of the deep-image-prior approach, which optimizes the network weights to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k-space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

I. INTRODUCTION · A. Contribution · B. Related Work

The paper introduces an unsupervised deep-learning framework for reconstructing dynamic MRI from partially sampled k-space without relying on unavailable ground-truth data. Its contribution combines a CNN structural prior with a constrained temporal manifold and distinguishes sequence reconstruction from prior DIP-based video-compression work.

  • I. INTRODUCTION: Dynamic MRI accelerates acquisition through partial k-space sampling, requiring reconstruction methods that compensate for missing data while exploiting spatial or temporal redundancy –.The introduction motivates approaches including parallel MRI and k-t acceleration methods.
  • I. INTRODUCTION: Existing reconstruction methods face limitations involving SNR constraints, coil-design restrictions, hand-picked priors, multiple processing steps, or inefficient convex-optimization deployment.
  • I. INTRODUCTION: Supervised deep-learning MRI reconstruction methods – depend heavily on training datasets and fully sampled ground-truth measurements, which are typically unavailable for dynamic MRI.
  • A. Contribution: The proposed unsupervised framework optimizes a generative network to reconstruct golden-angle radial k-space spokes, using a CNN implicit structural prior and a one-dimensional manifold for temporal dependencies.The approach is inspired by deep image priors.
  • A. Contribution: The generative model combines MapNet latent variables with a structural prior to reconstruct dynamic images, while analyses show that both the manifold design and MapNet are essential for good reconstructions.
  • A. Contribution: The authors characterize the method as the first unsupervised deep-learning approach to reconstruct full temporal frames of dynamic MRI sequences at high spatial resolution.
  • B. Related Work: Prior unsupervised medical-imaging applications of DIP addressed static MRI and positron-emission-tomography reconstruction rather than dynamic image sequences [40],,.
  • B. Related Work: The closest prior work applied DIP to video compression [45], optimizing network weights and latent variables for compact frame codes, whereas this paper reconstructs image sequences and constrains the latent space.Without latent-space constraints, codes may diverge into an arbitrary space.

II. METHODS … 1) Forward Model:

The methods extend deep image prior to dynamic MRI by combining an untrained CNN-based reconstruction framework with a radial k-space acquisition model sampled over time. The forward model represents each temporal image through Fourier measurements collected along golden-angle radial spokes.

  • II. METHODS: The method extends deep image prior to dynamic MRI, adding a mapping network and learning underlying latent manifolds.DIP uses an untrained network and requires neither prior training nor additional image data.
  • A. Deep Image Prior: DIP reconstructs images by optimizing an untrained CNN from a fixed random latent variable, exploiting the architecture’s bias toward natural-looking outputs.The optimized output is x∗ = fθ∗(z), and the CNN prior favors structured images over noisy unstructured ones.
  • B. Dynamic MRI: The dynamic MRI acquisition records a temporal sequence of radial Fourier-transform lines from a fixed slice of a 3D object.Each image is represented as x ∈ C^N, while measurements at time t are vectors of k-space samples along a radial orientation.
  • 1) Forward Model:: At each time t, the MRI measurement process is linear and relates the image to k-space data through an orientation-dependent system matrix H(ϑ).H(ϑ) combines the 2D Fourier transform with resampling along a radial line; each resulting measurement is an angular spoke.
  • 1) Forward Model:: The acquisition collects K spokes at regularly spaced times t_k = t_0 + k∆t, with temporal step size ∆t.The spoke orientations vary over the acquisition, providing measurements for reconstructing the image sequence.
  • 1) Forward Model:: Spoke orientations follow the golden-angle strategy, approximated by an angular increment of 111.25°.The strategy is based on the irrationality condition (ω_0∆t/π) ∉ Q and uses angular velocity ω_0.

2) Spoke-Sharing:

Spoke-sharing reconstructs each dynamic MRI frame by pooling neighboring radial spokes under a local temporal-smoothness assumption. Increasing the sharing parameter improves measurement coverage but trades off temporal resolution, while the resulting inverse problem remains ill-posed.

  • Spoke-Sharing: A single radial orientation per frame is insufficient to recover the instantaneous two-dimensional image, motivating reconstruction from temporally neighboring measurements.The broader goal is recovering the dynamic sequence x(t_k), or continuously varying x(t), from finite measurements.
  • Spoke-Sharing: The acquisition uses a nonuniform fast Fourier transform with golden-angle sampling and spoke-sharing, ensuring that no radial direction is measured twice.The golden-angle strategy is part of the framework’s dynamic MRI data-acquisition procedure.
  • Spoke-Sharing: Spoke-sharing pools ns neighboring radial spokes to approximate x(t) by x(t_k) over a local time interval, with ns controlling temporal resolution.The method assumes changes are slow across T_k and uses ns ∈ 2N + 1 radial lines per reconstructed frame.
  • Spoke-Sharing: The augmented forward model combines ns spokes into M = ns M0 measurements, improving coverage while remaining ill-posed because M < N.Here N is the number of unknown image values, and H_k encodes the pseudosimultaneous acquisition at time t_k.

C. Proposed Framework

The proposed time-dependent DIP reconstructs dynamic MRI sequences by optimizing an untrained network in the measurement domain, combining a temporal manifold, a learned mapping network, and image generation. The framework enforces measurement consistency while regularizing reconstructed images and flexibly representing temporal dynamics.

  • Proposed Framework: The framework optimizes an untrained neural network to map input sequences to reconstructed dynamic MRI images, with optimization performed in the measurement domain.This enforces consistency with acquired measurements while the modified DIP regularizes the reconstructed image sequence.
  • Manifold Design: A designed manifold Z encodes temporal variations while preserving individual-frame structure, using input proximity to guide temporal similarity.A straight-line manifold is given as an example of a temporally ordered input design.
  • Mapping Network (MapNet): MapNet introduces nonlinear fully connected layers that warp the fixed manifold, improving flexibility and helping the reconstruction network represent true dynamics.This learned mapping reduces the limitations of manually designed manifolds and exploits neural-network representation power.
  • Final Algorithm: The algorithm samples manifold inputs, repeatedly updates network parameters using randomly sampled batch losses and gradient descent, then reconstructs the full image sequence.The stated implementation uses Adam with niter = 10, 000 and batch size B = 1.

III. EXPERIMENTS

The experiments section covers the datasets, baseline methods, cardiac-cycle estimation, evaluation setups, and implementation details.

  • The section describes the datasets and baseline methods used in the experiments.
  • It explains the cardiac-cycle estimation and evaluation setups.
  • It also provides implementation details for the experiments.

A. Datasets … B. Baseline Methods

The experiments use breath-hold MR datasets with twofold measurement upsampling, spanning retrospective and fetal cardiac acquisitions. Comparisons include BP, GRASP, and the Reordering Method (RD),.

  • A. Datasets: Twofold upsampling is assumed for every dataset, producing reconstructed fields of view whose size is half the measurements’ first dimension.
  • 1) Retrospective Dataset:: The retrospective cardiac cine dataset was acquired on a 3T Siemens Tim Trio scanner using a 32-element cardiac coil, bSSFP, and prospective cardiac gating.
  • 1) Retrospective Dataset:: The retrospective acquisition contains 23 frames with 43.2 ms temporal resolution and uses a 300 × 300 mm2 field of view with a 128 × 128 acquisition matrix.
  • 2) Fetal Cardiac Dataset:: The fetal cardiac dataset uses an untriggered continuous 2D bSSFP sequence with golden-angle radial readouts on a 1.5T Siemens scanner with 18-channel body and 32-channel spine coils.
  • B. Baseline Methods: BP provides a zero-filled discrete Fourier transform and serves as the most basic baseline.
  • B. Baseline Methods: GRASP extends k-t SPARSE-SENSE to volumetric golden-angle radial acquisitions, while this evaluation omits its spoke-sharing strategy.
  • B. Baseline Methods: The Reordering Method (RD), is a three-step algorithm that first reconstructs limited-quality real-time images and uses them to reorder spokes.

C. Estimation of cardiac Cycles … F. Architectures

The method estimates cardiac cycles from radial k-space data, evaluates reconstructions using RSNR, and uses specified optimization and network architectures for implementation. Radial sampling of central k-space enables physiological-motion extraction, while the mapping and generative networks produce the latent representation and images.

  • C. Estimation of cardiac Cycles: Cardiac-cycle estimates are required for both RD and the proposed algorithm when processing fetal cardiac sequences.The estimate covers the total acquisition duration and can typically be obtained from k-space.
  • C. Estimation of cardiac Cycles: Radial acquisitions sample k-space’s center at every readout, enabling physiological-motion signals for estimating cardiac cycles.Prior radial-data estimation techniques are cited as,, and.
  • D. Evaluation Metric: Reconstruction quality is quantified with regressed SNR (RSNR), computed using the oracle x and reconstructed image x∗.Higher RSNR indicates better reconstruction.
  • E. Implementation Details: Training uses Adam with default settings for 10,000 iterations, batch size B = 1, and learning rate 10^-3.The implementation uses an Intel i7-7820X CPU, an NVIDIA Titan X (Pascal) GPU, PyTorch 1.0.0, and Python 3.6.
  • F. Architectures: The mapping network gφ contains two fully connected layers with 512 hidden dimensions and ReLU, producing an L = 64 latent vector reshaped to 8 × 8.This reshaped representation is input to the generative network hψ.
  • F. Architectures: The generative network combines convolution, batch normalization, ReLU, and nearest-neighbor interpolation layers, with zero-padding preserving output size.The final layer omits ReLU, and the architecture is summarized in Table I.

IV. DESIGN OF THE LATENT SPACE · A. Straight-Line Manifold for a Single Heart Cycle · B. Manifolds for Multiple Heart Cycles

The latent-space analysis evaluates a straight-line manifold for one heart cycle and segmented-line, circle, and helix manifolds for multiple cycles. It finds that manifold design and MapNet are both needed for best reconstruction, establishing Helix+MapNet as the default setup.

  • IV. DESIGN OF THE LATENT SPACE: The analysis proceeds from a simplest single-cycle setup to a more complicated multiple-cycle dataset, where the manifold choices are compared with baselines.The multiple-cycle setting reflects measurements that generally span several heart cycles.
  • A. Straight-Line Manifold for a Single Heart Cycle: For a single heart cycle, the straight-line manifold linearly interpolates between two sampled latent endpoints to encode temporal image variation.Intermediate latent points are obtained by linear interpolation between z0 and zK−1.
  • A. Straight-Line Manifold for a Single Heart Cycle: The straight-line configuration reconstructs single-cycle dynamics successfully and outperforms the other baseline methods.This result is reported for the single-cycle dataset in Table II.
  • B. Manifolds for Multiple Heart Cycles: The multiple-cycle candidates include a segmented line with 14 landmarks, circles with periodic coordinates and fixed slack variables, and a helix that scales slack coordinates over time.The segmented line uses linear interpolation between adjacent landmarks; circles and helices encode p cycles through unit-circle coordinates.
  • B. Manifolds for Multiple Heart Cycles: For multiple heart cycles, the straight-line manifold produces a static reconstruction, whereas more structured manifolds are introduced to capture quasi-periodic cardiac motion.The static failure occurs because most pixels are static and dynamic regions are localized, allowing optimization to reach a constant-image local minimum.
  • B. Manifolds for Multiple Heart Cycles: Helix+MapNet is adopted as the default because careful manifold design and a mapping network are both necessary for best performance.MapNet adapts the input manifold to the dataset and further improves reconstruction.

V. RESULTS

The results first evaluate reconstruction methods on a retrospective dataset with well-defined desired behaviors, then assess whether the findings extend to real fetal cardiac data.

  • Retrospective dataset: The study first evaluates reconstruction methods on a retrospective dataset where desired behaviors are well-defined.
  • Fetal cardiac dataset: The results then examine a fetal cardiac dataset to test whether the retrospective observations extend to a real scenario.

A. Retrospective Dataset: Multiple Heart Cycles · B. Fetal Cardiac Dataset · VI. DISCUSSION

Across retrospective and real fetal cardiac data, Helix + MapNet reconstructs dynamic heart motion and resolves dynamic and static structures better than competing methods. Its frame-local reconstruction avoids phase mixing while preserving quasi-periodic cycles and cycle-to-cycle motion variations.

  • A. Retrospective Dataset: Multiple Heart Cycles: Helix + MapNet outperforms BP, GRASP [15], and RD [24], [25] in retrospective visual comparisons of dynamic heart reconstructions and residuals.The comparisons use fully sampled Cartesian reconstructions as ground truth, while RD benefits from estimated period information.
  • B. Fetal Cardiac Dataset: Real fetal-data evaluation uses a static image reconstructed from all spokes as pseudo-gold standard, with high quality only in nonmoving regions.No ground-truth image is available for the real-data experiment.
  • B. Fetal Cardiac Dataset: On real fetal cardiac data, Helix + MapNet produces better-resolved features than GRASP [15] and RD [24], [25] in both dynamic and static regions.RD reconstructs dynamic phases well but exhibits edge artifacts and misses detailed static-background structures.
  • B. Fetal Cardiac Dataset: Helix + MapNet reconstructs fetal cardiac frames from neighboring spokes, avoiding RD’s [24], [25] superposition of spokes from different cardiac phases.BP fails to capture fetal beats, GRASP remains unsatisfactory, and RD’s reordering can mingle different cycles within one frame.
  • B. Fetal Cardiac Dataset: The reconstructed fetal sequence visibly preserves quasi-periodic cardiac motion while retaining variations between cycles.The complete sequence is shown as y-t cross sections in the bottom row of Figure 5.
  • VI. DISCUSSION: The method’s distinctive benefit is reconstructing a complete sequence that shows both quasi-periodicity and motion variation across cardiac cycles.The supplied discussion identifies this sequence-level behavior as unavailable from the other algorithms.

A. t-SNE Visualization of the Latent and Image Spaces · B. Benefits of Our Approach · VII. CONCLUSION

The approach adaptively warps a structured temporal manifold, enabling continuous dynamic-image reconstruction with sub-frame control, lower memory use, and simpler optimization. Its main limitation is reconstruction speed, driven largely by the CPU-dependent NuFFT forward model.

  • A. t-SNE Visualization of the Latent and Image Spaces: MapNet adaptively warps the fixed manifold while preserving its geometric prior, shortening the helix height and bringing corresponding cycles closer.The mapped variables retain a 13-period helical geometry while recovering similarity between different cycles, which supports better reconstructions.
  • A. t-SNE Visualization of the Latent and Image Spaces: The t-SNE embeddings show approximately 13 heart-motion cycles across the fixed latent, mapped latent, and corresponding image spaces.Temporal index is color-coded over 400 frames, and the cycles remain visible in the reconstructed-image embedding.
  • B. Benefits of Our Approach: The learned function fθ∗ captures nonlinear image-manifold dynamics, allowing reconstruction at specified timestamps and nontrivial intra-frame images through interpolation between consecutive inputs.This continuous navigation is not possible with standard temporal interpolation methods such as temporal b.
  • B. Benefits of Our Approach: The approach reduces memory requirements by optimizing the neural network in batches instead of storing the full reconstruction volume during compressed-sensing updates.A 5,000-frame sequence at 256 × 256 would otherwise require handling 256 × 256 × 5,000 data, exceeding a gigabyte of memory.
  • B. Benefits of Our Approach: Training visits each frame about seven times, but reconstruction is slowed by the NuFFT forward model, which accounts for 47% of total runtime because its implementation lacks GPU acceleration.The comparison reports 10,000 iterations over 1,400 frames for this method versus GRASP seeing all frames across 24 outer iterations.
  • B. Benefits of Our Approach: The fully automated method uses two interpretable hyperparameters, whereas k-t SENSE requires three interdependent parameters typically selected through substantial grid search.The two parameters are the initial learning rate and number of iterations.
  • VII. CONCLUSION: The method provides high-spatial-resolution dynamic MRI reconstruction with sub-frame or continuous temporal control by combining a one-dimensional manifold, mapping network, and generative model.The generative network exploits both its representation power and the structural priors injected through the manifold.
Loading 1910.01684v2…