PaperlayerView exact revision on arXiv

Source-linked AI summary

Active MR k-space Sampling with Reinforcement Learning

Luis Pineda, Sumana Basu, Adriana Romero, Roberto Calandra, Michal Drozdzal

arXiv:2007.10469v2eess.IVcs.CV

TL;DR

The paper addresses trajectory optimization in MRI, where much deep-learning research has focused on reconstruction from predetermined measurements. It learns sequential Cartesian sampling policies with reinforcement learning and reports that they outperform heuristics and the scoring-function approach of [31] on a large-scale single-coil knee dataset.

  • Problem

    Much deep-learning MRI work improves reconstruction for predetermined measurements, leaving the sequence of k-space measurements to optimize.

    Source: §1 · p. 1

  • Method

    The paper formulates active Cartesian MRI acquisition as a POMDP and uses deep reinforcement learning to learn policies with a fixed reconstruction model.

    Source: §1 · p. 2, §3 · p. 3, §3.1 · p. 3

  • Results

    On a large-scale single-coil knee dataset, learned policies outperform simple acquisition heuristics and the scoring function introduced in [31] on MSE, NMSE, SSIM, and PSNR.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The learned policies provide state-of-the-art active MRI acquisition results on the evaluated single-coil knee dataset.

    Source: §1 · p. 2, §5 · p. 8

    The experimental setup is simplified for model exploration and does not include all practical MRI phase-encoding sampling issues.

    Source: §5 · p. 9

Abstract

from arXiv · show

Deep learning approaches have recently shown great promise in accelerating magnetic resonance image (MRI) acquisition. The majority of existing work have focused on designing better reconstruction models given a pre-determined acquisition trajectory, ignoring the question of trajectory optimization. In this paper, we focus on learning acquisition trajectories given a fixed image reconstruction model. We formulate the problem as a sequential decision process and propose the use of reinforcement learning to solve it. Experiments on a large scale public MRI dataset of knees show that our proposed models significantly outperform the state-of-the-art in active MRI acquisition, over a large range of acceleration factors.

1 Introduction

MRI acquisition can be slow, while much deep-learning work improves reconstruction for fixed measurements rather than optimizing the measurement trajectory. This paper uses deep reinforcement learning to select sequential k-space measurements and reports better performance than common heuristics and a myopic approach.

1 Introduction

  • MRI acquisition can take up to an hour, causing patient discomfort and motion artifacts.MRI offers superior image quality and zero radiation exposure compared with modalities such as computed tomography.Source: §1 · p. 1
  • Deep-learning reconstruction methods improve image quality for a fixed acceleration factor and set of measurements, leaving trajectory optimization unaddressed.Collecting fewer k-space measurements can produce blur or aliasing artifacts.Source: §1 · p. 1
  • Earlier trajectory-optimization approaches used fixed trajectories at inference time, while [31] used a myopic training approximation that ignores sequential acquisition.The paper also describes subject-specific joint trajectory and reconstruction optimization in [5] as producing a slight image-quality increase at a fixed acceleration factor.Source: §1 · p. 2
  • The paper formulates active acquisition as a POMDP and applies deep reinforcement learning to optimize Cartesian sampling across acceleration factors while accounting for sequential decisions.It evaluates the approach on a large-scale single-coil knee dataset.Source: §1 · p. 2
  • The proposed approach outperforms common acquisition heuristics and the myopic approach of [31].The paper formulates active MRI acquisition as a POMDP and evaluates learned policies on a large-scale single-coil knee dataset.Source: §1 · p. 2

2 Background

The background introduces POMDPs as a framework for sequential decisions and describes Double DQN as the reinforcement-learning method used for high-dimensional problems with discrete actions.

2 Background

  • A POMDP represents sequential decisions through states, observations, actions, transitions, emissions, rewards, and a discount factor.A solution is a policy mapping observation histories to actions; here observations are k-space measurements and actions are measurement indices.Source: §2 · pp. 2–3
  • The paper identifies the NYU fastMRI Initiative database as the source of its data and directs readers to [27] for an overview of reinforcement learning in healthcare and medical imaging.The database note states that investigators provided data but did not participate in the analysis or writing.Source: §2 · p. 2
  • Double DQN estimates each action’s expected future cumulative reward and selects actions greedily by estimated value.Its value network is trained on replay-buffer samples using temporal-difference error and bootstrapped target values.Source: §2 · p. 3

3 Learning Active MR k-space Sampling

The method learns a policy that chooses unobserved Cartesian k-space columns from the current reconstruction and sampling mask. It casts acquisition as a sequential decision problem and solves it with DDQN variants.

3 Learning Active MR k-space Sampling

  • The fixed reconstruction model converts a zero-filled inverse-Fourier reconstruction into a de-aliased image, using a CNN proposed in [31].At each time step, the input is formed by masking the fully sampled k-space and applying the inverse Fourier transform.Source: §3 · p. 3

3.1 Active MRI Acquisition as POMDP

  • The policy maps the current de-aliased image and observed-frequency mask to an action selecting an unobserved k-space column.After sampling, the mask and reconstruction are updated; the objective minimizes aggregated reconstruction costs over future steps and across acquisition speed-ups.Source: §3.1 · pp. 3–4
  • The POMDP uses the hidden ground-truth image and current mask as state, while observations contain the reconstruction and mask; previously sampled columns are invalid actions.The mask transitions deterministically after a valid action, and the reconstruction model is assumed to return a deterministic reconstruction.Source: §3.1 · p. 4
  • The reward is the change in reconstruction metric between successive steps, corresponding to a final-step cost; the discount factor is treated as a hyperparameter.The authors report this reward was easier to optimize than rewards for more complex cost aggregations.Source: §3.1 · p. 4

3.2 Solving the Active MRI Aquisition POMDP with DDQN

  • DDQN uses the current observation as input because it captures the observation history, with subject-specific and dataset-specific value-network variants.The subject-specific network receives the reconstruction and mask; the dataset-specific variant receives time-step information and uses one trajectory across subjects. Previously observed columns are excluded, and modified epsilon-greedy exploration fills replay memory.Source: §3.2 · pp. 4–5

4 Experimental Results

The experiments evaluate learned DDQN acquisition policies against heuristics and baselines on single-coil knee MRI across two settings with different fixed low-frequency observations. Both DDQN models outperform the compared policies across most tested acceleration factors, while policy visualizations reveal differences in frequency-selection patterns.

4 Experimental Results

  • The test set contains 1,851 images from a held-out split of the fastMRI single-coil knee validation data, with vertical Cartesian masks and 332 possible column-acquisition actions.Training uses 536 volumes; validation and test sets contain 48 and 49 volumes, respectively.Source: §4 · p. 5
  • The comparison includes random, low-frequency-biased random, low-to-high, evaluator, and one-step oracle policies, with the same number of low-frequency observations fixed across methods.The evaluator uses the scoring function introduced by [31]; the oracle has test-time access to ground truth.Source: §4 · p. 5
  • Training varies acceleration factors and uses separate Scenario-30L and Scenario-2L reconstructors; policies are trained in stages, with DDQN evaluated using four reconstruction costs.Scenario-30L fixes 30 low frequencies, while Scenario-2L fixes 2 and covers accelerations up to 166×; DDQN training uses 5,000,000 state-transition steps.Source: §4 · pp. 5–6

4.1 Results

  • 3–7% lower mean MSE than evaluator at 4×–10× in Scenario-30L, and ds-ddqn beats lowToHigh by 10%–35% below 100× in Scenario-2L.Across the AUC comparisons, DDQN improvements over the best baselines range from 0.55% to 2.9% in Scenario-30L and 2.68% to 11.6% in Scenario-2L.Source: Figure 1 · p. 6, Table 1 · p. 6, §4.1 · p. 7
  • At 10× acceleration in Scenario-2L and 8× in Scenario-30L, subject-specific DDQN reconstructions appear sharper and more detailed than the three illustrated baselines.The examples show magnitude images and error maps for different knee images in the two scenarios.Source: §4.1 · p. 7, Figure 2 · p. 8

4.2 Policy Visualization

  • Learned policies tend to select low and middle frequencies first while introducing high frequencies relatively early; SSIM-based policies favor low frequencies more than MSE-based policies.The visualizations plot cumulative selection-time distributions by frequency; the SSIM policies show few selections in the upper-center region, consistent with the authors’ observation about Hermitian symmetry.Source: §4.2 · p. 8

5 Conclusion

The paper formulates active MRI acquisition as a POMDP and solves it with DDQN, reporting better performance than simple heuristics and the scoring function in [31] across four metrics. The authors also note a remaining gap to oracle performance and a simplified experimental setup that omits practical phase-encoding sampling issues.

5 Conclusion

  • On a large-scale single-coil knee dataset, DDQN policies outperform simple acquisition heuristics and the scoring function in [31] across MSE, NMSE, SSIM, and PSNR.Source: §5 · p. 8
  • Dataset-specific DDQN slightly outperforms subject-specific DDQN, but both remain below the oracle policy’s best possible performance.Source: §5 · p. 9
  • The experiments use a simplified setup that does not consider all practical MRI phase-encoding sampling issues.Source: §5 · p. 9

Supplementary

The supplementary figures compare reconstruction quality across acquisition strategies, visualize sampling policies, and show example reconstructions and error maps at different acceleration factors.

Supplementary

  • DDQN models outperform all baselines for most acceleration factors across the reported metrics.The figure compares reconstruction quality against acceleration factor; each DDQN model is trained with rewards based on its corresponding metric.Source: Figure 4 · p. 12
  • Policy visualizations cover heuristics and baselines in Scenario-2L, plus DDQN policies trained with NMSE and PSNR.DDQN policies trained with MSE and SSIM are shown in the main body.Source: Figure 5 · p. 13
  • Example reconstructions and error maps compare seven acquisition policies at 4×, 16×, and 64× acceleration in Scenario-2L.The columns show random, random-lb, lowToHigh, evaluator, ss-ddqn, ds-ddqn, and oracle policies.Source: Figure 6 · p. 14

Related papers

Loading 2007.10469v2…