Source-linked AI summary
Self-Supervised Deep Active Accelerated MRI
Kyong Hwan Jin, Michael Unser, Kwang Moo Yi
TL;DR
Deep MRI methods commonly assume fixed sampling, while jointly optimizing acquisition and reconstruction is difficult because sampling is categorical. This paper trains progressive sampling and reconstruction networks with MCTS-generated supervision, and reports that the joint framework outperforms the state of the art.
Problem
Deep-learning MRI methods mainly reconstruct data under fixed sampling, while jointly optimizing sampling and reconstruction is difficult because sampling is categorical.
Source: §1 · p. 1Method
The framework jointly trains progressive sampling and reconstruction networks using MCTS-generated supervision signals, without direct supervision on sampling.
Source: §1 · p. 2Results
The framework is reported to outperform the state of the art by jointly learning sampling and reconstruction.
Source: §7 · p. 12Takeaways & Limitations
The authors present incorporating data acquisition into deep-network training as a promising direction for MRI.
Source: §7 · p. 12The paper notes that dataset-specific variable-density choices can require manual intervention, while offline power-spectrum selection may not match the test set.
Source: (a) Fully sampled MRI · p. 2, §2.2 · p. 3
Abstract
from arXiv · showhide
We propose to simultaneously learn to sample and reconstruct magnetic resonance images (MRI) to maximize the reconstruction quality given a limited sample budget, in a self-supervised setup. Unlike existing deep methods that focus only on reconstructing given data, thus being passive, we go beyond the current state of the art by considering both the data acquisition and the reconstruction process within a single deep-learning framework. As our network learns to acquire data, the network is active in nature. In order to do so, we simultaneously train two neural networks, one dedicated to reconstruction and the other to progressive sampling, each with an automatically generated supervision signal that links them together. The two supervision signals are created through Monte Carlo tree search (MCTS). MCTS returns a better sampling pattern than what the current sampling network can give and, thus, a better final reconstruction. The sampling network is trained to mimic the MCTS results using the previous sampling network, thus being enhanced. The reconstruction network is trained to give the highest reconstruction quality, given the MCTS sampling pattern. Through this framework, we are able to train the two networks without providing any direct supervision on sampling.
1 INTRODUCTION
The introduction motivates learning MRI acquisition and reconstruction together because limited scan time makes efficient sampling important, while existing deep methods largely keep sampling fixed. It presents a self-supervised, MCTS-guided framework for training both stages.
1 INTRODUCTION
- MRI scans acquire frequency-domain k-space data that are converted into spatial images, and patients can sustain a scan or breath-hold only for limited time.This motivates making the most of each scan.Source: §1 · p. 1
- Deep-learning MRI methods generally reconstruct data under a fixed sampling pattern, leaving acquisition design outside the learning process.The introduction characterizes these reconstruction-focused methods as passive.Source: §1 · p. 1
- 33.52 dB PSNR for the joint method versus 27.73 dB for variable-density sampling [2] illustrates the reported reconstruction gain.The comparison appears in Fig. 1 alongside sampling patterns, reconstructions, and a zoomed region.Source: Figure 1 · p. 1
- Joint optimization is difficult because sampling is categorical and non-differentiable, while exhaustive search and greedy alternatives are described as impractical.The passage notes that greedy search would require investigating too many cases when sampling and reconstruction are optimized jointly.Source: §1 · pp. 1–2
- The framework trains progressive sampling and reconstruction networks together, using MCTS-generated supervision for both stages without direct sampling supervision.MCTS improves on the current sampling policy; ReconNet is trained on MCTS-sampled data against the fully sampled image.Source: §1 · p. 2
2 RELATED WORKS
The related-work section frames accelerated MRI as partial k-space acquisition followed by reconstruction, then reviews compressed-sensing approaches that exploit image sparsity.
2 RELATED WORKS
- Accelerated MRI samples k-space partially and reconstructs higher-resolution images; the section organizes prior work around compressed sensing, sampling-pattern learning, and reconstruction.The figure contrasts full sampling with fewer-line acquisition and an aliased image.Source: §2 · p. 2
2.1 Compressive Sensing in Accelerated MRI
- Compressed-sensing reconstruction exploits sparse transform-domain coefficients as a prior, with random variable-density sampling measuring low frequencies more densely.The cited work describes iterative sparse reconstruction of sub-Nyquist measurements; Poisson-disc sampling is also noted as a way to reduce correlated effects.Source: §2.1 · p. 2
(a) Fully sampled MRI
The reviewed accelerated-MRI approaches include handcrafted and learned sampling patterns as well as neural reconstruction methods; the section notes practical limits in dataset transfer and fixed acquisition patterns.
(a) Fully sampled MRI
- Accelerated MRI acquires fewer k-space lines than fully sampled MRI and reconstructs the downsampled acquisition from an aliased image; compressed-sensing reconstructions rely on image-domain sparsity.Source: Figure 2 · p. 2
2.2 Learned Sampling patterns for Accelerated MRI
- Prior learned sampling approaches include k-space energy selection and greedy quality-based search, but their assumptions can limit test-set suitability or optimality.Offline power-spectrum selection may not match the test set, and greedy selection has no guarantee of an optimal pattern.Source: §2.2 · pp. 2–3
2.3 Neural Networks for Accelerated MRI
- CNN reconstruction methods use approaches such as residual learning, cascades with data-consistency constraints, magnitude/phase networks, and GANs, while retaining fixed sampling patterns.The paper states that fixed acquisition patterns limit the capabilities of these reconstruction methods.Source: §2.3 · p. 3
3 PROBLEM FORMULATION
The problem formulation treats accelerated MRI as choosing both a reconstruction process and a sampling pattern to estimate a full-resolution image from limited measurements. It contrasts this joint objective with prior approaches that hold one component fixed.
3 PROBLEM FORMULATION
- The signal model assumes a finite, discrete, complete image, with the full-resolution image represented as a vector in C^N.N denotes the number of image pixels.Source: §3 · p. 3
- The objective is to choose reconstruction parameters θ and sampling matrix P to maximize expected image quality over images, but Boolean P makes the optimization combinatorial.The formulation allows quality measures such as PSNR, Euclidean distance, or SSIM.Source: §3 · p. 3
- Traditional compressed sensing fixes the sampling pattern, deep-learning reconstruction methods also keep it fixed, and sampling-pattern methods instead fix reconstruction.The proposed formulation jointly optimizes θ and P rather than optimizing either component alone.Source: §3 · p. 3
4 METHOD
The method ties a reconstruction network to a progressive sampling network, then uses Monte Carlo tree search (MCTS) to generate supervision for both. ReconNet improves reconstructions from MCTS sampling patterns, while SampleNet learns to reproduce MCTS’s sampling policy.
4 METHOD
- The method trains two connected networks to jointly sample and reconstruct, using MCTS-generated targets rather than direct sampling supervision.The training strategy gives ReconNet MCTS sampling patterns and trains SampleNet to emulate MCTS’s policy.Source: §4 · p. 3, §4.2 · p. 4
4.1 Overall Framework
- ReconNet reconstructs high-resolution images from sampled data, while SampleNet uses the current reconstruction to estimate the next sampling position.The two networks are tied together in a progressive acquisition loop.Source: §4.1 · p. 3
4.1.1 ReconNet
- ReconNet operates in the image domain from a zero-filled backprojection of the sampled data.The sampling matrix marks observed positions, and its transpose pads unobserved signal values with zeros.Source: §4.1.1 · p. 4
4.1.2 SampleNet
- SampleNet progressively draws one sample at a time and outputs a probabilistic policy for the next position based on the current reconstruction.The sampling pattern is updated recursively as sampling proceeds.Source: §4.1.2 · p. 4
4.2 Self-Supervised Learning
- MCTS supplies improved sampling policies; ReconNet trains on its sampling patterns, while SampleNet learns to emulate its policy.A single search tree is kept throughout sampling, and the two networks are trained on the resulting patterns and data.Source: §4.2 · pp. 4–5, Figure 4 · p. 5
4.3 Monte Carlo Tree Search for Sampling
- MCTS searches sampling-pattern trees, simulates sampling to the budget, and backs up actual reconstruction rewards using average and maximum values; a value network yielded worse reconstructions.Selection uses an upper-confidence-bound strategy, and expansion is guided by SampleNet and ReconNet.Source: §4.3 · pp. 5–6, Figure 5 · p. 6
5 IMPLEMENTATION DETAILS
The implementation couples a reconstruction network with a sampling network and uses MCTS-generated supervision to train them. It adapts sampling to 1D readout and uses specified network architectures and training settings.
5.1 Sampling with 1D Readout
- The implementation uses a vector of sampling probabilities to choose a k-space line, matching the 1D readout used in Cartesian MRI.Each row of the sampling pattern corresponds to a sampling vector for the selected row or column.Source: §5.1 · p. 6
5.2.2 SampleNet
- SampleNet takes ReconNet’s output and produces a probability distribution over sampling lines for the next acquisition.Its convolutional blocks use max pooling, increasing channel counts up to 256 before fully connected layers and a softmax output.Source: §5.2 · p. 6, §5.2.2 · p. 6, Figure 6 · p. 7
5.2.1 ReconNet
- ReconNet is a fully convolutional network with eight residual blocks, chosen over a more complex U-Net after early experiments found similar performance at higher computation cost.Its blocks use 3 × 3 convolutions, batch normalization, and leaky-ReLU activation.Source: §5.2.1 · p. 6
5.2.2 SampleNet
- The training procedure uses MCTS to generate sampling supervision, then optimizes both networks using examples from experience memory.The algorithm repeats reconstruction episodes and sampling steps before updating the network parameters.Source: §5.2.2 · p. 7
5.3 Training Setup
- Training uses ADAM with a 10^-4 learning rate and weight decay, limits training iterations and epochs, and retains examples from the ten most recent optimization rounds.MCTS uses 10 simulation repetitions; supervision examples are generated in parallel with 16 threads, and training runs for 540 rounds.Source: §5.3 · p. 7
6 RESULTS
The evaluation uses cardiac and knee MRI datasets, with different preprocessing choices for their image characteristics. The supplied passages describe patient-based splits and image preparation.
6.1 Datasets
- The study evaluates the method on cardiac and knee image datasets.Source: §6.1 · p. 7
- The cardiac dataset contains 4,999 training images and 6,963 test images, split by patient after severe-noise images are discarded and images are cropped to 128 × 128.The data originate in the spatial domain and are transformed numerically into the Fourier domain.Source: §6.1 · p. 7
- The knee experiments use the central 2D slice of 3D volumes, crop images to 128 × 128, and compress eight-coil k-space data to one channel.The dataset contains fewer images than the cardiac dataset.Source: §6.1 · p. 7
- The cardiac dataset uses a dual-channel representation, while the knee dataset uses a square-root-of-sum-of-squares operation.The authors report that dual-channel representation helps for real-valued cardiac images.Source: §6.1 · p. 8
6.2 Baselines and Evaluation Metric
The evaluation compares standard accelerated-MRI baselines and variants using the paper’s reconstructor. All methods use the same training configuration and are assessed at fourfold upsampling with PSNR.
6.2.1 Baselines
- The baselines include variable-density sampling with TV reconstruction, learned compressive-sensing patterns with TV, and variable-density sampling with a deep CNN reconstructor.The LCS baseline uses publicly available code with tuned parameters.Source: §6.2.1 · p. 8
- Additional comparisons pair the paper’s ReconNet with lowpass filtering, variable-density sampling, and the learned sampling pattern from LCS.These variants assess reconstruction performance and the sampling pattern when used with ReconNet.Source: §6.2.1 · p. 8
6.2.2 Evaluation Protocol
- All methods use the same training configurations, and evaluation uses a sample budget of one-fourth the full-resolution signal with PSNR as the metric.This corresponds to upsampling by a factor of four.Source: §6.2.1 · p. 8, §6.2.2 · p. 8
6.3 Quantitative Results
The proposed method achieves the best reported PSNR on both cardiac and knee datasets, and qualitative examples also favor its reconstructions. Comparisons indicate that jointly learning sampling and reconstruction matters, while lowpass sampling behaves differently from other patterns.
6.3 Quantitative Results
- The proposed method performs best on both cardiac and knee datasets in the PSNR results.Table 1 reports both zero-filled inverse Fourier reconstructions and reconstructions using the compared methods.Source: §6.3 · pp. 8–9, Table 1 · p. 9
- Comparisons with LCS and FBPConv indicate that learning only sampling or only reconstruction does not provide optimal performance; the joint method also avoids per-dataset tuning required by TV methods.The authors report that simultaneous learning favors performance over variants using LCS sampling patterns with ReconNet.Source: §6.3 · p. 9
- Lowpass sampling gives high reconstruction quality by itself, but ReconNet performance decreases with this pattern because it omits high-frequency information; ReconNet improves performance with other patterns.Source: §6.3 · p. 9
- The method is described as easily scalable because it uses stochastic gradient-based optimization.Source: §6.3 · p. 9
6.4 Qualitative Results
- The proposed method gives the highest reconstruction quality in the qualitative examples and clearly reconstructs inner structures where competing methods show texture or folding artifacts.The examples include cardiac and knee datasets.Source: §6.4 · p. 9
6.5 t-SNE Analysis
The section examines reconstruction distributions and learned sampling evolution, then uses ablations to assess rollout, minimax backup, and linking the networks.
6.5 t-SNE Analysis
- On the cardiac test set, the reconstructed signals’ distribution and structure are very close to the ground truth in the t-SNE map.For clarity, the map includes a random one-fifth of images.Source: §6.5 · p. 9
6.6 Sampling Evolution
- The learned sampling pattern evolves from essentially random to a dominant pattern shared across training images, with minor image-dependent components.The authors relate the dominant shared component to the similar appearance of MRI data.Source: §6.6 · p. 9
6.7.1 Importance of MCTS Rollout
- The V-Net reward approximation gives significantly worse results than full simulation, so the authors do not retain it.Table 2 reports PSNR for the V-Net and the authors’ method without that approximation.Source: Table 2 · p. 9, §6.7.1 · p. 10
6.7.2 Effectiveness of Implicit Minimax Backup
- The backup ablation tests α = 0, 0.5, and 1.0, and the authors report that their method performs best.Table 3 presents the comparison in PSNR.Source: §6.7.2 · p. 10, Table 3 · p. 10
6.7.3 Using ReconNet Output as SampleNet Input
- Using ReconNet output to train SampleNet yields faster convergence and better final results than bypassing ReconNet.The comparison uses the cardiac dataset.Source: §6.7.3 · p. 11
7 CONCLUSION
The conclusion presents a self-supervised framework that learns progressive sampling and reconstruction together using Monte Carlo tree search-generated supervision. It reports that joint learning outperforms the state of the art and identifies implementation in an MRI device as future work.
7 CONCLUSION
- The framework learns progressive sampling and reconstruction jointly, using Monte Carlo tree search to generate supervision signals for its two networks.The authors say these sampling patterns improve reconstruction quality.Source: §7 · pp. 11–12
- The authors report that jointly learning sampling and reconstruction enables their framework to outperform the state of the art.Source: §7 · p. 12
- The authors identify implementation in an actual MRI device for high-speed data acquisition as future work.Source: §7 · p. 12