Source-linked AI summary
RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
Jiaming Liu, Yu Sun, Cihat Eldeniz, Weijie Gan, Hongyu An, Ulugbek S. Kamilov
TL;DR
Learned RED priors often depend on fully sampled groundtruth, which can be impractical to acquire for some imaging problems. RARE broadens RED to artifact-removal CNNs and learns priors from undersampled, noisy measurements; experiments in free-breathing MRI support its practical relevance and competitive image quality.
Problem
Some imaging settings cannot practically provide fully sampled groundtruth for training learned reconstruction priors.
Method
RARE uses artifact-removal CNNs within iterative reconstruction, with Artifact2Artifact training from pairs of undersampled and noisy measurements.
Results
RARE reconstructs free-breathing whole-body 3D MRI into ten respiratory phases from heavily undersampled k-space measurements in simulated and experimental data.
Takeaways & Limitations
The results indicate competitive image quality against several baseline methods and practical relevance for priors learned directly from undersampled and noisy data.
Takeaways & Limitations
The learning strategy assumes that multiple undersampled acquisitions provide complementary information whose expected images match the groundtruth vector.
Abstract
from arXiv · showhide
Regularization by denoising (RED) is an image reconstruction framework that uses an image denoiser as a prior. Recent work has shown the state-of-the-art performance of RED with learned denoisers corresponding to pre-trained convolutional neural nets (CNNs). In this work, we propose to broaden the current denoiser-centric view of RED by considering priors corresponding to networks trained for more general artifact-removal. The key benefit of the proposed family of algorithms, called regularization by artifact-removal (RARE), is that it can leverage priors learned on datasets containing only undersampled measurements. This makes RARE applicable to problems where it is practically impossible to have fully-sampled groundtruth data for training. We validate RARE on both simulated and experimentally collected data by reconstructing a free-breathing whole-body 3D MRIs into ten respiratory phases from heavily undersampled k-space measurements. Our results corroborate the potential of learning regularizers for iterative inversion directly on undersampled and noisy measurements.
1 Introduction
Image reconstruction commonly combines measurement fidelity with an imaging prior, but learned RED priors can require fully sampled groundtruth that some acquisition settings cannot provide. RARE broadens RED to artifact-removal priors trained from undersampled and noisy measurements, and demonstrates this approach for free-breathing motion-compensated MRI.
- Motivation: Image reconstruction from undersampled observations is typically formulated by combining data fidelity with a regularizing imaging prior.The prior addresses ill-posedness while the data-fidelity term enforces consistency with the measurement model.
- Prior work: RED uses learned CNN denoisers as imaging priors and has achieved strong performance across imaging problems.The cited background places RED among denoiser-based reconstruction frameworks with demonstrated results in multiple applications.
- Motivation: Fully sampled groundtruth can be impractical to collect when the measurement setup fundamentally limits acquisition.This constraint creates a practical challenge for training corresponding learned priors.
- Contribution: RARE broadens RED by using CNNs trained for general artifact removal rather than only additive white Gaussian noise removal.The proposed framework retains the iterative reconstruction setting while expanding the type of learned prior it can use.
- Contribution: Artifact2Artifact trains imaging priors from pairs of artifact- and noise-contaminated images obtained directly from undersampled measurements.This strategy is inspired by Noise2Noise and avoids requiring fully sampled groundtruth training data.
- Evaluation: RARE is evaluated on simulated and experimental free-breathing motion-compensated MRI data, reconstructing 4D images from heavily undersampled acquisitions.The reported experiments support the practical relevance of learning regularizers directly from undersampled and noisy measurements.
2 Background
The background develops inverse-problem, deep-learning, and denoiser-prior perspectives on reconstruction from undersampled measurements. It motivates RARE as a RED generalization that can use artifact-removal CNNs trained on undersampled and noisy data while retaining RED’s theoretical structure.
- Imaging inverse problems: In parallel MRI, the measurement operator combines Fourier transformation, coil sensitivity maps, and time-dependent k-space sampling.The operator uses F for the Fourier transform, S_i for the ith coil sensitivity map, and P^(t) for sampling at time t.
- Imaging inverse problems: Undersampled k-space makes the inverse problem ill-posed, motivating regularized inversion methods.Conventional approaches use sparsity, total variation, or spatio-temporal correlations, often producing nonsmooth optimization problems.
- Optimization background: Proximal-gradient and ADMM methods minimize nonsmooth reconstruction objectives using proximal operators rather than differentiating the regularizers.The proximal operator can be interpreted as a regularized image denoiser for AWGN with variance µ.
- Deep learning: Traditional deep-learning reconstruction trains networks to invert measurement operators, commonly mapping low-quality images to clean images using supervised targets.For CS-MRI, prior methods trained mappings from zero-filled images to reconstructions using fully sampled groundtruth data.
- Learning from noisy data: Noise2Noise shows that networks can learn from pairs of independently degraded images when the noisy input has the same expected value as the clean signal.This principle motivates learning without clean labels, although the degradation assumptions matter.
- Deep learning: End-to-end unfolded reconstruction can impose a substantial memory burden because intermediary activation maps must be stored across iterations.The burden is especially relevant for higher-dimensional inverse problems using complex 3D CNNs.
- Denoiser-based priors: PnP replaces the proximal operator with an arbitrary denoiser, while RED uses a denoiser to define an explicit regularizer with a simple gradient under stated assumptions.These frameworks enable advanced denoisers without requiring every denoiser to correspond to an explicit conventional prior.
- RARE: RARE generalizes RED to artifact-removal CNNs, enabling priors learned from undersampled and noisy data rather than only denoising data.The framework is presented as compatible with RED’s explicit-regularizer and simple-gradient properties under the same assumptions.
3 Regularization by artifact removal
RARE extends RED from denoising priors to artifact-removal networks, enabling iterative reconstruction with priors trained solely on undersampled and noisy measurements. Its framework combines data consistency with artifact-removal residuals and is evaluated on simulated 4D MRI using a groundtruth-free training strategy.
- Framework: RARE broadens RED by replacing denoising priors with artifact-removal networks that can target modality-specific artifacts.The framework retains RED’s explicit regularizer and simple-gradient properties under the same assumptions.
- Framework: RARE initializes reconstruction with an artifact-removal network and iteratively combines data-fidelity gradients with the network residual x − Rθ(x).Its operator is G(x) := ∇g(x) + τ(x − Rθ(x)), with τ > 0 controlling regularization.
- Optimization: The algorithm uses backtracking line search and Nesterov acceleration to adjust the step size and improve convergence.Backtracking decreases γ by β when the residual norm fails to decrease; setting qk = 1 removes acceleration.
- Evaluation: RARE achieves close agreement with RED on simulated data despite training its prior only with undersampled and noisy measurements rather than groundtruth images.The evaluation reports PSNR and SSIM across respiratory phases and compares sampling rates corresponding to 10×, 6.6×, and 5× acceleration.
- Learning without Groundtruth: Artifact2Artifact trains the imaging prior from pairs of artifact-contaminated images generated by applying measurement-operator pseudoinverses to repeated undersampled acquisitions.The artifacts need not be AWGN, while complementary information across the dataset supports training without fully sampled groundtruth.
- 4D MRI implementation: In 4D MRI, the CNN maps complex 3D volumes from different acquisition times, exploiting distinct artifact patterns across respiratory phases.The implementation applies a 3D CNN along x, y, and respiratory-phase dimensions and uses a mixed ℓ1/ℓ2 loss.
4 Numerical Validation
RARE was evaluated on simulated and experimentally collected MRI data against conventional, learning-based, and denoising-prior baselines. Across severe undersampling settings, it achieved competitive or improved reconstruction quality, sharper experimental images, and effective artifact reduction within about 10 iterations.
- Simulated data: RARE was quantitatively validated on simulated radial measurements at 10%, 15%, and 20% sampling rates with 30 dB and 40 dB input SNR.The simulated experiments used PSNR and SSIM against groundtruth images, with up to 500 iterations.
- Simulated data: Learning-based methods, including RARE, UNet3D, and RED-DnCNN*, outperformed k-t SLR and CS on the simulated reconstructions.The reported comparisons used PSNR and SSIM, while k-t SLR produced cartoon-like features and CS failed to preserve some details.
- Simulated data: RARE achieved comparable imaging quality to RED-DnCNN* despite training its prior only on zero-filled reconstructions rather than groundtruth images.RED-DnCNN* used the actual groundtruth images employed to synthesize the simulated measurements.
- Experimental data: On 400-spoke experimental scans, RARE improved A2A by using measurement-operator information, producing sharper results than direct artifact-removal CNN reconstruction.UNet3D produced blurring that reduced tumor contrast, while A2A showed streaking artifacts; RARE reduced those artifacts after A2A initialization.
- Experimental data: RARE reduced streaking artifacts effectively, with no significant visual improvements observed after 10 iterations from A2A initialization.The iterative reconstructions were examined over eight iterations, with differences amplified 20× for visualization.
- Experimental data: For 400, 800, and 1200 spokes, RARE generally provided improved artifact removal and sharper, higher-contrast images than the compared learning-based methods.At 1200 spokes, all methods improved substantially, while RARE showed slightly better visual quality; reference-based rPSNR and rSSIM require caution because references were not groundtruth.
5 Conclusion
RARE reconstructs images using priors learned directly from undersampled and noisy data, avoiding the need for fully sampled training data. Experiments support competitive image quality and suggest applicability beyond MRI to modalities where fully sampled measurements are difficult to obtain.
- Conclusion: RARE uses imaging priors learned directly from undersampled and noisy measurements.The proposed framework is intended for reconstruction settings where fully sampled training data are unavailable.
- Conclusion: Experiments indicate that RARE provides competitive image quality compared with several baseline methods.The reported validation focused on MRI and included extensive experimental results.
- Conclusion: Although demonstrated on MRI, RARE is broadly applicable to imaging modalities where several distinct object views are easy to collect for training.The paper identifies CT and optical diffraction tomography as potential applications where fully sampled measurements are often impossible to obtain.