Source-linked AI summary
Image reconstruction by domain transform manifold learning
Bo Zhu, Jeremiah Z. Liu, Bruce R. Rosen, Matthew S. Rosen
TL;DR
The paper models reconstruction from corrupted sensor-domain inputs by learning a denoising step together with a reconstruction mapping. Under smooth, low-dimensional manifold assumptions, the learned distributions converge toward the corruption and conditional-output distributions.
Problem
Reconstruction must account for corrupted sensor-domain inputs while inferring the corresponding output-image distribution.
Method
The approach combines a learned denoising operator with a reconstruction model and approximates the resulting continuously differentiable mapping using a neural network.
Results
As the KL-divergence approaches zero, the denoising and reconstruction models converge toward the corrupted-input and conditional-output distributions, respectively.
Takeaways & Limitations
Jointly collapsing denoising and reconstruction provides a predictive distribution for the output image given noisy input.
Takeaways & Limitations
The formulation assumes a smooth homeomorphic mapping between low-dimensional manifolds embedded in the ambient spaces.
Abstract
from arXiv · showhide
Image reconstruction plays a critical role in the implementation of all contemporary imaging modalities across the physical and life sciences including optical, MRI, CT, PET, and radio astronomy. During an image acquisition, the sensor encodes an intermediate representation of an object in the sensor domain, which is subsequently reconstructed into an image by an inversion of the encoding function. Image reconstruction is challenging because analytic knowledge of the inverse transform may not exist a priori, especially in the presence of sensor non-idealities and noise. Thus, the standard reconstruction approach involves approximating the inverse function with multiple ad hoc stages in a signal processing chain whose composition depends on the details of each acquisition strategy, and often requires expert parameter tuning to optimize reconstruction performance. We present here a unified framework for image reconstruction, AUtomated TransfOrm by Manifold APproximation (AUTOMAP), which recasts image reconstruction as a data-driven, supervised learning task that allows a mapping between sensor and image domain to emerge from an appropriate corpus of training data. We implement AUTOMAP with a deep neural network and exhibit its flexibility in learning reconstruction transforms for a variety of MRI acquisition strategies, using the same network architecture and hyperparameters. We further demonstrate its efficiency in sparsely representing transforms along low-dimensional manifolds, resulting in superior immunity to noise and reconstruction artifacts compared with conventional handcrafted reconstruction methods. In addition to improving the reconstruction performance of existing acquisition methodologies, we anticipate accelerating the discovery of new acquisition strategies across modalities as the burden of reconstruction becomes lifted by AUTOMAP and learned-reconstruction approaches.
Methods
AUTOMAP was trained with task-specific sensor encodings and target images while retaining a common network architecture, and its manifold formulation models denoising and reconstruction jointly. Experiments used ImageNet, HCP brain images, synthetic phase data, and multiple MRI acquisition strategies.
- Training and evaluation: ImageNet training data were cropped, subsampled, converted to grayscale luminance, rotated for augmentation, centered, and normalized.The generic dataset contained 10,000 images each from Animal, Plant, and Scene categories before preprocessing.
- Training and evaluation: HCP brain training data comprised axial, sagittal, and coronal T1-weighted slices from 131 subjects, producing a 50,000-image dataset after cropping and subsampling.Images were tiled by reflection and randomly cropped to promote translation invariance.
- Training and evaluation: Sensor representations were generated with discrete Radon, spiral NUFFT, Poisson-disc undersampled Cartesian, and misaligned k-space encodings for the reconstruction tasks.The undersampled Cartesian experiment used 40% Fourier k-space undersampling; the spiral trajectory used a 10-interleave design.
- Training and evaluation: Training paired each task’s sensor-domain encoding with a target image while using the same network architecture and hyperparameters across experiments.Different networks were trained for corresponding sensor encodings and target images; the undersampled task used HCP brain images, whereas several others used ImageNet.
- Manifold formulation: The manifold model assumes smooth, low-dimensional sensor and image manifolds and combines denoising with reconstruction into a continuously differentiable mapping approximable by a neural network.The denoising operator models corrupted sensor inputs, while the reconstruction mapping relates sensor and image manifolds; universal approximation provides the stated theoretical guarantee.