Source-linked AI summary
Solving ill-posed inverse problems using iterative deep neural networks
Jonas Adler, Ozan Öktem
TL;DR
Ill-posed inverse problems require reconstruction from indirect, noisy data without relying solely on unstable data fitting. The paper combines regularization-based prior information with a learned iterative update for nonlinear forward operators, and reports better tomographic reconstructions than FBP and TV. The approach also supports practical transfer through modest fine-tuning when the forward operator changes.
Problem
Ill-posed inverse problems can make data-fit reconstruction unstable, while fully learned approaches require rich data to cover manifolds, physics, and high-dimensional signals.
Method
A partially learned gradient-like iteration combines the forward operator, noise model, regularizer, and a convolutional network that learns the update component.
Results
Tomographic experiments show notably better reconstructions than traditional FBP and TV regularization.
Takeaways & Limitations
Incorporating prior knowledge about the forward operator, data acquisition, noise model, and regularizer can significantly improve deep-learning-based inverse reconstruction.
Takeaways & Limitations
A significant change in the training dataset, notably its scaling, requires retraining, although dependence was empirically relatively weak for the considered tomography problems.
Abstract
from arXiv · showhide
We propose a partially learned approach for the solution of ill posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularization theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularizing functional. The method results in a gradient-like iterative scheme, where the "gradient" component is learned using a convolutional network that includes the gradients of the data discrepancy and regularizer as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against FBP and TV reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the TV reconstruction while being significantly faster, giving reconstructions of 512 x 512 volumes in about 0.4 seconds using a single GPU.
1 Introduction
The paper frames inverse reconstruction as unstable estimation from indirect, noisy data and motivates combining classical regularization with learned inverse mappings. Its partially knowledge-driven approach targets nonlinear, large-scale problems while reducing the data and model burden of fully learned methods.
- Problem setting: Inverse problems reconstruct an unknown signal from indirect observations generated by a forward operator and corrupted by noise.The signal and data may be function-valued, and tomographic data can lie on a manifold of lines rather than ordinary Euclidean coordinates.
- Classical regularization: Ill-posed data-fit minimization is unstable, so maximum-likelihood reconstruction can overfit small data perturbations.The paper identifies over-fitting as a central reason for introducing regularization.
- Classical regularization: Classical regularization avoids over-fitting through approximate inverses, early-stopped iterative schemes, or variational penalties such as total variation.TV uses the functional S(f) := ∥∇f∥1 for bounded-variation signals.
- Machine learning approaches: Fully learned pseudo-inverses can require rich training data to cover data manifolds, encode the forward physics, and represent high-dimensional signals.The resulting parameter space can become infeasible to learn without sufficient data.
- Contribution and overview: The proposed knowledge-driven reconstruction separates a known forward-operator component from learned operators, avoiding explicit inclusion of the data manifold in learning.The paper extends this framework to nonlinear inverse problems, adds initialization and regularization, and applies it to nonlinear ray-transform tomography.
2 Solving inverse problems by learned gradient descent
The method learns an iterative update while retaining gradients from the data discrepancy and regularizer as problem-specific inputs. Persistent memory, learned optimization parameters, and a convolutional operator produce the final partially learned gradient scheme.
- 2.1 Motivation: The method begins by replacing the unavailable true-signal error gradient with a learned update informed by known data-discrepancy and regularizer gradients.This yields the initial partially learned gradient-descent scheme.
- 2.1 Motivation: Persistent memory lets later updates use information from earlier iterates, while the initial reconstruction can come from a pseudo-inverse.The resulting learned operator maps memory and gradient-related inputs to updated memory and reconstruction variables.
- 2.2 Learned updating operators: The learned updating operator is implemented by chained affine and pointwise nonlinear layers with weights, biases, and response functions.The architecture uses convolutional affine maps and can use ReLU responses.
- 2.2 Learned updating operators: The examples use 3 × 3 convolutions, N = 3 layers, 32 convolutions in each hidden layer, I = 10 iterations, and memory M = 5.These low settings were selected to reduce the parameter space and help reduce over-fitting; hyper-parameters were chosen by trial and error.
- 2.3 Partially learned gradient descent: After hyper-parameters are fixed, training learns scalar update parameters from data and applies the resulting operator to solve the inverse problem.Algorithm 3 initializes with T †(g), iterates the learned update, and uses the forward operator and data within the scheme.
3 Implementation and evaluation
The method was evaluated on simulated CT problems involving linear and nonlinear forward operators, multiple noise models, and comparisons with FBP and TV reconstruction. It achieved improved reconstruction quality over TV while remaining substantially faster, with gradient inputs also improving PSNR and visual detail.
- Datasets and setup: The evaluation used ellipse and head CT simulations with distinct forward operators and noise models.The ellipse data used sparse-view parallel-beam geometry with additive Gaussian noise, while the head data used nonlinear fan-beam tomography with Poisson noise.
- Datasets and setup: 500 CT scans comprising 41,000 slices were used for training the head reconstruction model.
- Comparison methods: The comparisons included FBP, TV regularization, and the partially learned iterative algorithm, using PSNR, runtime, and visual inspection.FBP used a Hann filter optimized for PSNR, while TV used the same data discrepancy and a PSNR-selected regularization parameter.
- Results: Approximately 2 dB PSNR improvement over TV was observed for ellipse data, alongside fewer random structures and less stair-casing.FBP performed very poorly under the high noise, while TV and the learned method produced comparable results.
- Results: >5 dB PSNR improvement over TV was obtained on the head dataset, while runtime was significantly faster than TV but slightly slower than FBP.The learned head reconstruction was described as smooth, with good performance around air-skin and bone boundaries.
- Impact of gradient mappings: With both gradients, the method reached 32.02 dB PSNR, compared with 30.51 dB using only the data-discrepancy gradient and 29.65 dB without gradients.The gradient-enhanced reconstructions were visually sharper and showed more detail, especially in small inserts.
4 Discussion
The discussion highlights computational and theoretical trade-offs of partially learned reconstruction, including fixed runtime, extensions to feature and nuisance-parameter reconstruction, and unresolved regularization theory.
- Theoretical issues: Theoretical analysis remains incomplete because deep neural-network theory and probabilistic convergence are not well established in infinite-dimensional settings.Open issues include operator approximation, topology selection, and weak convergence of probability measures.
- Theoretical issues: A formal proof of existence, stability, and convergence as a regularization method remains an open theoretical topic.Convergence generally requires retraining hyperparameters and training data as the data noise tends to zero, and zero-error convergence after noiseless retraining is not proved.
- Computational feasibility: The scheme has an a priori defined runtime adjustable through the number of iterates or the complexity of the updating operator.This allows optimization of the reconstruction scheme for a specified execution time.
- Computational feasibility: The method is reported to run significantly faster than TV regularization on the presented examples.
- Nuisance parameters: The learned scheme can be extended to intertwined reconstruction of signals and nuisance parameters.Each iteration can update the signal and nuisance parameters using the previous values of the other component.
- Feature reconstruction: The framework can be extended to feature reconstruction by composing the learned reconstruction operator with a differentiable feature-extraction operator.This supports end-to-end optimization of task-dependent reconstruction schemes for general inverse problems.
5 Conclusion and future work
The paper concludes that partially learned methods can combine inverse-problem priors with training data and outperform traditional tomographic reconstruction methods. It also identifies extensions involving transfer across acquisition settings, alternative iterative schemes, higher-dimensional problems, representations, and error functionals.
- Transfer across settings: Changing the forward operator requires only modest fine-tuning, with existing parameters usable as an initial guess.
- Conclusion: The method integrates prior information about the forward operator, data acquisition, noise model, and regularizer with learning from training data.
- Conclusion: Numerical experiments show notably better reconstructions than traditional FBP and TV regularization, with prior information improving reconstruction.
- Future work: Future work includes fully three-dimensional tomography, richer regularizers, alternative discretizations, and error functionals better aligned with human observer performance.
- Iterative schemes: The gradient-descent basis is sub-optimal for non-differentiable objectives, motivating nonlinear Chambolle–Pock and other iterative schemes.
- Iterative schemes: The approach can learn either the primal proximal, the dual proximal, or both, while dual-proximal learning can incorporate memory into the algorithm.