Source-linked AI summary
One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models
J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan
TL;DR
Existing deep inverse solvers perform well but are problem-specific, whereas traditional priors apply broadly with weaker performance on difficult tasks. This paper learns a single projection-based prior for arbitrary linear inverse problems and reports comparable performance to specially-trained networks while improving robustness to operator and noise changes.
Problem
Specially-trained networks require separate training for different inverse problems, while traditional signal priors are broadly applicable but often perform worse on challenging tasks.
Method
The framework learns a projection operator from image data that acts as a proximal operator within ADMM and projects signals toward a classifier-defined natural-image set.
Results
The proposed method achieves comparable performance to specially-trained networks and is more robust to changes in linear operators and measurement noise.
Takeaways & Limitations
A single trained projection network can address diverse image inverse problems, reducing the scalability and deployment burden of problem-specific solvers.
Takeaways & Limitations
ADMM convergence is not guaranteed because the learned projection network may not reach the global optimum; early stopping is sometimes needed for harder problems.
Abstract
from arXiv · showhide
While deep learning methods have achieved state-of-the-art performance in many challenging inverse problems like image inpainting and super-resolution, they invariably involve problem-specific training of the networks. Under this approach, different problems require different networks. In scenarios where we need to solve a wide variety of problems, e.g., on a mobile camera, it is inefficient and costly to use these specially-trained networks. On the other hand, traditional methods using signal priors can be used in all linear inverse problems but often have worse performance on challenging tasks. In this work, we provide a middle ground between the two kinds of methods --- we propose a general framework to train a single deep neural network that solves arbitrary linear inverse problems. The proposed network acts as a proximal operator for an optimization algorithm and projects non-image signals onto the set of natural images defined by the decision boundary of a classifier. In our experiments, the proposed framework demonstrates superior performance over traditional methods using a wavelet sparsity prior and achieves comparable performance of specially-trained networks on tasks including compressive sensing and pixel-wise inpainting.
1 INTRODUCTION
Linear inverse problems are difficult because underdetermined measurements admit infinitely many feasible solutions, while existing priors and learned solvers each have important limitations. The paper proposes one learned projection operator that can be integrated with ADMM to solve diverse image inverse problems without retraining.
- Motivation: Underdetermined measurements leave infinitely many feasible solutions, but only a few correspond to natural images.The measurement model is y = Ax + n, with A often having a non-trivial null space.
- Motivation: Hand-designed signal priors apply broadly but are often too generic for challenging tasks such as inpainting and super-resolution.They may fail to constrain the solution set specifically to natural images.
- Motivation: Specially-trained neural solvers achieve strong performance but generally require retraining when the inverse problem or measurement operator changes.Examples include separate networks for different super-resolution factors or measurement types.
- Motivation: The paper asks whether a signal prior learned from a large image dataset can handle arbitrary linear inverse problems and reduce deployment costs.The motivation includes consumer products and specialized hardware.
- Proposed framework: The proposed framework learns a signal prior and projection operator, then integrates the same operator into ADMM for challenging image inverse problems.The projection operator is learned from large image datasets, while convergence conditions guide network design.
- Results: The method is reported as more robust than specially-trained networks to changes in linear operators and measurement noise.The authors contrast this robustness with the inefficiency and sensitivity of problem-specific networks.
2 RELATED WORK
Related work divides linear inverse methods between hand-designed priors, dataset-trained mappings, and generative models. The paper emphasizes the trade-off between broad applicability and task-specific performance or retraining cost.
- Hand-designed signal priors: Hand-designed priors regularize inverse problems using penalties such as φ(x) = ∥Wx∥1, where W produces sparse image features.ℓ1 priors offer convex optimization, efficient solvers, and theoretical guarantees under conditions such as restricted isometry and sufficient sparsity.
- Hand-designed signal priors: Hand-designed priors can admit non-image signals because sparse wavelet coefficients or gradients do not uniquely characterize natural images.Thus, they may not constrain the solution set to images.
- Learning-based methods: Learning-based methods use large datasets and neural networks to map measurements to images for tasks including super-resolution, compressive sensing, and deblurring.These methods have enabled strong performance on challenging image problems.
- Learning-based methods: End-to-end mappings are operator-specific, so changing the inverse problem generally requires retraining and increases the number of needed networks.A mapping trained for 2× super-resolution or Gaussian measurements cannot directly cover substantially different operators.
- Deep generative models: Deep generative models learn image distributions or generate samples, with inverse problems addressed through procedures such as maximum a posteriori estimation.This forms another dataset-based approach to incorporating image priors.
3 ONE NETWORK TO SOLVE THEM ALL
The framework learns a projection-based signal prior from image data and integrates it into ADMM so one network can address varied linear inverse problems. Its optimization behavior depends on proximal-operator assumptions and practical training and convergence choices.
- Motivation: Traditional priors are often too generic, while specially trained neural solvers usually require retraining for different inverse-problem instances.Examples include different super-resolution scales and measurement operators.
- Learning a proximal operator: The proposed approach directly learns a proximal operator that projects estimates toward the natural-image set and can be separated from the linear operator in ADMM.This allows the same learned prior to be used with arbitrary linear operators.
- Learning a proximal operator: A classifier learns a decision boundary for natural images, and a projection network is trained to approximate the corresponding proximal operation.The projection network then replaces the proximal update within ADMM.
- Convergence: Theoretical convergence requires suitable smoothness and sufficiently large ρ, while nonconvex classification costs do not guarantee general convergence.Theorem 1 guarantees attainment of a stationary point under its stated assumptions.
- Implementation details: The projector and classifier are jointly trained adversarially in image and latent spaces, with the projector producing outputs closer to the natural-image dataset.The latent classifier is added to support training and help avoid overfitting.
- Limitations: The prior weight is fixed after training, and challenging large-block inpainting or 4× super-resolution cases may require stopping ADMM early.Adjusting ρ sometimes has effects similar to adjusting λ.
4 EXPERIMENTS
The experiments evaluate the proposed projection network across datasets and inverse problems, comparing it with specially trained networks and hand-designed priors. Results emphasize broader task robustness, while also identifying convergence and failure cases.
- Experimental setup: The evaluation spans MNIST, MS-Celeb-1M, and ImageNet across compressive sensing, inpainting, denoising, and super-resolution tasks.The experiments vary measurement ratios, missing-pixel patterns, noise levels, and resolution factors.
- Experimental setup: The proposed framework is compared with specially-trained networks, including networks deliberately omitted for 4× super-resolution and scattered inpainting.This tests whether specialized models generalize beyond their training configurations.
- Comparison with hand-designed priors: For denoising, the proposed method consistently outperforms BM3D when σ is larger than 40.The comparison uses the ImageNet-trained projection network on 64 × 64 patches and sets A = I.
- Robustness: Specialized networks overfit their operators: the blockwise inpainting model fails on smaller scattered holes, and the 2× super-resolution model fails on 4× reconstruction.The compressive sensing specialized network also degrades when the measurement matrix changes or Gaussian noise is added.
- Convergence: Exponential linear units produce lower and less spiky RMS error than leaky rectified linear units, indicating a more stable ADMM process.The result supports following the sufficient convergence condition associated with Theorem 1.
- Failure cases: The framework can fail on ImageNet blockwise inpainting, where increased ρ may improve convergence but yield low-quality, overly smoothed outputs.The failure is associated with the projection network not projecting inputs onto the natural-image set assumed by Theorem 1.
5 CONCLUSION
The paper proposes a projection-operator framework that implicitly learns a signal prior for generic linear inverse problems, including denoising, inpainting, super-resolution, and compressive sensing.
- The framework implicitly learns a signal prior as a projection operator for solving generic linear inverse problems.
APPENDIX
The appendix describes the projection network and classifier architectures used in the paper, along with shorthand definitions for their components and bottleneck variants.
- Component definitions: The appendix defines convolution, transpose convolution, virtual batch normalization, bottleneck variants, channel-wise fully connected layers, and fully connected layers used in the networks.
- Projection network P: The projection network P is an autoencoder whose encoder maps inputs to a 1024-dimensional latent space and whose decoder maps latent representations back to image space.
- Projection network P: The encoder uses convolutional layers followed by a channel-wise fully connected layer, producing a 1024-channel latent representation.
- Projection network P: The decoder is a symmetric counterpart of the encoder and uses transpose-convolution layers to produce the output image.
- Classifiers: Two classifiers are used: an image-space classifier distinguishes natural images from projection outputs, while a latent-space classifier operates on encoder outputs.
- Classifiers: The image-space classifier uses a 50-layer architecture with bottleneck blocks, while the latent-space classifier uses fewer blocks because its input dimension is smaller.