Source-linked AI summary
Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration
Mauricio Delbracio, Peyman Milanfar
TL;DR
Image restoration must recover a high-quality image from a degraded observation even though multiple reconstructions may be plausible, making direct regression prone to averaging away detail. InDI learns an iterative restoration path from paired examples, progressively improving the input without requiring an analytic degradation model, and reports higher-quality, high-fidelity results across four tasks.
Problem
Single-step supervised restoration averages plausible reconstructions of ill-posed inverse problems, while point-distortion metrics such as PSNR do not reliably match human perception.
Method
InDI iteratively restores the low-quality input through progressively less corrupted intermediate images learned from paired low- and high-quality examples, without requiring an analytic degradation model.
Results
InDI produces higher-quality samples than state-of-the-art regression formulations while maintaining high fidelity to the original sample across four restoration tasks.
Takeaways & Limitations
The formulation applies to virtually any degradation with paired training data and directly restores the input rather than denoising an initial pure-noise image.
Takeaways & Limitations
The approach assumes paired low-quality and high-quality training examples and may require a small amount of noise to ensure plausible intermediate samples.
Abstract
from arXiv · showhide
Inversion by Direct Iteration (InDI) is a new formulation for supervised image restoration that avoids the so-called "regression to the mean" effect and produces more realistic and detailed images than existing regression-based methods. It does this by gradually improving image quality in small steps, similar to generative denoising diffusion models. Image restoration is an ill-posed problem where multiple high-quality images are plausible reconstructions of a given low-quality input. Therefore, the outcome of a single step regression model is typically an aggregate of all possible explanations, therefore lacking details and realism. The main advantage of InDI is that it does not try to predict the clean target image in a single step but instead gradually improves the image in small steps, resulting in better perceptual quality. While generative denoising diffusion models also work in small steps, our formulation is distinct in that it does not require knowledge of any analytic form of the degradation process. Instead, we directly learn an iterative restoration process from low-quality and high-quality paired examples. InDI can be applied to virtually any image degradation, given paired training data. In conditional denoising diffusion image restoration the denoising network generates the restored image by repeatedly denoising an initial image of pure noise, conditioned on the degraded input. Contrary to conditional denoising formulations, InDI directly proceeds by iteratively restoring the input low-quality image, producing high-quality results on a variety of image restoration tasks, including motion and out-of-focus deblurring, super-resolution, compression artifact removal, and denoising.
1 Introduction
Single-step supervised restoration averages multiple plausible reconstructions, producing perceptually poor images as ill-posedness increases. InDI instead iteratively solves easier restoration problems through intermediate images to reduce this aggregation effect.
- Problem: Image restoration is ill-posed because multiple sharp, clean images can correspond to the same degraded observation.Supervised methods learn from paired low-quality and high-quality examples to infer the underlying image.
- Problem: Pixel-loss regression predicts the clean image directly from the degraded observation by minimizing an expected Lp reconstruction error.For p = 2, the MMSE-optimal prediction is the conditional expectation E[x|y].
- Problem: The conditional expectation averages plausible reconstructions, wiping out details and producing unnatural images through regression to the mean.This effect becomes more severe as the inverse problem admits more plausible reconstructions.
- InDI approach: InDI avoids single-step clean-image prediction by iterating inferences that reconstruct progressively less corrupted intermediate images.Each step addresses a less ill-posed problem because it predicts only a slightly less degraded image.
2 Background
InDI learns iterative restoration directly from paired low- and high-quality images rather than specifying an analytic degradation model. The formulation uses minimal assumptions and reports high-quality, high-fidelity results across four restoration tasks.
- InDI versus diffusion: Unlike diffusion formulations, InDI does not require an analytic degradation model and learns its iterative restoration process from paired low- and high-quality images.The procedure can therefore be applied to virtually any degradation when paired examples are available.
- Formulation: InDI is motivated by splitting the inverse problem into smaller restoration problems without modeling probability or conditional distributions at each step.The authors describe this as a unified formulation for supervised image restoration with minimal assumptions.
- InDI versus diffusion: Conditional DDPMs generate restored samples by repeatedly denoising an initial pure-noise image conditioned on the low-quality input, whereas InDI iteratively restores the input image.Both approaches use multiple steps, but their starting points and restoration procedures differ.
- Results: InDI produces higher-quality samples than state-of-the-art regression formulations while maintaining high fidelity to the original sample.The formulation is evaluated on four restoration tasks using different perceptual quality metrics.
3 Related Work
Prior restoration methods use direct regression, adversarial objectives, posterior sampling, or diffusion-based inversion, each emphasizing different perceptual, fidelity, or modeling properties. InDI decomposes restoration into easier intermediate problems while learning degradation implicitly from paired images.
- Regression methods: Direct regression methods achieve strong PSNR but produce an average of plausible solutions, reflecting the regression-to-the-mean problem.When the observation is completely obfuscated, the PSNR-optimal prediction is the average of the image distribution.
- Adversarial methods: Adversarial formulations push outputs toward the natural-image manifold but can suffer unstable training and significant hallucinations.The cited account attributes hallucinations to the non-reference objective used to minimize distance to the natural-image manifold.
- Perception-distortion trade-off: Perceptual quality and distortion cannot both be minimized simultaneously, so minimizing average point distortion can reduce perceptual quality.This trade-off is described for metrics such as PSNR.
- Generative methods: Posterior-sampling approaches can generate multiple plausible reconstructions for uncertainty quantification or fairness applications.Variational auto-encoders, normalizing flows, and diffusion probabilistic models have been applied to restoration tasks.
- Diffusion and InDI: Diffusion and score-based models generate high-quality samples by inverting a known degradation process.InDI instead decomposes restoration into sequentially easier problems and represents intermediate steps through paired low- and high-quality signals.
- Conditional diffusion: Conditional denoising diffusion models generate restored samples by repeatedly denoising pure noise while conditioning on the low-quality input.InDI shares an iterative structure but uses a different restoration path.
- Related formulations: Related ODE/SDE methods differ from InDI because InDI is formulated from elementary principles by splitting restoration into multiple smaller problems.The distinction is methodological rather than a claim about a specific benchmark result.
4 InDI: Our Proposed Formulation
InDI decomposes restoration into iterative, less ill-posed reconstructions from degraded input toward a clean target, using paired data rather than an explicitly known degradation. Its toy example and continuous formulation explain how small steps avoid regression to the mean and define a residual-flow interpretation.
- Forward degradation process: InDI defines intermediate degraded states between the clean target at t = 0 and degraded observation at t = 1.The forward process linearly connects paired clean and degraded images, with t serving as the diffusion-style time-step.
- Iterative restoration: At each reverse step, InDI estimates a slightly less corrupted state, converting the original inverse problem into a sequence of easier problems.Starting from the degraded input, the procedure repeatedly moves from time t to t −δ.
- Iterative restoration: The ideal procedure estimates the clean image at each time using E[x0 | xt], then uses that estimate to infer the state at the preceding time.Training uses a family of time-specialized regressors to support incremental reconstruction.
- Iterative restoration: The iteration starts from ˆx1 = y, while δ controls reverse-process speed and is typically set to 1/N for N steps.The practical scheme uses a constant step size with 0 < δ ≤ 1.
- A Toy Example:: In the multimodal toy example, iterative regression converges to one of four valid modes, whereas single-step regression produces their weighted-average, blurry reconstruction.The figure compares strong Gaussian denoising and missing-information recovery; green circles show iterative paths and black circles show final points.
- A Toy Example:: As δ approaches zero, the iterative scheme becomes a residual-flow ODE initialized at y at t = 1 and solved toward t = 0.The ODE’s right-hand side is the normalized residual of the inversion process, and standard ODE solvers are a possible extension.
- A Toy Example:: For Gaussian-noise degradation, InDI’s ODE coincides with the denoising score-matching probabilistic ODE, with exact correspondence when the regressor is the MMSE estimator.This connection is established through Tweedie’s formula under the stated ideal estimator condition.
- A Toy Example:: Small noise is added to the degraded input to ensure the conditional mean required by the iteration is well defined, improving image quality for some deterministic degradations.The regularity condition requires positive density at the current estimate; the perturbation can be constant or time-varying.
5 Experiments
InDI is evaluated across four image-restoration tasks using distortion and perceptual metrics, with iterative steps improving perceptual quality while preserving competitive fidelity. Results show stronger detail than regression baselines and comparable or competitive quality against generative methods.
- The framework is evaluated on motion deblurring, defocus deblurring, compression artifact removal, and single-image super-resolution using PSNR, LPIPS, FID, and KID.Experiments use paired image data across four restoration settings.
- 5.1 Motion Deblurring: On GoPro motion deblurring, iterative reconstruction produces more detail than Restormer and MAXIM, similar visual quality to DvSR, and state-of-the-art perceptual metrics with competitive PSNR.The comparison covers both visual detail and quantitative distortion/perceptual performance.
- 5.1 Motion Deblurring: More inference steps improve perceptual metrics, whereas a single step produces the best PSNR, exposing the perception–distortion trade-off.The paper states that distortion and perceptual quality cannot be optimized simultaneously.
- 5.2 Single-Image Super-resolution: For 4× super-resolution, the framework produces more defined structure and larger PSNR than RRDB, fewer high-frequency artifacts than SRFlow, and visual quality comparable to LDL.LDL provides slightly better fine-grain details, while the proposed method compares favorably with regression and generative baselines.
- 5.2 Single-Image Super-resolution: Adding a small amount of noise in deterministic super-resolution substantially improves perceptual quality, while zero noise yields a blurry reconstruction with high PSNR but low FID.The comparison is between ϵ = 0 and ϵ > 0 inputs.
- 5.3 Defocus deblurring: In defocus deblurring and JPEG artifact removal, multiple inference steps improve perceptual results by producing more texture and detail than direct regression or fewer-step reconstructions.For defocus deblurring, single-step reconstruction gives the best PSNR while multiple steps give the best perceptual metrics; JPEG results use quality factor 15.
6 Discussion
The discussion examines InDI’s generative interpretation, inference variants, training-noise choices, and comparison with conditional DDPMs. InDI can generate samples and achieve comparable results to DDPMs with substantially fewer inference steps.
- Generative interpretation: InDI can generate 64×64 CelebA samples from pure Gaussian noise, achieving FID=9.19 with 150 steps.The result is presented as an illustration rather than a state-of-the-art generative score.
- Inference algorithms: The proposed sampler outperforms naive and adapted Cold Diffusion inference at typical step counts, although Cold Diffusion is more stable at very large counts.The naive sampler performs well at N=2–3 but diverges; InDI deteriorates after sufficiently many steps.
- Training distribution: Training performs best when the distribution of t is biased toward t=1, corresponding to more degraded examples.The authors suggest this may reflect the need for greater certainty about the restoration direction early in the procedure.
- Input noise: Adding a small amount of noise improves JPEG compression removal, with no practical difference between per-step Brownian noise and constant initial noise.JPEG compression is described as nonlinear and deterministic.
- Comparison with conditional DDPM: InDI produces comparable results to a conditional DDPM using significantly fewer inference steps.The DDPM comparison combines multiple noise schedules to represent its perception–distortion tradeoff.
7 Conclusions and Limitations
The paper concludes that InDI addresses regression to the mean through iterative small-step restoration, yielding realistic, perceptually strong reconstructions. Its supervised, degradation-specific formulation has important data and inference-scope limitations.
- 7 Conclusions and Limitations: InDI circumvents regression to the mean by splitting restoration into many smaller, less ill-posed problems.The method transforms a degraded image into a high-quality image through iterative restoration, in a manner related in spirit to generative diffusion.
- 7 Conclusions and Limitations: The method produces superior realism and perceptual quality while maintaining low distortion error.This is the paper’s stated overall conclusion about restored-image quality.
- 7 Conclusions and Limitations: InDI requires paired training data and a specialized model for each degradation type.This contrasts with unsupervised formulations that use degradation operators at inference time.
- 7 Conclusions and Limitations: Performance on out-of-distribution samples is not guaranteed, and some tasks degrade after sufficiently many inference steps.The paper attributes the latter behavior likely to accumulated errors and calls for more robust inference schemes.
A Proof of Proposition 4.1
The proof derives the relationship between intermediate corrupted samples by expressing both x_s and x_t in terms of the clean image and eliminating y.
- A Proof of Proposition 4.1: The proof starts from x_s=(1−s)x+sy and x_t=(1−t)x+ty, then substitutes between these relations.This algebraic substitution establishes the connection between intermediate states.
- A Proof of Proposition 4.1: The derivation uses the conditional-density identity p_xs|x_t(x_s|x_t)=p_x|x_t(x|x_t) t/(t−s) together with x=tx_s−sx_t.These identities connect the intermediate variable x_s and clean variable x conditioned on x_t.
B Denoising with a Gaussian Prior
For Gaussian image priors with additive white Gaussian denoising, the appendix analyzes InDI’s continuous behavior and its limiting distribution. In the limit, InDI recovers the prior distribution rather than merely a point estimate.
- B Denoising with a Gaussian Prior: The appendix studies InDI when the prior p(x) is Gaussian and the observation is y=x+n with white Gaussian noise.The noise has a fixed standard deviation σ_N.
- B Denoising with a Gaussian Prior: The interpolated state satisfies x_t=(1−t)x+ty=x+tn, linking the continuous path to the noisy observation.This relation is used to analyze the restoration dynamics.
- B Denoising with a Gaussian Prior: InDI’s ideal ODE is solved at t=0 using the boundary condition x_1=y at t=1.The appendix formulates the denoising problem as integrating backward from the observed noisy sample.
- B Denoising with a Gaussian Prior: In the limit, InDI’s output has the same Gaussian mean and covariance as the prior, so its generated distribution equals p(x).The appendix states that InDI generates samples from the prior distribution in this limit.
- B Denoising with a Gaussian Prior: For this Gaussian case, the MMSE and MAP estimates coincide but differ from InDI’s estimate.The appendix explicitly contrasts these conventional estimates with InDI’s result.
C Model and Training Details
The restoration experiments use a U-Net-like architecture and common training settings, with task-specific model definitions summarized in Table 2.
- Architecture: All restoration experiments use a U-Net-like architecture adapted to be fully convolutional by removing self-attention and group normalization.The architecture is similar to those used in SR3 and DvSR.
- Model configurations: Table 2 summarizes the model definition for each tested restoration application.
- Training: Models are trained for 500K steps on 32 TPUv3 cores using Adam, a fixed learning rate, and EMA decay of 0.9999.For super-resolution, 64 × 64 low-resolution crops are bilinearly upscaled to 256 × 256 before model input.
D Additional Results
Additional results show InDI across multiple reconstruction-step settings for GoPro deblurring, 4× super-resolution, and JPEG artifact removal, alongside DDPD defocus-deblurring results.
- GoPro deblurring: Figures 13–15 present additional GoPro deblurring results using different numbers of InDI reconstruction steps.The results are identified as additional GoPro deblurring results for the proposed method.
- Super-resolution: Figures 16–19 present 4× super-resolution results on the DIV2K dataset using different numbers of InDI reconstruction steps.
- Artifact removal: Figures 20–23 present JPEG compression artifact removal results at quality factor 15 on the DIV2K test dataset using different numbers of InDI reconstruction steps.
- Defocus deblurring: Table 3 reports defocus deblurring results on the DDPD dataset, with best and second-best metric values color-coded and KID scaled by 1000.