Source-linked AI summary
Implicit Diffusion Models for Continuous Super-Resolution
Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, Baochang Zhang
TL;DR
Super-resolution methods often suffer from quality problems and fixed-magnification constraints. IDM unifies implicit neural representations with diffusion denoising and scale-adaptive conditioning, and extensive experiments report state-of-the-art performance with continuous-resolution outputs.
Problem
Existing SR methods often suffer from suboptimal quality and fixed-resolution outputs, while multiple magnifications can require cascaded pipelines, two-stage training, or retraining.
Method
IDM combines a denoising diffusion model with coordinate-based implicit image functions and scale-adaptive conditioning using an LR network and scaling factor.
Results
IDM achieves state-of-the-art qualitative and quantitative results on natural and facial image SR benchmarks while producing high-fidelity resolution-continuous outputs.
Takeaways & Limitations
The framework provides end-to-end continuous-resolution SR while dynamically balancing LR information and generated fine details.
Abstract
from arXiv · showhide
Image super-resolution (SR) has attracted increasing attention due to its wide applications. However, current SR methods generally suffer from over-smoothing and artifacts, and most work only with fixed magnifications. This paper introduces an Implicit Diffusion Model (IDM) for high-fidelity continuous image super-resolution. IDM integrates an implicit neural representation and a denoising diffusion model in a unified end-to-end framework, where the implicit neural representation is adopted in the decoding process to learn continuous-resolution representation. Furthermore, we design a scale-controllable conditioning mechanism that consists of a low-resolution (LR) conditioning network and a scaling factor. The scaling factor regulates the resolution and accordingly modulates the proportion of the LR information and generated features in the final output, which enables the model to accommodate the continuous-resolution requirement. Extensive experiments validate the effectiveness of our IDM and demonstrate its superior performance over prior arts.
1. Introduction
Existing super-resolution methods face quality and fixed-resolution limitations. IDM combines implicit representations, diffusion denoising, and scale-adaptive conditioning to produce high-fidelity outputs across continuous resolutions.
- Existing SR models often produce suboptimal-quality images and require fixed-resolution outputs, restricting practical use.
- Regression methods can support continuous outputs but often miss high-fidelity details, while generative methods add textures yet commonly produce artifacts or fixed-magnification results.
- IDM formulates continuous image super-resolution as a denoising diffusion process combined with a coordinate-based implicit image function.The implicit function is incorporated into the diffusion model to represent images continuously across resolutions.
- IDM uses a scale-adaptive conditioning mechanism with an LR conditioning network and scaling factor to adjust LR information against generated fine details.The mechanism dynamically controls this balance for size-varied SR outputs.
- IDM is evaluated on natural and facial image SR benchmarks and reports state-of-the-art qualitative and quantitative results with high-fidelity resolution-continuous outputs.
2. Related Work
Prior work includes implicit representations for continuous image modeling and several generative SR families. IDM combines diffusion refinement with implicit continuous representations to address detail quality and variable-resolution output requirements.
- Implicit neural representations use functions such as MLPs to model continuous image or scene representations and have been extended from 3D to 2D images.
- IDM introduces diffusion denoising to recover realistic details while retaining the continuous image function associated with implicit representations.
- Generative Image Super-Resolution Models: Regression-based SR methods can achieve continuous outputs but commonly produce duller details, whereas GANs, flows, VAEs, and autoregressive models involve distinct quality or computational limitations.
- Generative Image Super-Resolution Models: Diffusion models have achieved strong synthesis results and have been applied to SR through repeated refinement and conditioning mechanisms.
3. Method
IDM combines denoising diffusion with implicit neural representations in an end-to-end U-Net framework to produce high-fidelity outputs at continuous resolutions. Its scale-adaptive conditioning uses LR features and a scaling factor to control resolution and fuse input information with generated details.
- Problem Statement: IDM formulates continuous image super-resolution as a denoising diffusion process conditioned on LR-HR image pairs and a continuous scaling factor.The model learns p(y | x) through a fixed Markov chain and trains across randomly sampled scales s ∼ U(1, M].
- Implicit Diffusion Model: A U-Net denoising model encodes noisy images into multi-resolution features, while implicit image functions in its decoder generate resolution-continuous outputs.The implicit image function is inserted into the U-Net upsampling layers and unified with iterative diffusion refinement.
- LR Conditioning Network: The LR conditioning network extracts multi-resolution features from LR images without relying on additional priors.It initializes LR features with EDSR, aligns them to the noisy image resolution, and progressively downsamples them through convolution, bilinear filtering, and leaky ReLU operations.
- Scaling Factor Modulation: The scaling factor controls continuous output resolution and adjusts the balance between conditioning-network LR information and denoising-network features.An adaptive MLP maps s to scaling vectors that are normalized and applied channel-wise to fuse the two feature sources.
- Implicit Neural Representation: Coordinate-based MLPs in the U-Net upsampling path learn continuous image representations from features and their corresponding coordinates.The implicit representation uses current features around reference coordinates to calculate target features through a two-layer MLP.
4. Experiments
Experiments across face, natural-image, and general-scene benchmarks show that IDM produces high-fidelity super-resolution with continuous magnification control. It preserves identities and textures qualitatively, improves quantitative metrics over prior methods, and remains effective beyond its training scale range.
- Experimental Setup: IDM is evaluated on face, natural-image, and general-scene SR benchmarks using qualitative and quantitative comparisons with prior methods.Experiments include CelebA-HQ, LSUN, and DIV2K, alongside comparisons to SR3, GLEAN, LIIF, and other generative models.
- Qualitative Comparisons: IDM maintains facial identities and high-fidelity details more effectively than SR3 in 8× face super-resolution.The comparison highlights more faithful eyes, teeth, wrinkles, and other facial attributes.
- Qualitative Comparisons: IDM reconstructs detailed textures more effectively than GLEAN on 16× LSUN natural-image super-resolution.The reported examples include facial features, windows, doors, curtains, wall pictures, and lamps.
- Quantitative Comparisons: 0.97 dB higher PSNR, 0.06 higher SSIM, and 0.53 lower Consistency distinguish IDM from SR3 on 8× face super-resolution.The passage reports better IDM results on all three metrics.
- Quantitative Comparisons: 0.60 dB, 0.89 dB, and 1.03 dB PSNR improvements over GLEAN are reported across LSUN categories, with lower LPIPS throughout.IDM outperforms GLEAN on all reported categories despite GLEAN using pretrained latent banks.
- General Scene Dataset: 0.50 dB on PSNR and 0.03 on SSIM are the reported improvements over other generative methods on DIV2K 4× SR.IDM retains this advantage even with 800 training images versus 2,800 for the compared methods.
- Continuous SR: IDM produces realistic continuous-resolution results at arbitrary magnifications, including 9× and 10× outside the training range (1, 8].Regression methods show over-smoothing, while SR3 encounters extreme distortions when magnification changes.
- Ablation Studies: The scaling factor controls the balance between LR information and generated details, with smaller assigned magnifications causing blurred textures for 8× face SR.The LR conditioning network was separately tested on 16× Cats SR against upsampled-LR and EDSR-feature conditioning mechanisms.
5. Conclusion
The paper concludes that IDM combines implicit image functions with diffusion denoising for high-fidelity continuous-resolution super-resolution. Its scale-adaptive conditioning dynamically balances LR information and generated details, and experiments report state-of-the-art performance.
- 5. Conclusion: IDM introduces an implicit image function into diffusion-model decoding for end-to-end high-fidelity continuous-resolution image super-resolution.The framework iteratively combines diffusion denoising with implicit neural representation.
- 5. Conclusion: The scale-adaptive conditioning mechanism dynamically adjusts the proportion of LR information and generated details according to output size.The mechanism uses a low-resolution image condition and a controllable scaling factor.
- 5. Conclusion: Extensive experiments indicate that IDM achieves state-of-the-art performance.