Source-linked AI summary
Unsupervised Degradation Representation Learning for Blind Super-Resolution
Longguang Wang, Yingqian Wang, Xiaoyu Dong, Qingyu Xu, Jungang Yang, Wei An, Yulan Guo
TL;DR
Blind SR is difficult when real degradations differ from fixed assumptions and degradation estimation is costly or error-prone. The paper learns unsupervised degradation representations with contrastive learning and uses them in a degradation-aware SR network. Experiments report accurate degradation information and state-of-the-art blind-SR performance across varied degradations.
Problem
Fixed-degradation SR methods can fail on real images with different degradations, while estimation-based blind SR is vulnerable to costly or inaccurate degradation estimates.
Method
The method uses contrastive learning to distinguish same-degradation and different-degradation image patches in a latent representation space without ground-truth degradation supervision.
Results
State-of-the-art blind-SR performance is reported across various degradations, with experiments covering synthetic and real-world images.
Takeaways & Limitations
Learned degradation representations provide accurate degradation information for flexible degradation-aware SR adaptation without explicit degradation estimation.
Takeaways & Limitations
The formulation assumes degradation is constant within each image and varies across different images.
Abstract
from arXiv · showhide
Most existing CNN-based super-resolution (SR) methods are developed based on an assumption that the degradation is fixed and known (e.g., bicubic downsampling). However, these methods suffer a severe performance drop when the real degradation is different from their assumption. To handle various unknown degradations in real-world applications, previous methods rely on degradation estimation to reconstruct the SR image. Nevertheless, degradation estimation methods are usually time-consuming and may lead to SR failure due to large estimation errors. In this paper, we propose an unsupervised degradation representation learning scheme for blind SR without explicit degradation estimation. Specifically, we learn abstract representations to distinguish various degradations in the representation space rather than explicit estimation in the pixel space. Moreover, we introduce a Degradation-Aware SR (DASR) network with flexible adaption to various degradations based on the learned representations. It is demonstrated that our degradation representation learning scheme can extract discriminative representations to obtain accurate degradation information. Experiments on both synthetic and real images show that our network achieves state-of-the-art performance for the blind SR task. Code is available at: https://github.com/LongguangWang/DASR.
1. Introduction
Blind SR must handle unknown degradations because fixed-degradation CNN methods can fail when real degradations differ. The paper replaces explicit degradation estimation with unsupervised representation learning and degradation-aware adaptation.
- Fixed-degradation CNN SR methods suffer severe performance drops when real degradations differ from assumed degradations such as bicubic downsampling.
- Prior methods for varied degradations often require the test degradation to be known, while blind methods rely on degradation estimation that can introduce artifacts.Estimation errors may be magnified by the SR network.
- The proposed scheme learns degradation representations by contrasting positive and negative pairs in latent space instead of explicitly estimating degradation in pixel space.This approach distinguishes degradations without requiring ground-truth degradation supervision.
- The method assumes degradation is constant within each image but varies across images, making same-image patches positive pairs and cross-image patches negative pairs.
- The degradation-aware SR network adapts to different degradations using the learned representations, with experiments reporting promising blind-SR results on synthetic and real-world images.The network predicts convolutional kernels and channel-wise modulation coefficients from the degradation representation.
2. Related Work
Related work spans CNN-based SR for fixed or multiple degradations, zero-shot adaptation, degradation estimation, and contrastive representation learning. These approaches motivate blind SR based on learned degradation representations.
- SR with Single Degradation: CNN-based SR methods have achieved strong performance but are generally tailored to fixed bicubic degradation.
- SR with Multiple Degradations: Multiple-degradation methods incorporate degradation information into SR, but blind use depends on estimating the degradation for the input image.
- SR with Multiple Degradations: Zero-shot SR adapts at test time but ZSSR requires thousands of iterations and is time-consuming, while MZSR reduces adaptation to a few iterations through meta-learning.
- SR with Multiple Degradations: Degradation estimation errors can introduce undesired artifacts, motivating iterative correction methods such as IKC and DAN.
- Contrastive Learning: Contrastive learning maximizes mutual information in representation space, and this paper uses same-degradation patches as positive counterparts to learn content-invariant degradation representations.
3. Methodology
The blind SR framework learns degradation representations from LR patches and uses them to condition a degradation-aware SR network. Contrastive learning separates patches by degradation, while adaptive convolutions exploit the resulting representation for SR.
- 3.1. Problem Formulation: The degradation model forms an LR image by blurring an HR image, downsampling it, and adding noise.The paper uses bicubic downsampling and studies isotropic, anisotropic, noisy, and real-world degradations.
- 3.2. Framework: The framework encodes the LR image into a degradation representation, then feeds that representation into DASR to produce the SR result.It consists of a degradation encoder and a degradation-aware SR network.
- 3.2.1. Degradation Representation Learning: Contrastive learning treats patches from one LR image as positive samples and patches from other images as negative samples.The encoder and projection head produce representations whose positive pairs are attracted and negative pairs repelled.
- 3.2.1. Degradation Representation Learning: A queue supplies diverse negative samples so the learned representation can remain insensitive to image content while distinguishing degradations.Training samples two patches from each of B LR images and maintains a queue of representations.
- 3.2.1. Degradation Representation Learning: Unlike pixel-level degradation estimation, the scheme learns an abstract degradation representation in a single inference without ground-truth degradation supervision.The paper contrasts this with iterative estimation methods such as KernelGAN, which can take over 60 seconds per image.
- 3.2.2. Degradation-Aware SR Network: DASR uses degradation-aware convolutional layers that predict depth-wise convolution kernels and modulation coefficients from the representation, avoiding direct feature concatenation interference.The network uses a residual-group structure based on RCAN and adapts its processing to specific degradations.
4. Experiments
Experiments evaluate DASR on synthetic and real degradations, testing representation learning, degradation-aware components, blind versus non-blind settings, and comparisons with prior SR methods. Results show that learned representations improve degradation discrimination and SR quality across varied degradations, while DASR reduces runtime relative to iterative correction.
- 4.1. Datasets and Implementation Details: Training uses DIV2K and Flickr2K images, evaluates on Set5, Set14, B100, and Urban100, and synthesizes degradations with Gaussian kernels and optional noise.For ×2/3/4 SR, Gaussian-kernel width ranges are [0.2,2.0], [0.2,3.0], and [0.2,4.0], respectively; the Gaussian kernel size is 21 × 21.
- 4.2. Experiments on Noise-Free Degradations with Isotropic Gaussian Kernels: Removing degradation representation learning prevents clear separation of kernel widths and lowers PSNR, especially for large kernels, whereas Model 4 learns discriminative representations and improves SR.The comparison uses T-SNE visualizations of representations from Models 1 and 4 and corresponding Set14 results in Table 1.
- 4.2. Experiments on Noise-Free Degradations with Isotropic Gaussian Kernels: DASR benefits from both dynamic convolutional kernels and channel-wise modulation coefficients to produce better results across various degradations.Model variants removing either component were adjusted to comparable model sizes before comparison.
- 4.2. Experiments on Noise-Free Degradations with Isotropic Gaussian Kernels: Degradation representations remain relatively stable when learned from different image contents, demonstrating robustness to content variations.The study generates multiple LR images with the same Gaussian kernel from different HR images and uses their representations to super-resolve one target image.
- 4.3. Experiments on General Degradations with Anisotropic Gaussian Kernels and Noises: DASR outperforms IKC in PSNR across varied blur kernels and noise levels while reducing running time by over 7 times, and produces clearer details on synthetic and real images.Compared methods include RCAN, SRMDNF, MZSR, and IKC; DASR also achieves visually clearer details and fewer blurring artifacts on real images.
5. Conclusion
The paper proposes unsupervised degradation representation learning for blind SR and introduces DASR to adapt to different degradations. Experiments report accurate degradation information and state-of-the-art blind-SR performance across various degradations.
- The proposed scheme learns discriminative degradation representations instead of explicitly estimating degradations.
- DASR adapts SR reconstruction to different degradations using the learned representations.
- The learned representations provide accurate degradation information.
- The network achieves state-of-the-art performance for blind SR with various degradations.