Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution

Jie Liang; Hui Zeng; Lei Zhang

Source-linked AI summary

Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution

Jie Liang, Hui Zeng, Lei Zhang

arXiv:2203.14216v1Mar 27, 2022cs.CVeess.IV

TL;DR

Real-world super-resolution must address complex, unknown degradations under limited computational resources, while existing approaches can be heavy and inflexible across degradation levels. DASR predicts each input’s degradation, mixes jointly trained experts into one adaptive network, and is reported to handle varied degradation levels with effective and efficient inference.

Problem
Real-world image super-resolution is difficult because degradation processes vary widely, while existing methods often use heavy backbones and lack flexibility across degradation levels.
Method
DASR predicts degradation parameters with a tiny regression network and non-linearly mixes jointly optimized same-topology experts into an input-specific network.
Results
DASR achieved superior performance across a wide range of degradation levels while retaining good inference efficiency for deployment.
Takeaways & Limitations
Degradation-aware parameter adaptation provides a practical Real-ISR pipeline that combines flexibility across degradation levels with efficient inference.
Takeaways & Limitations
DASR relies on a predefined degradation space whose parameter vector controls synthesized training samples and trains the degradation predictor.

Abstract

from arXiv · show

Efficient and effective real-world image super-resolution (Real-ISR) is a challenging task due to the unknown complex degradation of real-world images and the limited computation resources in practical applications. Recent research on Real-ISR has achieved significant progress by modeling the image degradation space; however, these methods largely rely on heavy backbone networks and they are inflexible to handle images of different degradation levels. In this paper, we propose an efficient and effective degradation-adaptive super-resolution (DASR) network, whose parameters are adaptively specified by estimating the degradation of each input image. Specifically, a tiny regression network is employed to predict the degradation parameters of the input image, while several convolutional experts with the same topology are jointly optimized to specify the network parameters via a non-linear mixture of experts. The joint optimization of multiple experts and the degradation-adaptive pipeline significantly extend the model capacity to handle degradations of various levels, while the inference remains efficient since only one adaptively specified network is used for super-resolving the input image. Our extensive experiments demonstrate that the proposed DASR is not only much more effective than existing methods on handling real-world images with different degradation levels but also efficient for easy deployment. Codes, models and datasets are available at https://github.com/csjliang/DASR.

1 Introduction

Real-world super-resolution must handle unknown, complex degradations while remaining computationally practical. DASR addresses this by adaptively specifying an efficient network for each input according to its degradation.

1 Introduction: Real-world images expose weaknesses in conventional and blind super-resolution because training degradations may mismatch reality and existing degradation spaces remain restricted.Real-world degradations can include diverse kernels, stronger noise, and compression.
1 Introduction: BSRGAN and Real-ESRGAN improve perceptual quality but struggle with mildly degraded inputs and rely on heavy backbones.The introduction identifies flexibility across degradation levels and practical computation as unresolved needs.
1 Introduction: DASR adaptively specifies its parameters from each input’s degradation using a regression network and a non-linear mixture of jointly optimized experts.The experts share a topology, while their adaptive weighting constructs one network for the input.
1 Introduction: The proposed pipeline uses only one adaptively specified network during inference, making expert mixing negligible and supporting practical deployment.The paper reports extensive experiments validating both effectiveness and efficiency.

2 Related Work

Related work expands degradation modeling and adaptive architectures for real-world super-resolution, but existing methods remain limited by domain gaps, unbalanced degradation sampling, or computational cost. DASR combines balanced degradation modeling with degradation-conditioned parameter mixing.

2.1 Real-World Image Super-Resolution: Real-world super-resolution methods model broader degradation spaces, yet heavy backbones and limited flexibility across degradation levels remain central challenges.These limitations are reported for methods including BSRGAN, Real-ESRGAN, and SwinIR.
2.2 Image Degradation Modeling: DASR partitions the degradation space into three balanced levels to improve optimization across degradation severities and better approximate real-world inputs.The paper contrasts this with unbalanced parameter sampling in BSRGAN and Real-ESRGAN.
2.3 Mixture of Experts and Dynamic Convolution: Unlike conventional mixture-of-experts inference, DASR mixes expert parameters to produce one adapted network rather than forwarding every expert and combining outputs.The paper associates this design with increased non-linearity and fast inference.
2.3 Mixture of Experts and Dynamic Convolution: DASR conditions convolutional parameter mixing on degradation parameters and computes weighting factors once across layers to preserve efficiency.This avoids the computation associated with repeatedly obtaining feature self-attention for linear parameter combinations.

3 Methodology

DASR combines degradation prediction with jointly optimized lightweight experts to construct an adaptive Real-ISR network. Its degradation modeling and parameter-fusion design target varied degradation levels while keeping inference close to that of a single expert.

Degradation-Adaptive Super-Resolution: DASR predicts each input’s degradation and uses it to adaptively specify a lightweight super-resolution network.A regression network estimates degradation parameters, which determine expert weighting and the resulting adaptive network.
Degradation-Adaptive Super-Resolution: Multiple same-topology convolutional experts are jointly optimized so different experts can handle subspaces of the degradation space.The experts share topology but have independent parameters and are trained under the same supervision.
Degradation-Adaptive Super-Resolution: A tiny network maps estimated degradation parameters to expert weights, which are fused at each convolution layer into an adaptive network through nonlinear parameter mixing.The shared weighting factors avoid independently computing feature-conditioned weights for every layer, while intermediate activations make the mixture nonlinear.
Degradation-Adaptive Super-Resolution: Inference deploys one adaptively specified network rather than all experts, keeping cost near that of a single lightweight expert.The prediction and weighting modules are lightweight, and parameter mixing is negligible compared with computing multiple feature maps.
Degradation Modeling: The degradation space models blur, resizing, noise, and JPEG compression, representing degradation types and levels with interpretable parameters.Blur is quantified using kernel size, two principal-axis standard deviations, and rotation rather than kernel coefficients.
Training Losses: Training jointly optimizes experts, degradation prediction, and weighting using pixel, regression, perceptual, and adversarial losses.The total loss combines these terms with balancing parameters λ1, λ2, and λ3.

4 Experiments

DASR consistently handles diverse degradation levels while improving reconstruction quality and reducing deployment costs relative to competing backbones. Ablations and qualitative results attribute this behavior to degradation-adaptive parameter mixing, jointly optimized experts, and a single efficient forward pass.

Effectiveness: DASR improves across the first three degradation types and remains among the best two on the most severe dataset.It outperforms Real-ESRGAN by about 1.7dB PSNR and 26% LPIPS on Level-I, while matching BSRGAN nearly on Level-III.
Effectiveness: With the same topology and similar overhead, DASR outperforms SRResNet on every dataset, including +0.5dB PSNR on Bicubic and about 5% LPIPS on Level-II.The result indicates that degradation-adaptive mixing extends capacity without abandoning the lightweight backbone.
Efficiency: DASR uses about 1/3 and 1/12 the latency of RRDB and SwinIR while outperforming these heavier models in reconstruction fidelity.The comparison supports efficient deployment without sacrificing fidelity across the evaluated datasets.
Efficiency: Only one adaptive network is forwarded, so increasing the number of experts adds little computational cost while extending model capacity.Parameter fusion requires only N × 1.52M parallel multiplications and additions, unlike classical MoE inference that forwards all experts.
Qualitative Comparison: Qualitatively, DASR restores sharper realistic details and suppresses artifacts across varied degradations where competing models become blurry, noisy, or overly textured.The comparisons include bicubic, mild, severe, and real-world inputs, with competing models showing degradation-specific weaknesses.
Ablation Study: Five experts are sufficient, classical MoE is heavier and less detailed, and the design also generalizes to an EDSR-M backbone.The ablations report similar quality for N = 5 and N = 9, while classical MoE costs N times more and produces fewer fine details.

5 Conclusion

DASR addresses Real-ISR by jointly learning multiple experts and adaptively mixing them according to degradation, while remaining efficient during inference. Experiments support strong performance across degradation levels and practical deployment.

5 Conclusion: DASR achieves superior performance across images with a wide range of degradation levels while maintaining good inference efficiency for easy deployment.The method adaptively mixes multiple super-resolution experts into one degradation-aware expert during inference.
5 Conclusion: DASR jointly learns multiple super-resolution experts and adaptively mixes them into one expert according to the input degradation.This design improves modeling capacity and flexibility for different degradation levels.
5 Conclusion: DASR also allows user control over customized super-resolution results.

6 Appendix

The appendix details DASR’s degradation-space construction, training objective, and qualitative evaluation across degradation levels and image types. Balanced sampling and adaptive expert mixing support realistic reconstructions across varied degradations.

6 Appendix: DASR is trained with pixel, regression, perceptual, and adversarial losses combined in a single total objective.The perceptual loss compares VGG19 feature maps, while the adversarial loss uses a spectrally normalized U-Net discriminator.
6 Appendix: Balanced sampling across degradation levels improves DASR’s generalization to real-world images with different degradations.The constructed datasets partition the degradation space into S1, S2, and S3 and sample them with probabilities [0.3, 0.3, 0.4].
6 Appendix: The appendix provides qualitative comparisons on real-world, bicubic, and Level-I to Level-III degradation datasets using models trained on DIV2K, Flickr2K, and OutdoorScene-Training.The comparisons evaluate generalization across degradation conditions and image contents.
6 Appendix: The degradation subspaces specify operations including blur, resizing, noise, sinc filtering, and JPEG compression through parameterized settings.Table 3 documents the settings for S1, S2, and S3, including operation order and normalized parameter representations.