Source-linked AI summary
Component Divide-and-Conquer for Real-World Image Super-Resolution
Pengxu Wei, Ziwei Xie, Hannan Lu, Zongyuan Zhan, Qixiang Ye, Wangmeng Zuo, Liang Lin
TL;DR
Real-world SR is challenged by degradation that differs from simplified simulations and by unequal reconstruction difficulty across image components. The paper introduces the DRealSR benchmark, the CDC component-aware model, and GW loss; experiments report improvements for CDC and evidence that simulated bicubic degradation generalizes poorly to real images.
Problem
Real-world SR lacks adequate modeling of diverse degradation processes, while homogeneous pixel-wise losses do not balance the differing difficulties of image regions.
Method
The paper builds DRealSR and trains CDC with three component-attentive branches, intermediate supervision, and Gradient-Weighted loss for flat, edge, and corner regions.
Results
CDC improves PSNR by 0.29dB over HGSR, while CAB and GW loss provide 0.12 and 0.15 dB PSNR improvements, respectively.
Takeaways & Limitations
DRealSR provides a real-world SR benchmark, and CDC offers a component-aware approach for handling reconstruction difficulties in diverse degradation settings.
Abstract
from arXiv · showhide
In this paper, we present a large-scale Diverse Real-world image Super-Resolution dataset, i.e., DRealSR, as well as a divide-and-conquer Super-Resolution (SR) network, exploring the utility of guiding SR model with low-level image components. DRealSR establishes a new SR benchmark with diverse real-world degradation processes, mitigating the limitations of conventional simulated image degradation. In general, the targets of SR vary with image regions with different low-level image components, e.g., smoothness preserving for flat regions, sharpening for edges, and detail enhancing for textures. Learning an SR model with conventional pixel-wise loss usually is easily dominated by flat regions and edges, and fails to infer realistic details of complex textures. We propose a Component Divide-and-Conquer (CDC) model and a Gradient-Weighted (GW) loss for SR. Our CDC parses an image with three components, employs three Component-Attentive Blocks (CABs) to learn attentive masks and intermediate SR predictions with an intermediate supervision learning strategy, and trains an SR model following a divide-and-conquer learning principle. Our GW loss also provides a feasible way to balance the difficulties of image components for SR. Extensive experiments validate the superior performance of our CDC and the challenging aspects of our DRealSR dataset related to diverse real-world scenarios. Our dataset and codes are publicly available at https://github.com/xiezw5/Component-Divide-and-Conquer-for-Real-World-Image-Super-Resolution
1 Introduction
Real-world SR remains limited by mismatched simulated degradation and unequal reconstruction difficulty across image components. The paper introduces DRealSR, CDC, and GW loss to address these challenges through component-aware learning.
- Simulated degradation often deviates from real degradation, limiting the applicability of trained SR models to real-world images.
- SR targets differ across flat regions, edges, and textures, while homogeneous pixel-wise losses emphasize easier regions.
- CDC divides images into flat, edge, and corner components and uses three Component-Attentive Blocks with intermediate supervision.
- DRealSR is a large-scale real-world SR benchmark collected from five DSLR cameras to mitigate limitations of simulated degradation.
- GW loss uses image structural information to balance reconstruction difficulties across image regions.
2 Related Work
Prior SR datasets commonly rely on simulated bicubic degradation, whereas real-world datasets capture more complex imaging processes. Existing real-world benchmarks remain limited in scale, device diversity, or degradation coverage, motivating DRealSR and CDC.
- Conventional SR datasets typically form LR images through simulated HR downsampling, especially bicubic degradation.
- Real-world degradation can be complex and nonlinear, limiting conclusions drawn from datasets built solely with simulated degradation.
- City100 targets realistic imaging but contains 100 aligned postcard pairs captured indoors, while SR-RAW and RealSR address other real-world settings.
- RealSR uses two DSLR cameras, but existing methods remain limited in modeling degradation diversity across devices.
- CDC parses images into flat, edge, and corner components and uses GW loss to balance pixel-wise reconstruction difficulties.
3 DRealSR: A Large-scale Real-world SR Dataset
DRealSR is designed to expose real-world SR challenges arising from complex degradation and camera-dependent imaging differences. Its component analysis motivates separate treatment of flat regions, edges, and corners in CDC.
- DRealSR collects real LR-HR pairs by zooming DSLR cameras across four scaling factors, ×1∼×4.
- The dataset uses five DSLR cameras and natural indoor and outdoor scenes while avoiding moving objects.
- Realistic LR acquisition combines anisotropic blurring, downsampling, signal-dependent noise, and in-camera processing.
- CDC separately incorporates flat regions, edges, and corners through three CAB branches that blend intermediate SR outputs.
- Camera lens and sensor differences produce distinct imaging patterns, causing diverse degradation processes and potentially poor cross-dataset generalization.
- Flat regions are less affected by degradation diversity, whereas edges and corners change more across degradation settings.
4 Real-word Image Super-Resolution
The paper frames real-world super-resolution as a divide-and-conquer problem over flat, edge, and corner components, using CDC and GW loss to address differing reconstruction difficulties. CDC predicts component-specific intermediate results, aggregates them with attentive masks, and applies intermediate supervision during training.
- Motivation: Harris-inspired parsing divides image content into flat, edge, and corner components according to gradient changes and reconstruction difficulty.Flat regions are less affected by degradation diversity, whereas edges and corners vary more across degradation settings.
- Component Divide-and-Conquer: The model organizes learning from easier flat regions to harder edges and corners, then separately produces intermediate component SR results for merging.Three HG modules learn component-attentive masks, while the three intermediate outputs are combined into the final prediction.
- Component Divide-and-Conquer: CDC uses three CABs associated with flat, edge, and corner components, with HR-derived component guidance used during training rather than test-time detection.Each CAB learns an attentive mask and an intermediate SR prediction; HR images provide the component parsing guidance.
- Gradient-Weighted Loss: CDC merges component-specific SR outputs by weighting them with learned attentive masks, while GW loss dynamically adjusts reconstruction emphasis according to regional gradient differences.Flat and single-edge regions are treated as simple, whereas corners are treated as difficult because they contain fine details; α is set to 4 in experiments.
- Hourglass Super-Resolution Network: HGSR provides a stacked hourglass backbone with multi-scale inference, Residual Inception Blocks, and non-recursive intermediate supervision.Unlike the basic HG design, HGSR does not recursively feed intermediate predictions into the next hourglass module.
- Component Divide-and-Conquer: The learned masks exhibit distinct characteristics for flat regions, edges, and corners, and the corresponding intermediate SR results follow those component characteristics.The three CAB outputs are designed to preserve, sharpen, or enhance information according to the represented component.
5 Experiments
Experiments evaluate CDC’s architecture, component guidance, GW loss, and performance on RealSR and DRealSR. Ablations and cross-dataset tests support the model’s effectiveness and the challenge of real-world degradation.
- Model configuration: HGSR performance improves as hourglass blocks increase from 2 to 4, stabilizing beyond 6; the experiments therefore use 6 blocks.The number of HG blocks is selected based on the observed performance trend.
- Model ablation: Intermediate supervision improves HGSR by 0.18dB PSNR, while replacing connecting convolutions with RIBs adds a further 0.02dB.These changes are evaluated as components of the base model.
- Model ablation: CAB and GW loss improve PSNR by 0.12 and 0.15dB, respectively, while CDC improves 0.29dB over HGSR.The final CDC version combines the evaluated components.
- Component guidance: Removing corner branches reduces PSNR by 0.19dB, indicating that corner regions contribute substantially despite occupying a small image area.The study compares guidance from flat, edge, and corner regions.
- Loss evaluation: GW loss improves PSNR over L1 by 0.30dB for EDSR, 0.25dB for SRResNet, 0.10dB for HGSR, and 0.15dB for CDC.The reported gains indicate applicability across multiple SR models.
- Comparison with state-of-the-art methods: CDC outperforms state-of-the-art methods on RealSR and DRealSR, achieving the best results at all scales and about 0.4dB higher performance at ×4 on DRealSR.LPIPS and SSIM are not reported for LP-KPN and DDet because those models are trained on the Y channel.
- Visual comparison: CDC reconstructs sharp and natural details, whereas competing methods can produce blurry details, aliasing, or artifacts.The visual comparison includes EDSR, RCAN, and LP-KPN among the competing methods.
- Cross-dataset evaluation: Models trained on DRealSR retain superior performance when tested on RealSR, whereas models trained on RealSR drop greatly on DRealSR, especially at ×4.This cross-testing result indicates that DRealSR is more challenging than RealSR.
6 Conclusion
The paper introduces DRealSR and CDC to address realistic image degradation and differing reconstruction difficulty across image components. Experiments support the dataset’s challenge and the value of the divide-and-conquer model and GW loss.
- Conclusion: DRealSR is a large-scale real-world SR dataset intended to support research on realistic image degradation.The dataset targets complex and diverse degradation in real-world scenarios.
- Conclusion: CDC uses a stacked HGSR network, three CABs, and intermediate supervision to learn component-aware SR predictions.The model separates flat, edge, and corner components during learning.
- Conclusion: GW loss adapts learning objectives to the reconstruction difficulties of different image components.The conclusion presents this as part of the divide-and-conquer solution.
- Conclusion: Experiments validate DRealSR’s challenging real-world scenarios and support CDC and GW loss for real-world SR and other low-level vision tasks.The stated scope includes the real-world SR task and possible extension to other low-level vision tasks.