Source-linked AI summary
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results
Andreas Lugmayr, Martin Danelljan, Radu Timofte, Namhyuk Ahn, Dongwoon Bai, Jie Cai, Yun Cao, Junyang Chen, Kaihua Cheng, SeYoung Chun, Wei Deng, Mostafa El-Khamy, Chiu Man Ho, Xiaozhong Ji, Amin Kheradmand, Gwantae Kim, Hanseok Ko, Kanghyu Lee, Jungwon Lee, Hao Li, Ziluan Liu, Zhi-Song Liu, Shuai Liu, Yunhua Lu, Zibo Meng, Pablo Navarrete Michelini, Christian Micheloni, Kalpesh Prajapati, Haoyu Ren, Yong Hyeok Seo, Wan-Chi Siu, Kyung-Ah Sohn, Ying Tai, Rao Muhammad Umer, Shuangquan Wang, Huibing Wang, Timothy Haoning Wu, Haoning Wu, Biao Yang, Fuzhi Yang, Jaejun Yoo, Tongtong Zhao, Yuanbo Zhou, Haijie Zhuo, Ziyao Zong, Xueyi Zou
TL;DR
The paper addresses real-world super-resolution when paired low- and high-resolution images are unavailable. It reviews a challenge that uses unpaired source and clean target images across synthetic-artifact and smartphone tracks, with perceptual quality central to evaluation. The challenge results show strong progress toward supervised performance, with 22 teams competing in the final phase.
Problem
Real-world super-resolution lacks paired training images, while bicubic-generated pairs alter image characteristics and often fail to generalize to natural images.
Method
The challenge provides unpaired source images and clean high-quality target images across two tracks, using degradation modeling for synthetic artifacts and human studies for perceptual evaluation.
Results
The winning Impressionism method achieved a 9.5% better MOS than the second entry in Track 1 and outperformed other approaches by a large margin in Track 2’s human study.
Takeaways & Limitations
All top-10 Track 1 approaches surpassed the Bicubic MOS baseline, and the winning method even beat the fully supervised ESRGAN baseline.
Takeaways & Limitations
LPIPS was treated as an indication of perceptual quality but not used to determine final rankings.
Abstract
from arXiv · showhide
This paper reviews the NTIRE 2020 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided along with a set of unpaired high-quality target images. In Track 1: Image Processing artifacts, the aim is to super-resolve images with synthetically generated image processing artifacts. This allows for quantitative benchmarking of the approaches \wrt a ground-truth image. In Track 2: Smartphone Images, real low-quality smart phone images have to be super-resolved. In both tracks, the ultimate goal is to achieve the best perceptual quality, evaluated using a human study. This is the second challenge on the subject, following AIM 2019, targeting to advance the state-of-the-art in super-resolution. To measure the performance we use the benchmark protocol from AIM 2019. In total 22 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.
1. Introduction
Real-world super-resolution must handle image degradations that differ from conventional bicubic downsampling, using unpaired source and clean target images. The challenge evaluates methods across synthetic processing artifacts and low-quality smartphone images, emphasizing perceptual quality.
- Single-image super-resolution increases image resolution by filling in additional high-frequency content.
- Paired low- and high-resolution training images are usually unavailable in real-world applications.
- Bicubic downsampling changes low-level image characteristics, including by severely reducing sensor noise, limiting generalization to natural images.
- The challenge instead targets super-resolution from unpaired data without restrictive assumptions about input image formation.
- Track 1 uses synthetically generated image-processing artifacts for reference-based evaluation, whereas Track 2 uses untouched low-quality smartphone images without ground truth.
- Both tracks seek perceptually pleasing outputs, with final ranking performed through a human study.
2. NTIRE 2020 Challenge
The NTIRE 2020 challenge benchmarks weak and unsupervised real-world super-resolution using two tracks with unpaired source and target data. It combines synthetic-degradation evaluation for Track 1 with human perception evaluation for smartphone images in Track 2.
- The challenge promotes weak and unsupervised super-resolution, establishes a benchmark protocol and dataset, and probes the field’s state of the art.
- Both tracks upscale images by a factor of 4× and were organized using the Codalab platform.
- Track 1: Image Processing Artifacts: Track 1 uses an undisclosed degradation operator that creates structured artifacts associated with very low-end image-processing pipelines.
- Track 2: Smartphone Images: Track 2 super-resolves low-quality iPhone3 images toward a clean target domain, using human perception studies because ground truth above sensor size is unavailable.
- The final test phase required participants to submit super-resolved test images together with method descriptions, code, and model weights.
3. Challenge Results
The challenge compared real-world super-resolution methods across two tracks using human perceptual rankings alongside reference-based and no-reference image-quality metrics. Top methods generally learned degradation operators before training super-resolution networks, while careful degradation modelling emerged as more important than the choice of SR architecture.
- Challenge scope: 19 valid methods from 16 teams entered Track 1, while Track 2 received 15 valid methods from 14 teams.The final test results are reported in Tables 1 and 2.
- Architectures and Main Ideas: Most top methods used a two-step pipeline that first learned a clean-to-source degradation operator and then trained the SR network on generated pairs.The learned operator models source-domain corruptions to create paired training data for super-resolution.
- Architectures and Main Ideas: Impressionism explicitly estimated blur kernels and noise statistics, while AITA-Noah alternated degradation and SR learning and Samsung-SLSI-MSL sampled varied degradations for robustness.For Track 2, Impressionism used KernelGAN for kernel estimation, which was associated with sharper and higher-quality outputs.
- Architectures and Main Ideas: Training strategy and degradation modelling appeared more important than SR architecture, with leading methods commonly adopting RRDB/ESRGAN or RCAN and perceptual or adversarial losses.The challenge results therefore emphasized how source-domain modelling shaped performance across otherwise popular architectures.
- Track 1: Image Processing Artifacts: 9.5% better MOS than the second entry was achieved by Impressionism in Track 1, while ITS425 achieved the second-best PSNR and best SSIM.BOE-IOT-AIBD also achieved very strong PSNR and SSIM results, but AITA-Noah was not ranked because it used test-source images for training.
- Track 1: Image Processing Artifacts: All top-10 Track 1 approaches exceeded the Bicubic MOS baseline, and Impressionism even surpassed the fully supervised ESRGAN baseline, narrowing the gap to supervised SR.The paper notes that other modifications and hyper-parameter settings may partly explain this comparison.
- Track 2: Smartphone Images: In Track 2, Impressionism outperformed other approaches by a large margin in MOR and produced sharper images with almost no noise and few artifacts.AITA-Noah and ITS425 also produced clean images, but lacked Impressionism’s sharpness and detail.
- Track 2: Smartphone Images: Impressionism achieved the best average IQA-Rank, while a similarly ranked Relbmag-Eht result contained a severe structured-noise pattern.This contrast suggests that standard IQA metrics may be poorly suited as evaluation criteria for this setting.
4. Challenge Methods and Teams
Participating teams primarily learned realistic degradation processes to synthesize paired data before training super-resolution networks. Methods varied from kernel and noise estimation to iterative adaptation, image translation, diverse degradation sampling, weak supervision, and unsupervised pseudo-pair generation.
- Impressionism: Impressionism estimated blur kernels from real blurry images and injected collected noise patches to construct realistic low-resolution training data.The resulting pairs combined degraded low-resolution images with clean high-resolution targets for ESRGAN-based training.
- AITA-Noah: AITA-Noah used iterative domain adaptation for Track 1 and CycleSR for Track 2 to learn degradation operators and generate paired training data.Its Track 2 pipeline translated between real and synthetic low-resolution domains before supervised super-resolution.
- Architectures and objectives: Several methods used specialized architectures and training objectives, including ESRGAN variants, LR-conditional frequency separation, AutoML loss weighting, and physical image formation-inspired encoder-decoder designs.The reported objectives included pixel, perceptual, adversarial, L1, total-variation, and VGG losses.
- Unsupervised learning: SimUSR relaxed zero-shot super-resolution by collecting multiple low-resolution images and generating pseudo-pairs, enabling supervised techniques without runtime training.The challenge implementation used pretrained RCAN with an attached BM3D denoiser.
- Other approaches: Other teams explored integrated degradation learning, generic models trained on sampled degradations, weakly supervised mobile pairs, and two-stage CycleGAN-to-RCAN training.These approaches included LR- and HR-phase discriminators, diverse blur and downscaling choices, DPED-derived mobile pairs, and perceptual or adversarial losses.
5. Conclusions
The challenge established a real-world super-resolution benchmark with unpaired source and target images, two tracks, and 22 competing teams.
- 22 teams competed in the final phase, contributing innovative solutions to real-world super-resolution.
- Training used unpaired source images with unknown degradations and clean, high-quality target images.
- The two tracks targeted image-processing artifacts and low-quality smartphone images.
- The challenge aimed to stimulate unsupervised super-resolution research through a standard benchmark and new baseline methods.
KU-ISPL
The KU-ISPL entry is represented by six researchers from Korea University’s Intelligent Signal Processing Laboratory.
- Kanghyu Lee, Jaihyun Park, Junyeop Lee, Jeongki Min, Bokyeung Lee, and Hanseok Ko are listed as team members.
- All listed members are affiliated with Korea University’s Intelligent Signal Processing Laboratory.
- The passage identifies the KU-ISPL group by researcher names and institutional affiliation, without describing its method or results.
TeamAY
The listed TeamAY-related entries cover several real-world super-resolution approaches, including DSGAN, kernel estimation, residual dense blocks, domain adaptation, and GANs.
- DSGAN and Triple-V MGBPv2 for Real Super-Resolution is listed as a BOE Technology Group entry led by Pablo Navarrete Michelini.
- Real World Super-Resolution via Kernel Estimation and Noise Injection is listed as a Tencent Youtu Lab entry led by Xiaozhong Ji.
- An Adaptive Residual Dense Block Network for Real Image Super-Resolution appears among the listed entries.
- Huawei’s listed approach combines iterative domain adaptation, CycleSR, and conditional frequency separation GAN.
- Samsung’s listed entry addresses real-world super-resolution using generative adversarial networks.