Source-linked AI summary
ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network
Nathanaël Carraz Rakotonirina, Andry Rasoanaivo
TL;DR
The paper addresses the remaining perceptual-quality limitations of ESRGAN for single-image super-resolution. It introduces a higher-capacity residual block and generator noise inputs, yielding models with more natural textures, sharpness, and detail, although noise does not always improve perceptual quality.
Problem
ESRGAN produces perceptually strong images, but the paper seeks further improvement in their perceptual quality.
Method
The paper replaces ESRGAN’s block with a higher-capacity RRDRB and adds noise inputs to the generator for stochastic variation.
Results
ESRGAN+ and nESRGAN+ outperform other approaches in perceptual quality and generate more natural textures, sharpness, and details.
Takeaways & Limitations
The new block and noise inputs improve perceptual detail, while stochastic noise is most beneficial for images that exploit variation.
Takeaways & Limitations
Adding noise does not always improve perceptual quality for image categories that do not fully exploit stochastic variation, including buildings in Urban100 and OST300.
Abstract
from arXiv · showhide
Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) is a perceptual-driven approach for single image super resolution that is able to produce photorealistic images. Despite the visual quality of these generated images, there is still room for improvement. In this fashion, the model is extended to further improve the perceptual quality of the images. We have designed a novel block to replace the one used by the original ESRGAN. Moreover, we introduce noise inputs to the generator network in order to exploit stochastic variation. The resulting images present more realistic textures. The code is available at https://github.com/ncarraz/ESRGANplus .
1. INTRODUCTION
The paper targets further improvement of ESRGAN’s perceptual image quality. It introduces a higher-capacity block and noise inputs to exploit stochastic variation.
- ESRGAN improves SRGAN with RRDB blocks without Batch Normalization, but the paper seeks further perceptual-quality gains.
- The proposed RRDRB block has higher capacity than ESRGAN’s RRDB block.
- Noise inputs are introduced into the network to benefit from stochastic variation.
2. RELATED WORK
SISR research spans interpolation-based, reconstruction-based, and learning-based methods, with deep learning advancing beyond earlier approaches. Perceptual-driven GAN methods prioritize visual realism even when standard scores are lower.
- SISR methods are categorized as interpolation-based, reconstruction-based, or learning-based.
- Deep learning approaches have surpassed earlier interpolation-based, reconstruction-based, and simple learning-based methods.
- SRCNN introduced end-to-end convolutional SISR, followed by deeper, recursive, residual, and multi-scale architectures.
- SRGAN uses perceptual and adversarial losses to produce visually convincing images despite lower PSNR and SSIM scores.
- ESRGAN enhances SRGAN with a higher-capacity RRDB block and removes Batch Normalization layers.
3. METHOD
The method preserves ESRGAN’s overall architecture while replacing its Dense block with a higher-capacity residual design and adding localized stochastic inputs. These changes target richer perceptual detail without increasing block complexity.
- 3.1. Network architecture: ESRGAN+ maintains ESRGAN’s overall architecture but replaces the Dense block with a new block.
- 3.1. Network architecture: RRDRB adds residual learning inside Dense blocks, with residuals added every two layers to increase capacity without increasing complexity.
- 3.1. Network architecture: The new residual design combines feature reuse and feature discovery, producing images with superior perceptual quality to those from a simple Dense block.
- 3.2. Noise inputs: Gaussian noise is added to residual dense block outputs with learned per-feature scaling factors to introduce stochastic detail.
- 3.2. Noise inputs: Noise effects remain localized, preserving global structure and higher-level information while supporting finer details.
4. EXPERIMENTS
The experiments train on DIV2K and evaluate ESRGAN+ models against perceptual-driven approaches using perceptual index and PSNR, with qualitative comparisons on Set14. The results report stronger perceptual quality, more detailed structures, and preserved textures, while noise injection does not consistently help every image category.
- 4.1. Data: DIV2K is used for training, with BSD100, Urban100, OST300, Set5, Set14, and PIRM as benchmark datasets.
- 4.3. Results: The evaluation reports perceptual index as the primary measure, with PSNR provided for reference.The perceptual index combines Ma’s score and NIQE; lower perceptual-index values are better, whereas higher PSNR is better.
- 4.3. Results: Both ESRGAN+ models outperform ESRGAN on the PIRM datasets, and nESRGAN+ achieves the better perceptual score.The authors attribute this result to the noise inputs in the generator network.
- 4.3. Results: Noise injection does not always improve perceptual quality for image categories that do not fully exploit stochastic variation, including buildings in Urban100 and OST300.The paper identifies this as a limitation of noise-injection generalization.
- 4.3. Results: Set14 comparisons show that the proposed models reconstruct more detailed structures, appear less distinguishable from ground truth, and preserve original textures.Figure 4 compares perceptual-driven models using PSNR and perceptual index.
5. CONCLUSION
ESRGAN+ and nESRGAN+ extend ESRGAN with a higher-capacity basic block and noise inputs for stochastic variation. The authors report better perceptual quality, with more natural textures, greater sharpness, and more details.
- ESRGAN+ and nESRGAN+ outperform other approaches in perceptual quality.
- The new basic block increases network capacity, while noise inputs provide stochastic variation.
- The improvements generate images with more natural textures, greater sharpness, and more details.