PaperlayerView exact revision on arXiv

Source-linked AI summary

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, Yun Fu

arXiv:1802.08797v2cs.CV

TL;DR

Image super-resolution models often underuse hierarchical features from the original low-resolution image. The paper proposes a residual dense network with local and global feature fusion, and reports favorable performance against state-of-the-art methods across benchmark degradation models.

  • Problem

    Most deep learning super-resolution methods neglect hierarchical features from the original low-resolution image, despite their relevance to reconstructing objects with varied scales, viewpoints, and aspect ratios.

  • Method

    RDN uses residual dense blocks with dense connections, contiguous memory, local feature fusion, and global feature fusion to exploit hierarchical features from the original low-resolution image.

  • Results

    RDN is evaluated against six state-of-the-art super-resolution methods under the bicubic degradation model.

  • Takeaways & Limitations

    The proposed network provides a unified framework for using local and global hierarchical features in image super-resolution.

Abstract

from arXiv · show

A very deep convolutional neural network (CNN) has recently achieved great success for image super-resolution (SR) and offered hierarchical features as well. However, most deep CNN based SR models do not make full use of the hierarchical features from the original low-resolution (LR) images, thereby achieving relatively-low performance. In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers. Specifically, we propose residual dense block (RDB) to extract abundant local features via dense connected convolutional layers. RDB further allows direct connections from the state of preceding RDB to all the layers of current RDB, leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is then used to adaptively learn more effective features from preceding and current local features and stabilizes the training of wider network. After fully obtaining dense local features, we use global feature fusion to jointly and adaptively learn global hierarchical features in a holistic way. Extensive experiments on benchmark datasets with different degradation models show that our RDN achieves favorable performance against state-of-the-art methods.

1. Introduction

The paper proposes RDN for image super-resolution to exploit hierarchical features from the original LR image, addressing limitations of existing deep methods. Its RDBs and global feature fusion combine local and global features through dense, residual, and adaptive connections.

  • Motivation: Hierarchical features from very deep networks provide reconstruction clues, but many deep SR methods neglect them.Objects vary in scale, viewing angle, and aspect ratio, making hierarchical information useful for reconstruction.
  • Motivation: Existing approaches face limitations including quadratic computation and detail loss from interpolating the original LR image, while dense SR blocks use relatively low growth rates.MemNet interpolates the LR image to the desired size, increasing computation complexity quadratically and losing original details.
  • Proposed framework: RDN is a unified framework that fully exploits hierarchical features from the original LR image for high-quality SR under different degradation models.The network avoids directly extracting every convolutional-layer output in LR space by using residual dense blocks as building modules.
  • Residual dense block: RDBs use local dense connections, contiguous memory from preceding RDBs, local feature fusion, and local residual learning to adaptively preserve accumulated features.These mechanisms connect preceding RDB states to current layers while exploiting all layers within each block.
  • Global feature fusion: Global feature fusion adaptively combines hierarchical features from all RDBs in LR space, while global residual learning joins shallow and deep features.This produces global dense features from the original LR image.

2. Related Work

Deep learning methods advanced image super-resolution through end-to-end mappings, deeper or shared-weight networks, and efficient upsampling with residual learning. However, existing methods lose useful hierarchical features from the original LR image, motivating RDN’s adaptive feature extraction and fusion.

  • Deep learning-based image SR: SRCNN established an end-to-end mapping between interpolated LR images and HR counterparts, while later methods mainly increased network depth or shared network weights.VDSR and IRCNN are cited as examples of deeper networks.
  • Upsampling methods: Transposed convolution and efficient sub-pixel convolution enabled upsampling from original LR inputs or final LR feature maps to HR outputs.The sub-pixel convolution layer was adopted in SRResNet and EDSR, which used residual learning.
  • Dense connections: DenseNet introduced direct connections between any two layers within a dense block, allowing each layer to read information from all preceding layers.Dense connections were also introduced among memory blocks and dense blocks.
  • Motivation: Existing deep learning SR methods improve over conventional methods but lose useful hierarchical features from the original LR image.Hierarchical features from very deep networks are useful for image restoration tasks such as image SR.
  • Motivation: RDN is proposed to efficiently extract and adaptively fuse features from all layers in the LR space.The paper introduces RDN to address the loss of hierarchical features from the original LR image.

3. Residual Dense Network for Image SR

RDN extracts and progressively fuses hierarchical features through shallow feature extraction, residual dense blocks, dense feature fusion, and upsampling. Its RDBs combine dense connectivity, local feature fusion, and local residual learning to improve feature reuse, information flow, and trainability.

  • Network architecture: RDN comprises shallow feature extraction, residual dense blocks, dense feature fusion, and an upsampling network.Two convolutional layers extract shallow features before RDB processing; DFF combines global feature fusion and global residual learning, followed by ESPCN-based upsampling and a convolutional layer.
  • Residual dense block: RDBs extract local dense features using direct connections from the preceding block and every preceding layer to subsequent layers.Concatenation produces G0+(c −1) ×G feature-maps at convolutional layer c, preserving feed-forward connections while extracting local dense features.
  • Residual dense block: Local feature fusion adaptively combines preceding-block and within-block features using a 1 × 1 convolution to control feature dimensionality.The paper notes that without LFF, very deep dense networks with larger growth rate G would be difficult to train.
  • Residual dense block: Local residual learning further improves information flow and the RDB’s representation ability.Together with dense connectivity, local residual learning defines the residual dense block architecture.

4. Discussions

RDN differs from DenseNet, SRDenseNet, and MemNet through residual dense blocks, contiguous memory, local and global feature fusion, and hierarchical feature extraction directly from LR images. These design choices distinguish RDN’s feature connectivity and processing space from prior models.

  • Difference to DenseNet: RDN combines dense connected layers with local feature fusion and local residual learning, while removing batch normalization layers.The passage contrasts these choices with DenseNet and states that their effectiveness is demonstrated in Section 5.
  • Difference to DenseNet: RDN connects each preceding RDB directly to every layer of the current RDB and uses global feature fusion to exploit hierarchical features.The preceding RDB also contributes to the input of the next RDB, extending feature reuse across blocks.
  • Difference to SRDenseNet: Compared with SRDenseNet’s basic dense block, RDN’s residual dense block introduces contiguous memory and supports larger growth rates through local feature fusion.The passage identifies these as improvements to the basic building block.
  • Difference to MemNet: Unlike MemNet, RDN extracts hierarchical features from the original LR image instead of extracting and reconstructing features in HR space after Bicubic upsampling.The passage states that this reduces computational complexity significantly and improves performance.

5. Experimental Results

RDN is evaluated across standard SR datasets and multiple degradation models, with experiments showing that deeper and wider networks, the CM/LRL/GFF components, and hierarchical feature extraction improve performance and robustness. It achieves strong quantitative and visual results against state-of-the-art methods under bicubic, blur, noise, and unknown degradations.

  • Datasets, Metrics, and Degradation Models: Experiments use DIV2K training data, five benchmark datasets, PSNR and SSIM metrics, and BI, BD, and DN degradation models.DIV2K contains 800 training, 100 validation, and 100 test images; testing uses Set5, Set14, B100, Urban100, and Manga109.
  • Network Parameter Investigation: Larger numbers of RDBs, convolutional layers, and growth rate improve performance, while smaller settings reduce performance but still outperform SRCNN.The results support deeper and wider RDNs for extracting more hierarchical features.
  • Ablation Study: The CM, LRL, and GFF components each improve the baseline, combinations perform better than individual components, and using all three performs best.The components improve information and gradient flow and stabilize training without obvious performance drop.
  • Benchmark Results with BI Degradation: Under BI degradation, RDN performs best against persistent CNN models across all datasets and scaling factors, achieves the best average results on most comparisons, and RDN+ improves further.The comparison covers ×2, ×3, and ×4 SR against SRCNN, LapSRN, DRRN, SRDenseNet, MemNet, and MDSR without deeper networks, multi-scale information, or larger input patches.
  • Results under BD, DN, and Unknown Degradations: RDN and RDN+ perform best under BD and DN degradation, suppress blur, handle noise, recover sharper edges and details, and remain robust to unknown degradation models.Visual comparisons also show sharper and clearer reconstructions, while unknown-degradation experiments compare RDN with VDSR, LapSRN, and MemNet.

6. Conclusions

The paper proposes a very deep residual dense network for image super-resolution, built from residual dense blocks that exploit local-layer and inter-block connections. Local feature fusion stabilizes training in wider networks and adaptively controls information preservation from current and preceding blocks.

  • 6. Conclusions: RDN uses the residual dense block as its basic build module for image super-resolution.The network is described as a very deep residual dense network.
  • 6. Conclusions: Dense connections between layers within each RDB enable full usage of local-layer features.
  • 6. Conclusions: Local feature fusion stabilizes training of wider networks and adaptively controls information preservation from current and preceding RDBs.
  • 6. Conclusions: RDBs also provide direct connections from the preceding RDB to each layer of the current RDB.The supplied passage ends while describing this inter-block connection mechanism.
Loading 1802.08797v2…