Residual Dense Network for Image Super-Resolution

1. Introduction

• Deep SR methods including VDSR [10], LapSRN [13], and EDSR [17] often neglect hierarchical features; MemNet [26] does not extract multilevel features from the original LR image, and dense blocks are hard to widen.MemNet interpolates the LR input to the desired size, increasing computation and losing some original detail.Source: §1 · pp. 1–2
• Direct access to the original LR input at each layer provides implicit deep supervision.Source: §1 · p. 2
• RDN is designed to use all hierarchical features from the original LR image for image SR under different degradation models.The network combines local dense features with global feature fusion.Source: §1 · p. 2
• RDB combines dense connections with contiguous memory from the preceding block, while local feature fusion adaptively preserves accumulated features and stabilizes wider networks.Each layer has access to preceding RDB states and earlier layers within the current block.Source: §1 · p. 2
• Global feature fusion adaptively combines hierarchical features from all RDBs in LR space, while global residual learning joins shallow and deep features.Source: §1 · p. 2

Related Work

• Earlier SR networks such as SRCNN, VDSR [10], IRCNN [38], DRCN [11], DRRN, and MemNet [26] interpolated LR inputs first, adding quadratic computation and blurring away detail.These methods consequently learned mappings from interpolated LR images rather than directly from original LR images to HR images.Source: §2 · p. 2
• LR-space methods including the transposed-convolution approach [4], ESPCN [22], SRResNet [14], and EDSR [17] upscale final LR features, but chained modules neglect information from earlier convolutional layers.LR-space feature extraction enabled real-time SR or very deep and wide networks, but the methods used only the last convolutional features for upscaling.Source: §2 · p. 2
• DenseNet [7] connects every layer within a dense block to preceding layers, and dense connections were also introduced among memory blocks [26] and dense blocks.The paper defers discussion of differences between DenseNet, SRDenseNet, MemNet, and RDN to Section 4.Source: §2 · p. 2
• Prior deep-learning SR methods lose useful hierarchical features from original LR images; RDN is proposed to efficiently extract and adaptively fuse features from all LR-space layers.The paper motivates the approach by the usefulness of very-deep-network hierarchical features for image restoration, including SR.Source: §2 · pp. 2–3

3.1. Network Structure

• RDN uses two convolutional layers for shallow feature extraction, a stack of RDBs, dense feature fusion, and an ESPCN-based upsampling network followed by a convolution.The shallow features feed the RDBs, and the upsampling network operates in HR space.Source: §3.1 · p. 3

3.2. Residual Dense Block

• Each RDB connects the preceding block’s features to every convolutional layer, concatenating them with earlier current-block outputs to extract local dense features.The concatenated input to the c-th layer contains G0+(c−1)×G feature maps.Source: §3.2 · pp. 3–4
• RDBs use 1 × 1 local feature fusion to adaptively combine features and reduce their number, while local residual learning improves information flow.The paper notes that very deep dense networks without local feature fusion are hard to train as growth rate increases.Source: §3.2 · p. 4

3.3. Dense Feature Fusion

• Dense feature fusion combines outputs from all RDBs with 1 × 1 and 3 × 3 convolutions, then uses global residual learning before upscaling.RDN extracts local and global features in LR space.Source: §3.3 · p. 4

3.4. Implementation Details

• Implementation uses 3 × 3 convolutions except in local and global feature fusion, which use 1 × 1 kernels; the final convolution outputs three color-image channels.Shallow extraction and fusion layers have G0=64 filters; other RDB layers have G filters and are followed by ReLU.Source: §3.4 · p. 4

Difference to DenseNet

• Compared with DenseNet [7], RDN combines local dense connections with residual feature fusion and global hierarchical fusion, while omitting batch normalization and pooling.Outputs from one RDB connect directly to every layer in the next RDB and contribute to the following RDB; pooling is omitted to preserve pixel-level information.Source: §4 · pp. 4–5

Difference to SRDenseNet

• RDN's RDB adds contiguous memory, wider growth enabled by local feature fusion, and local residual learning; RDN outperforms SRDenseNet.RDN uses global feature fusion and global residual learning rather than dense connections among RDBs, and uses L1 loss instead of SRDenseNet's L2 loss [17].Source: §4 · p. 5

Difference to MemNet

• Compared with MemNet [26], RDN extracts features from the original LR image rather than an upsampled HR image, reducing computation and improving performance.RDN also connects each RDB's output to every layer of the next RDB, passes convolutional-layer information forward within each RDB, and fuses hierarchical features globally in LR space.Source: §4 · p. 5

5.1. Settings

• Models are trained on DIV2K and evaluated with PSNR and SSIM on five standard datasets, using BI at ×2–×4, BD at ×3, and DN at ×3.Evaluation uses the Y channel of transformed YCbCr images; BD applies a 7×7 Gaussian blur with standard deviation 1.6 before downsampling, while DN adds Gaussian noise at level 30 after ×3 bicubic downsampling.Source: §5.1 · p. 5

5.2. Study of D, C, and G.

• Increasing the number of RDBs, convolutional layers per RDB, or growth rate improves performance; even smaller configurations still outperform SRCNN.The paper attributes the gains to deeper and wider networks extracting more hierarchical features.Source: §5.2 · pp. 5–6

5.3. Ablation Investigation

• With D = 20, C = 6, and G = 32, the baseline without CM, LRL, or GFF reaches only 34.87 dB PSNR; each component improves it, and using all three performs best.The components also stabilize training without obvious performance drop.Source: §5.3 · p. 6

5.4. Results with BI Degradation Model

• On BI benchmarks, RDN performs best on all datasets at ×2 and beats persistent CNN models at all scales; RDN+ improves further, while RDN’s advantage over MDSR narrows at ×3 and ×4.The comparison uses D = 16, C = 8, and G = 64; the paper attributes MDSR’s advantage at larger scales to its depth, multi-scale inputs, and larger training patches.Source: §5.4 · pp. 6–7, Table 2 · p. 7

5.5. Results with BD and DN Degradation Models

• For ×3 BD and DN, RDN and RDN+ perform best across all five datasets, suppressing blur under BD and handling noise while recovering details under DN.The DN results indicate RDN can jointly perform image denoising and super-resolution.Source: §5.5 · p. 7, Table 3 · p. 8

5.6. Super-Resolving Real-World Images

• On real-world images with unknown degradation, RDN recovers sharper edges and finer details than VDSR, LapSRN, and MemNet, indicating robustness to different or unknown degradations.The two images are “chip” and “hatc”.Source: Figure 9 · p. 8, §5.6 · p. 8

• The same RDN structure handles three degradation models and real-world data, and benchmark evaluations demonstrate superiority over state-of-the-art methods.Source: §6 · p. 8
• RDBs densely connect layers and preceding blocks to create contiguous memory; LFF adaptively controls feature preservation and stabilizes wider-network training, while LRL improves information and gradient flow.Source: §6 · p. 8
• GFF extracts hierarchical features in LR space; combining local and global features yields dense feature fusion and deep supervision.Source: §6 · p. 8