Densely Residual Laplacian Super-Resolution

Saeed Anwar; Nick Barnes

Source-linked AI summary

Densely Residual Laplacian Super-Resolution

Saeed Anwar, Nick Barnes

arXiv:1906.12021v2Jun 28, 2019eess.IVcs.CV

TL;DR

Super-resolution networks often require substantial depth and do not adequately exploit multi-scale features. DRLN addresses these issues with dense residual and cascading connections plus Laplacian attention, and evaluations report favorable efficiency and performance, while fine-detail hallucination remains difficult at 8× upscaling.

Problem
Existing super-resolution CNNs can require very deep architectures, while many treat features equally or at limited scales.
Method
DRLN combines densely connected residual blocks, cascading residual-on-residual connections, and Laplacian attention for multi-level feature modeling.
Results
DRLN achieves efficient, favorable performance across evaluations of low-resolution, noisy, and real-world images.
Takeaways & Limitations
The modular design provides a compact super-resolution network with reported accuracy and efficiency across multiple image conditions.
Takeaways & Limitations
At 8× upscaling, DRLN struggles to hallucinate finer details, a limitation shared by the compared super-resolution methods.

Abstract

from arXiv · show

Super-Resolution convolutional neural networks have recently demonstrated high-quality restoration for single images. However, existing algorithms often require very deep architectures and long training times. Furthermore, current convolutional neural networks for super-resolution are unable to exploit features at multiple scales and weigh them equally, limiting their learning capability. In this exposition, we present a compact and accurate super-resolution algorithm namely, Densely Residual Laplacian Network (DRLN). The proposed network employs cascading residual on the residual structure to allow the flow of low-frequency information to focus on learning high and mid-level features. In addition, deep supervision is achieved via the densely concatenated residual blocks settings, which also helps in learning from high-level complex features. Moreover, we propose Laplacian attention to model the crucial features to learn the inter and intra-level dependencies between the feature maps. Furthermore, comprehensive quantitative and qualitative evaluations on low-resolution, noisy low-resolution, and real historical image benchmark datasets illustrate that our DRLN algorithm performs favorably against the state-of-the-art methods visually and accurately.

1 INTRODUCTION

The paper targets efficient single-image super-resolution by addressing excessive network depth and limited multi-scale feature modeling. DRLN combines densely connected residual blocks, cascading residual-on-residual connections, and Laplacian attention to improve accuracy and efficiency.

Motivation: Single-image super-resolution reconstructs a high-resolution image from a low-resolution counterpart, making it useful but ill-posed.The low-resolution-to-high-resolution mapping is many-to-one.
Motivation: Very deep CNNs can improve quality but increase computational load, motivating efficient architectures that reuse computed features.RCAN has more than 400 convolutional layers, whereas SRCNN has three.
Motivation: Existing methods often treat features equally or at one scale, limiting adaptability to low-, mid-, and high-frequency content.The paper focuses on restoring mid- and high-level frequencies while low-level frequencies are largely available from the input.
Contributions: DRLN introduces densely connected residual blocks and Laplacian attention for accurate super-resolution through multi-shortcut connections and multi-level representation.Laplacian attention is intended to learn features at multiple sub-band frequencies and adaptively model feature dependencies.
Contributions: Cascading residual-on-residual connections help training deep networks by bypassing low-frequency information for more accurate representations.The design combines diverse connection types with cascading structures.
Results: Extensive experiments report that DRLN is efficient and achieves better performance.The supplied introduction states this as the main experimental conclusion.

2 RELATED WORKS

Prior super-resolution research progressed from linear CNNs to residual, recursive, dense, multi-branch, adversarial, and attention-based architectures. DRLN builds on these directions with cascading structures, dense residual modules, and multi-scale attention.

Early CNNs: SRCNN and FSRCNN established early fully convolutional approaches, with FSRCNN operating on the original low-resolution image and using deconvolution for upsampling.FSRCNN also used channel shrinking and expansion to approach real-time CPU operation.
Residual and recursive networks: Skip-connections enabled residual learning, while VDSR and DRCN respectively emphasized global residual learning and parameter sharing through recursion.DRCN reduced parameters significantly, but its performance lagged behind VDSR in the cited discussion.
Dense connections: Dense connections were adopted by SRDenseNet and related models so later convolutional layers could use outputs from earlier layers.SRDenseNet ordered dense blocks before terminal deconvolutional upsampling.
Multi-scale networks: Multi-branch networks were proposed to obtain distinct features at multiple scales by combining branches with different depths or parallel subnets.The cited examples use branch-specific processing followed by feature merging.
Perceptual methods: GAN-based methods added discriminators and perceptual, adversarial, or texture-matching objectives to improve perceptual quality and texture fidelity.SRResNet, EnhanceNet, SRFeat, and ESRGAN represent successive variants of this approach.
Proposed architecture: The proposed architecture uses a long skip connection, short skip connections, cascading structures, and Dense Residual Laplacian Modules as its backbone.Figure 2 presents the overall network above and the DRLM backbone below.
Attention-based methods: RCAN modeled inter-channel dependencies with channel attention, while DRLN combined dense residual blocks with multi-scale attention and reported visual and numerical improvements over RCAN.The comparison passage attributes DRLN’s approach to densely connected residual blocks and skip/cascading connections at multiple levels.

3 OUR MODEL

DRLN combines cascading residual-on-residual blocks, dense residual Laplacian modules, and Laplacian attention in a compact super-resolution pipeline. The design extracts, hierarchically processes, adaptively weights, upsamples, and reconstructs features while targeting a balance between performance, depth, and runtime.

Network Architecture: DRLN comprises feature extraction, cascading residual-on-residual processing, upsampling, and reconstruction components.The low-resolution input is transformed into primitive features, deeply processed, upscaled, and reconstructed into a super-resolved image.
Optimization: The model uses ℓ1 loss for optimization to align its training setup with competing state-of-the-art methods.The authors select ℓ1 among several candidate loss functions for fairness with EDSR, RCAN, and RDN.
Laplacian Attention: Laplacian attention learns nonlinear pyramid-level weighting for residual features across different sub-frequency bands.A global descriptor captures image-wide statistics, while the Laplacian pyramid models important features at multiple scales and adaptively rescales them.
Cascading Residual Structure: Cascaded blocks use medium and long skip connections to ease information flow across dense residual Laplacian modules and blocks.Each block contains DRLMs, feature concatenation, and a medium skip connection, while long skips connect cascaded blocks.
Dense Residual Laplacian Module: Each dense residual Laplacian module combines densely connected residual units, 1×1 compression, and Laplacian pyramid attention.Dense concatenation reuses earlier features, compression reduces the resulting parameter load, and attention follows the compressed features.

4 EXPERIMENTS

The experiments evaluate DRLN on five public super-resolution datasets using PSNR and SSIM, with an additional comparison of object-recognition performance.

DRLN is tested on SET5, SET14, URBAN100, B100, and MANGA109 using PSNR and SSIM on the luminance channel.

4.1 Training settings

The experiments use DIV2K and Flickr2K training data with augmentation, fixed optimization settings, and PyTorch implementation. Table 1 investigates network-component contributions, while Figure 6 provides 4× qualitative comparisons.

Training data and augmentation: The network is trained on DIV2K and Flickr2K with random 90° rotations and horizontal and vertical flipping.
Optimization: Training uses batch size 16, 48 × 48 low-resolution inputs, ADAM, and an initial learning rate of 10^-4 halved every 2 × 10^5 iterations.
Implementation: The implementation uses the PyTorch framework on a Tesla P100 GPU.
Evaluation setup: Table 1 investigates the performance contribution of different network components.
Qualitative evaluation: Figure 6 compares 4× super-resolution results on sharp-edge and textured images from URBAN100 and MANGA109.

4.2 Ablation Studies

The ablation studies examine skip connections and Laplacian attention, alongside the compactness rationale for concatenating previously computed features. The reported design combines multi-frequency feature selection with cascading residual architecture while reducing runtime relative to RCAN.

Skip connections: PSNR on SET5 for 2× is higher when long, medium, and dense local skip connections are all present.
Laplacian attention: Laplacian attention selects essential features through multiple frequency bands, supporting image enhancement and overall accuracy.
Overall architecture: The architecture combines Laplacian pyramid attention with cascading residual-on-residual design to weight features through selection criteria.
Compact design: Concatenating earlier feature maps balances performance, network depth, and runtime by reusing previously computed features.
Runtime comparison: For a 824×1168 MANGA109 image at 4×, DRLN takes 0.045s versus RCAN's 1.14s.

4.3 Comparisons

DRLN is evaluated against state-of-the-art methods across bicubic and blur degradation, standard, noisy, and historical-image settings. It reports favorable quantitative and qualitative results, including gains in PSNR, detail recovery, and robustness across noise levels.

Compared methods: The comparison includes SRCNN, FSRCNN, VDSR, SCN, SPMSR, LapSRN, MSLapSRN, MemNet, EDSR, SRMDNF, D-DBPN, IRCNN, RDN, RCAN, and CARN.Self-ensemble variants are marked with a '+'.
Degradations and scales: Evaluation covers bicubic scales of 2×, 3×, 4×, and 8×, plus 3× blur-downsampling with a Gaussian kernel having 1.6σ2.
Qualitative comparisons: Qualitative comparisons report sharper edges, recovered rectangular structures, separated horizontal lines, and less blended texture than competing methods at 4× and 8×.
Quantitative comparisons: DRLN outperforms all competing approaches across all datasets and scales in the reported PSNR and SSIM evaluation, even without self-ensemble.
Quantitative comparisons: On MANGA109, DRLN's average PSNR increment across all scales is 0.34dB versus RCAN and 3.98dB versus the pioneering SR method.
Blur-downsample comparisons: For 3× blur-downsampled images, DRLN gains 0.55dB and 0.33dB in average PSNR over RDN and RCAN, respectively, across all datasets.
Noisy-image comparisons: For noise levels σ = 10, 15, 20, and 25, DRLN consistently outperforms CNN-based and specifically designed methods.
Historical images: On real historical images with unavailable degradation operators and ground truth, DRLN produces clearer and sharper reconstructions than the compared methods.

4.4 Performance on Object Recognition

The method is evaluated for object recognition using 4× super-resolved ImageNet images, where it achieves the lowest recorded top-1 and top-5 errors among six compared methods. An 8× example also illustrates the difficulty of recovering fine details after severe downsampling.

Object-recognition evaluation: The method produces the lowest top-1 and top-5 object-recognition errors among Bicubic, DRCN, FSRCNN, PSyCo, ENet-E, and RCAN.The evaluation uses the initial 1k ImageNet images, downscaled from 224×224 to 56×56 and then super-resolved.

4.5 Limitations

Although the model renders sharp and clean images across upsampling scales, it struggles to hallucinate finer details when severe downsampling removes the necessary high-frequency information.

Limitation: The model struggles to hallucinate finer details in 8× super-resolution when large downsampling removes high-frequency information.All compared algorithms, including traditional and progressive-upsampling approaches, fail to recover the finest details in the cited example.

5 CONCLUSION

The paper presents a modular convolutional network for accurate super-resolution and evaluates its design across standard, noisy, real-world, and downstream object-recognition settings. Its architecture combines cascading residual connections with densely connected residual blocks, while the authors also describe broader applicability to other low-level vision tasks.

5 CONCLUSION: The proposed network is modular and designed for highly accurate image super-resolution, with comprehensive analysis of its design choices.The evaluation covers super-resolution datasets, noisy low-resolution images, real-world images, and multiple degradation kernels.
5 CONCLUSION: Cascading residual-on-residual connections use long, short, and local skips to pass low-frequency information while learning high- and mid-level frequency information.Densely connected residual blocks reuse previously computed features and provide implicit deep supervision.
5 CONCLUSION: The network is evaluated on super-resolution datasets, noisy images, real-world images with unknown blur downsampling, and object recognition from super-resolved images.Results are also shown for bicubic and blur-down kernels.
5 CONCLUSION: The authors state that the network can also be applied to image restoration, synthesis, and transformation problems.This broader applicability is presented as a potential extension beyond image super-resolution.