PaperlayerView exact revision on arXiv

Source-linked AI summary

Image Super-resolution with An Enhanced Group Convolutional Neural Network

Chunwei Tian, Yixuan Yuan, Shichao Zhang, Chia-Wen Lin, Wangmeng Zuo, David Zhang

arXiv:2205.14548v2cs.CVeess.IV

TL;DR

Single-image super-resolution seeks to recover a high-resolution image from a low-resolution observation, but deeper CNN architectures can raise computational cost. ESRGCNN uses group convolutions, residual operations, signal enhancement, and adaptive up-sampling to build a shallow, flexible model. The paper reports strong results across SISR performance, efficiency, model complexity, and visual quality.

  • Problem

    Deeper CNN architectures can improve SISR performance but may increase computational cost, motivating a more efficient restoration model.

    Source: §1 · p. 2

  • Method

    ESRGCNN uses group convolutions and residual operations to fuse deep and wide channel features, signal enhancement for long-distance context, and adaptive up-sampling for different LR image sizes.

    Source: §1 · p. 3

  • Results

    5.6% of RDN's parameters, 9.6% of CSFM's, and 3% of both methods' running time are reported for ESRGCNN on a 1024 × 1024 HR image, alongside strong results in SISR performance and visual quality.

    Source: §1 · p. 3, §5 · p. 20

  • Takeaways & Limitations

    The authors conclude that ESRGCNN offers a flexible SISR model for LR images of different sizes, with strong reported performance, efficiency, complexity, and visual quality.

    Source: §1 · p. 3, §5 · p. 20

Abstract

from arXiv · show

CNNs with strong learning abilities are widely chosen to resolve super-resolution problem. However, CNNs depend on deeper network architectures to improve performance of image super-resolution, which may increase computational cost in general. In this paper, we present an enhanced super-resolution group CNN (ESRGCNN) with a shallow architecture by fully fusing deep and wide channel features to extract more accurate low-frequency information in terms of correlations of different channels in single image super-resolution (SISR). Also, a signal enhancement operation in the ESRGCNN is useful to inherit more long-distance contextual information for resolving long-term dependency. An adaptive up-sampling operation is gathered into a CNN to obtain an image super-resolution model with low-resolution images of different sizes. Extensive experiments report that our ESRGCNN surpasses the state-of-the-arts in terms of SISR performance, complexity, execution speed, image quality evaluation and visual effect in SISR. Code is found at https://github.com/hellloxiaotian/ESRGCNN.

1. Introduction

Single-image super-resolution is ill-posed because many high-resolution images can produce the same low-resolution observation. ESRGCNN combines channel-feature fusion, signal enhancement, and adaptive up-sampling to target efficient, flexible restoration.

1. Introduction

  • SISR is ill-posed because numerous HR images can produce the same LR image, while some earlier methods lose detail, require manual tuning, or rely on complex optimization.The introduction describes those issues as limiting SISR performance, flexibility, and efficiency.Source: §1 · pp. 1–2
  • Deep networks offer strong self-learning ability for restoration, but deeper SR architectures can increase computational cost.The introduction also notes that bicubic input construction adds cost in some methods.Source: §1 · p. 2
  • Its group enhanced convolutional blocks fuse deep and wide channel features to extract more accurate low-frequency information, while signal enhancement carries long-distance context from shallow layers.Adaptive up-sampling maps low-resolution features to high-resolution features at different scales.Source: §1 · p. 3
  • ESRGCNN uses a 40-layer architecture with group convolutions and residual operations to enhance deep and wide channel correlations for efficient SISR.The introduction presents adaptive up-sampling as a separate contribution for a flexible SR model.Source: §1 · p. 3
  • 5.6% of RDN's parameters, 9.6% of CSFM's, and 3% of both methods' running time are reported for ESRGCNN when recovering a 1024 × 1024 HR image.The introduction also reports excellent visual effects.Source: §1 · p. 3

2. Related work

Related work covers channel-focused group-convolution methods and multilevel feature fusion for SISR. ESRGCNN differs by using adjacent group-convolution layers to enhance channel correlations and reduce complexity.

2.1. Deep CNNs based group convolutions for SISR

  • Prior group-convolution SISR work includes attention-based channel selection and feature-fusion approaches using residual learning or concatenation.The section also describes approaches aimed at reducing model complexity or supplying complementary features.Source: §2.1 · pp. 3–4
  • ESRGCNN enhances channel correlations through two adjacent layers rather than using the current layer as input to later layers, with the stated aim of more accurate low-frequency features and lower complexity.This distinguishes it from other group-convolution SR approaches.Source: §2.1 · p. 4

2.2. Multilevel feature fusion for SISR

  • Multilevel fusion is motivated by the limited robustness of single-feature learning and the poorer memory ability of deep networks for shallow-layer features.The section distinguishes low-frequency from high-frequency feature fusion.Source: §2.2 · p. 4
  • Prior fusion methods either amplify LR images before the network and incur high complexity, or input LR images directly and up-sample deep features to form HR images.The paper adopts the latter approach.Source: §2.2 · pp. 4–5

3. The proposed method

ESRGCNN combines deep and wide channel features in a shallow architecture, with signal enhancement and adaptive upsampling for different input scales. Its group enhanced blocks are designed to strengthen low-frequency features while reducing complexity.

3.1. Network architecture

  • The 40-layer ESRGCNN has two Conv+ReLU stages, six GEBs, an upsampling mechanism, and a final convolution.Its first Conv+ReLU extracts key information; the final convolution constructs the predicted SR image.Source: §3.1 · p. 5

3.2. Loss function

  • MSE is the loss function used to train ESRGCNN parameters for predicting super-resolution images.Training uses low- and high-resolution image patches from the training dataset.Source: §3.2 · pp. 6–7

3.3. Group enhanced convolutional block

  • Each GEB splits channels into quarter-width GConv1 distilling and three-quarter-width GConv2 remaining parts, then fuses adjacent GConv2 outputs with residual learning.Residual learning also merges GConv1 outputs, and concatenation joins the final GConv1 and GConv2 features.Source: §3.3 · pp. 7–8
  • Signal enhancement overlays shallow-layer features on deep-layer features through residual learning to address the shallow layers’ memory problem.The method uses a Conv+ReLU after the sixth GEB to prevent transition enhancement.Source: §3.3 · pp. 8–9

3.4. Adaptive upsampling mechanism

  • A flexible valve selects ×2, ×3, or ×4 upsampling modes, enabling one blind SR model to handle LR images at different scales.With the valve set to 0, all three modes train in parallel; at testing, setting it to 2, 3, or 4 selects one mode.Source: §3.4 · p. 9

4. Experiments and analysis

The experiments evaluate ESRGCNN across datasets, scales, image sizes, and network variants. They report strong benchmark performance alongside speed, complexity, perceptual-quality, and visual comparisons, with network analysis examining feature-fusion mechanisms.

4.1. Training dataset

  • Training uses DIV2K images across ×2, ×3, and ×4 scales, enlarging the training set with validation images, 83 × 83 LR patches, flips, and 90° rotations.The paper notes that limited training samples can make the resulting SR model less robust.Source: §4.1 · p. 10

4.2. Test datasets

  • Testing uses Set5, Set14, B100, and U100 at ×2, ×3, and ×4; comparisons evaluate the Y channel of YCbCr images.The benchmark datasets contain five, fourteen, or 100 color images, depending on the dataset.Source: §4.2 · pp. 10–11

4.3. Experimental Settings

  • The reported training setup uses Adam, a batch size of 32, and a 1e-4 learning rate halved every 4e+5 steps during 6e+5 total steps.Experiments ran with PyTorch 0.41 and Python 2.7 on Ubuntu 16.04, using two Nvidia GeForce GTX 1080Ti GPUs.Source: §4.3 · p. 11

4.4. Network analysis

  • Adjacent GConv2 residual enhancement and wide-feature fusion combine deep and wide features; signal enhancement carries long-distance context, and adaptive upsampling supports varying scales.The analysis describes ablations assessing residual enhancement, wide-feature fusion, and the final Conv+ReLU operation.Source: §4.4 · pp. 11–14

4.5. Comparisons with state-of-the-arts

  • 0.36 dB PSNR and 0.0073 SSIM gains over MADNet-LF on U100 ×3, and 0.12 dB PSNR and 0.0020 SSIM gains over MADNet-LF on Set14 ×4; ESRGCNN is near-best across scales.The authors also report superiority over CARN-M, MADNet-LF, and LESRCNN in Set5 PSNR at ×3 and ×4, and strong performance on B100 and scene-specific comparisons.Source: Table 4 · p. 13, Table 5 · p. 14, Table 7 · p. 16
  • ESRGCNN is reported as the fastest tested method, uses the fewest FLOPs, and achieves better FSIM on B100 at ×2, ×3, and ×4.Running-time tests use ×2 upscaling at three image sizes; complexity is compared for ×2 restoration, and FSIM evaluates perceptual quality.Source: §4.5 · p. 17, Table 11 · p. 19, Table 12 · p. 19
  • Amplified regions are clearer with ESRGCNN than with Bicubic, CARN-M, and LESRCNN in the illustrated U100 ×3 and B100 ×2 comparisons.The figures also report PSNR and SSIM for the compared methods.Source: Figure 4 · p. 18, Figure 5 · p. 19, §4.5 · p. 20

5. Conclusion

ESRGCNN combines channel-feature fusion, signal enhancement, and adaptive up-sampling for SISR. Experiments report strong results across restoration, efficiency, complexity, and visual quality.

5. Conclusion

  • Experiments report that ESRGCNN achieves an excellent effect across SISR results, efficiency, model complexity, and visual quality.The paper says it plans to design lightweight CNNs for blind image super-resolution in future work.Source: §5 · p. 20
  • ESRGCNN uses correlations among deep and wide channel features to extract more accurate low-frequency information for SISR.Source: §5 · p. 20
  • A signal enhancement operation is fused into ESRGCNN to inherit more long-distance contextual information in addressing long-term dependence.Source: §5 · p. 20
  • Adaptive up-sampling is applied to support a super-resolution model for low-resolution images of different sizes.Source: §5 · p. 20
  • The authors identify lightweight CNNs for blind image super-resolution as future work.Source: §5 · p. 20

Related papers

Loading 2205.14548v2…