PaperlayerView exact revision on arXiv

Source-linked AI summary

Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections

Xiao-Jiao Mao, Chunhua Shen, Yu-Bin Yang

arXiv:1603.09056v2cs.CV

TL;DR

Image restoration seeks to recover clean images from corrupted observations, and the paper investigates deep networks for denoising and super-resolution. It proposes a deep convolutional encoder-decoder with symmetric skip connections and reports better performance than state-of-the-art methods on both tasks.

  • Problem

    Image restoration is an ill-posed inverse problem, and the paper asks whether deeper networks can improve performance and one model can handle different corruption levels.

    Source: §1 · p. 1

  • Method

    A very deep convolutional encoder-decoder learns mappings from corrupted images to clean ones, with symmetric skip connections between corresponding layers.

    Source: §1 · pp. 1–2

  • Results

    The network achieves better performance than state-of-the-art methods on image denoising and super-resolution.

    Source: §1 · p. 2, §4 · p. 8

  • Takeaways & Limitations

    The authors propose using one large-capacity model to process different levels of noise or corruption.

    Source: §1 · p. 2

Abstract

from arXiv · show

In this paper, we propose a very deep fully convolutional encoding-decoding framework for image restoration such as denoising and super-resolution. The network is composed of multiple layers of convolution and de-convolution operators, learning end-to-end mappings from corrupted images to the original ones. The convolutional layers act as the feature extractor, which capture the abstraction of image contents while eliminating noises/corruptions. De-convolutional layers are then used to recover the image details. We propose to symmetrically link convolutional and de-convolutional layers with skip-layer connections, with which the training converges much faster and attains a higher-quality local optimum. First, The skip connections allow the signal to be back-propagated to bottom layers directly, and thus tackles the problem of gradient vanishing, making training deep networks easier and achieving restoration performance gains consequently. Second, these skip connections pass image details from convolutional layers to de-convolutional layers, which is beneficial in recovering the original image. Significantly, with the large capacity, we can handle different levels of noises using a single model. Experimental results show that our network achieves better performance than all previously reported state-of-the-art methods.

1 Introduction

Image restoration is an ill-posed problem, and the paper asks whether deeper networks can improve performance and one model can handle different corruption levels. It proposes a deep convolutional encoder-decoder with symmetric skip connections for restoration.

1 Introduction

  • The paper frames image restoration as an ill-posed inverse problem and asks whether deeper networks improve performance and one model can handle varying corruption levels.The motivating tasks include denoising and super-resolution.Source: §1 · p. 1
  • The proposed very deep architecture maps corrupted images to clean ones using convolutional feature extraction and deconvolutional detail recovery.Convolutional layers encode image content while eliminating corruption; deconvolutional layers decode the abstraction to recover details.Source: §1 · pp. 1–2
  • Symmetric skip connections link corresponding convolutional and deconvolutional layers, aiding gradient back-propagation and image-detail transfer.The authors report faster convergence, a higher-quality local optimum, and performance gains as the network deepens.Source: §1 · p. 2
  • The large-capacity network is proposed to handle different corruption levels with a single model.The authors describe this as the first approach, to their knowledge, to achieve good accuracy across noise levels using one model.Source: §1 · p. 2
  • The network achieves better performance than state-of-the-art methods on image denoising and super-resolution.The authors report new records on both topics.Source: §1 · p. 2

2 Very deep RED-Net for Image Restoration

RED-Net pairs symmetric convolutional encoding with deconvolutional decoding to map corrupted images to clean images. Mirrored skip connections pass feature maps between these stages, supporting detail recovery and easier training of deeper networks.

2.1 Architecture

  • RED-Net combines convolutional feature extraction with mirrored deconvolutional reconstruction, summing corresponding feature maps through skip connections.The network uses no pooling or unpooling, accepts arbitrary image sizes, and the paper evaluates 20- and 30-layer versions.Source: §2 · p. 2, §2.1 · pp. 2–3, Figure 1 · p. 3

2.1.1 Deconvolution decoder

  • Unlike a fully convolutional model that reduces noise layer by layer, RED-Net uses convolution to preserve primary image content and deconvolution to compensate for details.In a comparison at noise level σ = 70, the convolution-deconvolution network had better PSNR than its fully convolutional counterpart.Source: §2.1.1 · p. 3

2.1.2 Skip connections

  • Skip connections pass image details to mirrored decoder layers and gradients toward bottom layers, addressing detail loss and making deeper networks easier to train.The paper places these shortcuts every two convolutional layers in its experiments.Source: §2.1.2 · pp. 3–4, Figure 2 · p. 4

2.2 Discussions

  • With five strided convolutional layers, deconvolution struggled to recover image details from an abstract vector, while skip connections enabled recovery of the input.Source: §2.2 · p. 4
  • With symmetric skip connections, deeper networks have smaller training loss, and element-wise RED-Net connections often converge to a better optimum than the compared He et al. connections.The comparison highlights the element-wise correspondence of RED-Net connections.Source: §2.2 · p. 5, Figure 3 · p. 5
  • A model trained for one noise level did not handle σ = 10, 30, 50, and 70 well; the authors therefore target one model for different corruption levels.They identify network capacity as key to this goal, including different noise levels for denoising and scaling parameters for super-resolution.Source: §2.2 · p. 5

2.3 Training

  • Training minimizes mean squared error between network outputs and clean-image targets; the authors use Adam with learning rate 10^-4 and train on patches from 300 Berkeley dataset images.The training set contains about 0.5 million denoising patches; super-resolution inputs are downsampled and then upsampled patches.Source: §2.3 · pp. 5–6

2.4 Testing

  • At test time, the network processes images of arbitrary size, and averaging outputs from multiple image orientations can yield slightly better, smoother restoration.Source: §2.4 · p. 6

3 Experiments

Experiments evaluate RED-Net on denoising and super-resolution benchmarks and test whether one model can handle multiple corruption levels. The reported results show gains over competing methods, with a small performance degradation in the single-model setting.

3 Experiments

  • The experiments compare denoising on 14 benchmark images and BSD200, and super-resolution on Set5, Set14 and BSD100, using PSNR and SSIM.Denoising tests Gaussian noise at σ = 10, 30, 50 and 70; super-resolution tests scales 2, 3 and 4.Source: §3 · p. 6

3.1 Image Denoising

  • On BSD200, RED-Net exceeds WNNM by 0.38dB, 0.47dB, 0.49dB and 0.42dB in PSNR at σ 10, 30, 50 and 70, respectively.The evaluation uses grayscale, resized images and reports average PSNR and SSIM.Source: §3.1 · p. 7, Table 2 · p. 7

3.2 Image super-resolution

  • On Set5, RED30 exceeds CSCN by 0.52dB, 0.56dB and 0.47dB at scales 2, 3 and 4; gains are smaller on Set14.On Set14, the corresponding PSNR gains are 0.23dB, 0.06dB and 0.1dB; BSD100 shows a similar pattern to Set5, with RED10 already surpassing CSCN.Source: §3.2 · p. 7, Table 3 · p. 7, Table 4 · p. 7

3.3 Evaluation with a single model

  • With different corruption levels, the single-model network’s performance only slightly degrades versus separate models, and it still beats existing methods except CSCN at super-resolution scales 3 and 4.The evaluation trains a 30-layer network for denoising and super-resolution using patches with different noise levels and scaling parameters.Source: §3.3 · pp. 7–8, Table 6 · p. 8

4 Conclusions

The paper concludes that its deep encoding-decoding framework achieves better performance than state-of-the-art methods on image denoising and super-resolution. It attributes deeper-network performance gains to skip connections that aid clean-image recovery and address gradient-vanishing optimization difficulty.

4 Conclusions

  • The framework combines convolutional feature extraction with deconvolutional recovery of image details.Convolutions extract primary image content; deconvolutions recover details.Source: §4 · p. 8
  • Skip connections help recover clean images and address gradient-vanishing optimization difficulty, yielding performance gains as the network deepens.Source: §4 · p. 8
  • The network achieves better performance than state-of-the-art methods on denoising and super-resolution.Source: §4 · p. 8

Related papers

Loading 1603.09056v2…