PaperlayerView exact revision on arXiv

Source-linked AI summary

S-Net: A Scalable Convolutional Neural Network for JPEG Compression Artifact Reduction

Bolun Zheng, Rui Sun, Xiang Tian, Yaowu Chen

arXiv:1810.07960v1cs.CV

TL;DR

JPEG artifact-reduction CNNs face challenges with color restoration, choosing network depth, and the costs of larger models. S-Net combines a symmetric encoder-decoder with greedy loss to adjust depth and evaluate its effects. The paper reports state-of-the-art benchmark performance and validation on WIN143.

  • Problem

    JPEG artifact-reduction CNNs face challenges with color restoration and selecting an effective network depth, while larger networks raise computational and training costs.

    Source: §1 · p. 2, §2 · p. 5

  • Method

    S-Net combines a symmetric convolutional encoder-decoder with greedy loss architecture to adjust network depth and evaluate convolutional-unit outputs.

    Source: §1 · p. 3, §3.2 · p. 7

  • Results

    S-Net achieved state-of-the-art PSNR and SSIM on all evaluated public benchmark datasets, and WIN143 evaluation validated the algorithm's generality and universality.

    Source: §1 · p. 3, §5 · p. 22

  • Takeaways & Limitations

    Greedy loss architecture supports rapid depth evaluation after one training run and helps identify the minimal network scale with maximum possible performance.

    Source: §5 · p. 21

Abstract

from arXiv · show

Recent studies have used deep residual convolutional neural networks (CNNs) for JPEG compression artifact reduction. This study proposes a scalable CNN called S-Net. Our approach effectively adjusts the network scale dynamically in a multitask system for real-time operation with little performance loss. It offers a simple and direct technique to evaluate the performance gains obtained with increasing network depth, and it is helpful for removing redundant network layers to maximize the network efficiency. We implement our architecture using the Keras framework with the TensorFlow backend on an NVIDIA K80 GPU server. We train our models on the DIV2K dataset and evaluate their performance on public benchmark datasets. To validate the generality and universality of the proposed method, we created and utilized a new dataset, called WIN143, for over-processed images evaluation. Experimental results indicate that our proposed approach outperforms other CNN-based methods and achieves state-of-the-art performance.

1 Introduction

JPEG compression can introduce blocking, ringing, and blurring artifacts, while existing CNN approaches face challenges with color restoration, choosing network depth, and deep columnar architectures. S-Net addresses these issues with color-image training and a scalable encoder-decoder approach, and the paper reports state-of-the-art benchmark performance.

1 Introduction

  • JPEG divides input images into 8 × 8 blocks, applies a separate DCT to each, then quantizes the 64 coefficients using quality-factor-adjusted standard tables, mainly introducing blackness, ringing, and blurring artifacts.Source: §1 · p. 1
  • Existing CNN approaches face challenges with color-image restoration, selecting a performance-maximizing depth, and columnar architectures with many layers before the loss layer.The passage notes that separate-channel restoration can introduce chromatic aberrations and that using only part of a transforming stack could degrade results.Source: §1 · p. 2
  • S-Net uses color image pairs for direct color restoration and a symmetric encoder-decoder with greedy loss to scale the network and evaluate depth effects.The proposed method aims to maximize the performance of each convolutional layer.Source: §1 · p. 3
  • S-Net achieved state-of-the-art performance on all evaluated public datasets in PSNR and SSIM.Models were trained on DIV2K and evaluated on LIVE1 and BSDS500.Source: §1 · p. 3

2 Related Work

Prior image-restoration CNNs include architectures for JPEG artifact reduction and encoder-decoder designs, but deeper networks can be difficult to train or evaluate. Larger networks also increase computational costs and training time.

2 Related Work

  • ARCNN was proposed for JPEG artifact reduction, but its results deteriorated with increasing depth and convergence depended on careful weight initialization.ARCNN is described as a four-layer CNN without pooling or fully connected layers.Source: §2 · p. 3
  • Image-restoration approaches include residual CNNs, DCT-domain processing, stepped and deconvolutional layers, and encoder-decoder designs.The cited work also notes that RED-Net-style convolutional autoencoders did not explicitly present the relationship between the whole network and its encoder-decoder pairs.Source: §2 · p. 4
  • Larger restoration networks incur higher computational costs and longer training, and evaluating whether they reach peak performance also adds cost.The study focuses on maximizing performance with fewer parameters and minimizing training time.Source: §2 · p. 5

3 Proposed Method

S-Net combines a symmetric convolutional encoder-decoder with a greedy-loss design that evaluates convolutional units at different depths. Its modular encoder, decoder, and feature mapper support scalable operation, with a wider, shallower network design.

3.1 Symmetric Convolutional Encoder-Decoder Model

  • The encoder-decoder replaces fully connected autoencoder layers with convolutional layers, and its encoder and decoder use symmetric kernel sizes.The decoder uses transposed convolutions, but this change did not improve benchmark performance.Source: §3.1 · pp. 6–7

3.2 Greedy Loss Architecture

  • Greedy loss attaches a decoder and loss layer after each convolutional unit, enabling depth evaluation without repeatedly training the whole network.Units receive different back-propagation gradients, and constraining their outputs is described as helpful for avoiding gradient explosion.Source: §3.2 · pp. 7–8

3.3 Building a Scalable Convolutional Neural Network

  • The encoder, decoder, and nonlinear feature mapper form independently operable parts, and the encoder-decoder pairs still produce improved outputs without the mapper.Removing only part of the mapper can also yield comparatively good results.Source: §3.3 · p. 9
  • S-Net uses a wider, shallower design with L = 8 and 3×3 filters of 256 channels in its convolutional units.The authors motivate this design by concerns about gradient flow and dynamic scaling in very deep networks.Source: §3.3 · p. 10
  • Convolutional units use residual connections; the advanced structure has two convolutional layers and a ReLU, while the classic structure has one convolutional layer and a ReLU.Batch normalization is omitted based on the authors’ finding that this modification is also effective for artifact reduction.Source: §3.3 · pp. 10–11
  • The training objective uses equally weighted MSE losses across metrics, while replacing ReLU with PReLU did not improve results and increased computational and memory costs.The models use Adam with default parameters.Source: §3.3 · pp. 11–12

4 Experimental Evaluation

On public benchmarks, S-Net with advanced residual units significantly improved over other methods, and WIN143 extends evaluation to over-processed images. The experiments also assess greedy depth scaling, convolutional-unit design, and computational efficiency.

4.1 Dataset

  • LIVE1 and BSDS500 were the public evaluation datasets; the experiments used 29 LIVE1 images and 200 BSDS500 test images.DIV2K test results were unavailable because ground truth had not been released.Source: §4.1 · p. 13

4.2 Training

  • Training used 48×48 RGB patches from DIV2K, with QF40 models trained for 2×10^5 updates and QF10/QF20 models fine-tuned from them.Experiments used Keras with TensorFlow on an NVIDIA K80 GPU server.Source: §4.2 · p. 14

4.3 Greedy Loss Architecture Performance Evaluation

  • Greedy loss training outperformed columnar training on both PSNR and SSIM and produced more stable intermediate outputs when units were removed.The comparison used matching patches and learning rates after 2×10^5 batch updates.Source: §4.3 · p. 16
  • S-Net's successive metrics outperformed same-scale columnar CNNs after the second metric, while both approaches showed little gain beyond four to six units.For columnar CNNs, the PSNR decrease from eight to four units was less than 0.1 dB; performance did not significantly improve beyond six units.Source: §4.3 · pp. 16–17

4.4 Evaluation of Different Convolutional Units

  • Advanced residual blocks improved each metric by an average of 0.20 dB over classic residual blocks, at the cost of twice as many nonlinear-feature-mapper parameters.Source: §4.4 · p. 17, Figure 8 · p. 17

4.5 Comparisons to State-of-the-Art

  • S-Net with advanced residual units significantly improved over ARCNN, DDCN, and CAS-CNN on all public benchmark datasets.Table 3 compares the approaches on BSDS500 and LIVE1.Source: §4.5 · p. 18
  • S-Net with one or two convolutional units improved image quality and computational efficiency in the K80 GPU comparison.Efficiency was measured in million color pixels per second; ARCNN was excluded because its reconstructed image quality differed substantially from the other methods.Source: §4.5 · p. 18, Figure 10 · p. 20

4.6 Extensional Performance Evaluation on the WIN143 Dataset

  • On WIN143, 28 of 143 ARCNN restorations had worse PSNR, while S-Net maintained good performance and better color, intensity, and texture recovery.The WIN143 images were specially shot, post-processed, or computer-generated; the experiment used JPEG quality 20 and images downscaled by half.Source: §4.6 · pp. 19–21

5 Conclusion

S-Net combines a symmetric convolutional encoder-decoder with greedy loss to adjust network depth and evaluate performance across scales. The authors report top-ranking public-benchmark results and validation on WIN143.

5 Conclusion

  • S-Net applies a symmetric convolutional encoder-decoder and greedy loss architecture to dynamically adjust network depth.Source: §5 · p. 21
  • Greedy loss enables depth-performance evaluation after one training run and helps identify the minimum network scale with maximum possible performance.The conventional architecture required several training sessions for the same evaluation.Source: §5 · p. 21
  • The authors report top-ranking results on public benchmarks and say WIN143 evaluation validated the algorithm's generality and universality.Source: §5 · pp. 21–22

Related papers

Loading 1810.07960v1…