Source-linked AI summary
Compression Artifacts Removal Using Convolutional Neural Networks
Pavel Svoboda, Michal Hradis, David Barina, Pavel Zemcik
TL;DR
Deep CNN training for JPEG artifact removal faces initialization effects that leave many deeper units nearly useless. The paper combines residual learning, skip connections, and symmetric initialization to train eight-layer networks; L8 outperforms the other methods across three quality measures, and residual learning gives the best reconstruction among the tested objectives.
Problem
Random filter offsets and positive ReLU offsets can leave most units in deep layers nearly useless after initialization.
Source: §3 · p. 5Method
The paper combines residual learning, skip architecture, and symmetric weight initialization to train eight-layer JPEG artifact-removal networks.
Source: §5 · p. 8Results
Takeaways & Limitations
One network trained across multiple JPEG qualities provides an acceptable trade-off.
Source: §5 · p. 9Generalization of L8 and larger networks is not guaranteed when trained on the small BSD dataset.
Source: §5 · p. 9
Abstract
from arXiv · showhide
This paper shows that it is possible to train large and deep convolutional neural networks (CNN) for JPEG compression artifacts reduction, and that such networks can provide significantly better reconstruction quality compared to previously used smaller networks as well as to any other state-of-the-art methods. We were able to train networks with 8 layers in a single step and in relatively short time by combining residual learning, skip architecture, and symmetric weight initialization. We provide further insights into convolution networks for JPEG artifact reduction by evaluating three different objectives, generalization with respect to training dataset size, and generalization with respect to JPEG quality level.
1 INTRODUCTION
The section frames JPEG compression artifacts as consequences of lossy information removal and introduces direct CNN restoration that predicts image corrections.
1 INTRODUCTION
- A fully convolutional network maps a degraded image directly to a high-quality image by predicting a residual describing changes to apply.Skip connections let information bypass middle layers; centered filters reduce deep-layer ReLU saturation and allow faster learning rates.Source: §1 · p. 1
- JPEG quantization produces blocking at 8×8 cell edges, ringing near sharp edges, and blurring through high-frequency removal.Blocking is most noticeable in low-frequency regions, while ringing is especially visible around sharp edges.Source: §1 · pp. 1–2
- CNN restoration must recognize compression artifacts and repair them using neighboring image content, taking on data-term and prior-regularization roles.The networks can also use correlations between image content and the degradation.Source: §1 · p. 2
- Prior CNN work includes image-restoration applications, while Dong et al. achieved artifact-reduction state of the art with small networks that struggled to scale because of training convergence.The paper draws inspiration from large deblurring networks and residual learning for super-resolution.Source: §1 · p. 2
2 RELATED WORK
Earlier compression-artifact removal methods ranged from hand-designed filters to learned approaches, while CNN restoration work established precedents for deeper networks and residual learning. The paper adapts residual learning to JPEG reconstruction and reports gains over classical methods and AR-CNN.
2 RELATED WORK
- Compression-artifact removal spans simple filters, probabilistic priors, and advanced machine-learning methods.Source: §2 · p. 2
- Simple compression-artifact methods include FFmpeg's spp filter, which averages JPEG recompressions of shifted versions of an already-compressed image.The filter uses the image's original quantization matrix.Source: §2 · p. 2
- SA-DCT is described as state-of-the-art deblocking, but it can oversmooth images and cannot sharpen edges; several other approaches rely on DCT-grid knowledge.The paper's method is contrasted as able to process images without that knowledge.Source: §2 · p. 2
- CNNs had been applied across restoration tasks, including denoising, deconvolution, and deblurring; prior work also showed deep super-resolution networks could benefit from residual learning and suitable initialization.Earlier examples include a denoising network trained on a sufficiently large dataset and CNNs for text deblurring and license-plate deblurring.Source: §2 · pp. 2–3
- AR-CNN extends SRCNN with feature-enhancement layers and uses a shallow network to initialize a four-layer model after full-network training difficulties.The enhancement layer augments feature extraction.Source: §2 · p. 3
- The proposed JPEG CNN adapts residual learning and significantly outperforms classical artifact-removal methods and AR-CNN on PSNR, PSNR-B, and SSIM.The section describes this as following the assumption that deeper networks are better and training them in a single step.Source: §2 · p. 3
3 CNN IMAGE ENHANCEMENT
The section describes fully convolutional networks for image restoration and compares direct, residual, and edge-emphasized objectives. It also presents skip connections and symmetric weight initialization as design choices for deeper networks.
Fully convolutional networks
- Fully convolutional networks apply convolutional and element-wise operations across image positions and can process images of arbitrary size.The network maps distorted input y to restored output F(y).Source: §3 · p. 3
Objectives
- Direct mapping learns to transform corrupted images into clean images, whereas residual learning predicts changes to apply to the input.The section notes that direct mapping can require the whole image to pass through every layer when inputs and desired outputs are highly correlated.Source: §3 · p. 4
Edge emphasized objective
- The edge-emphasized objective adds loss on vertical and horizontal Sobel-filtered image derivatives to focus on high-frequency structures.The authors propose that this may lead to perceptually better reconstructions.Source: §3 · p. 4
Symmetric weight initialization
- Symmetric initialization forces individual filters to have zero mean, addressing deeper-layer unit saturation and allowing higher initial learning rates.The paper says this results in trained networks with fewer saturated neurons; it does not use a whole-training zero-mean constraint because that prevents use of the input's DC component.Source: §3 · p. 5
Skip architecture
- Skip connections concatenate first-layer activations with deeper-layer activations, helping pass geometric information while middle layers reason about image content.This differs from the cited approach that adds activations together.Source: §3 · p. 5
Network architectures
- L8 uses eight convolutional layers with skip connections, while L4 has four layers; their receptive fields are 25×25 and 19×19, respectively.L8 concatenates activations from the first layer with those from the fourth and sixth layers.Source: §3 · p. 5
4 EXPERIMENTAL RESULTS
The experiments evaluate grayscale JPEG restoration on BSDS500 and LIVE1 images, using JPEG qualities 10–60 and comparing the networks with established artifact-removal methods. They also examine objective choice, generalization across JPEG qualities and training-set sizes, and processing speed.
Artifacts reduction quality
- L8 outperforms all other methods on all three quality measures, while the smaller, more computationally efficient L4 surpasses them in most cases.The reported comparisons cover LIVE1 and BSDS500 validation results at JPEG qualities 10 and 20.Source: Table 3 · p. 6, Table 4 · p. 6, §4 · p. 7
JPEG quality generalization
- A single L4 trained across JPEG qualities gives stable results across the range, while quality-specific networks perform better at their respective qualities.The paper suggests a single network may handle a narrower range, such as 10–20 quality points, when trained on data spanning that range.Source: §4 · p. 7
Impact of learning objective
- Residual-network first-layer filters become more complex and higher-frequency; edge-preserving filters include low-pass filters, while direct-mapping filters remain noisy.The paper suggests the edge-preserving network's low-pass filters may transfer general image appearance; it presents possible explanations for the noisy direct-mapping filters.Source: §4 · pp. 7–8
- Residual learning improves L4 reconstruction quality and training speed, while the edge-preserving objective adds no noticeable quality improvement.Learning rates and weight decay were selected separately for each objective, then used across JPEG qualities.Source: §4 · pp. 7–8
Dataset size
- L8 overfits substantially more on smaller training sets and requires more images, while L4 appears to generalize well with 400 images.The comparison uses residual networks trained on 4, 16, 64, 256, or 400 images; L4 and L8 have approximately 70 K and 220 K parameters, respectively.Source: §4 · p. 8
Speed
- L4 processes a 1 Mpx image in 220 ms, compared with 1052 ms for L8.The networks require approximately 140 K and 440 K floating-point operations per pixel, respectively.Source: §4 · p. 8
5 CONCLUSIONS
The conclusions report that large, deep CNNs outperform previous state-of-the-art results from smaller networks, and residual learning gives the best reconstruction among the objectives tested. They also find an acceptable trade-off from training one network across JPEG quality levels, while generalization of larger networks on the small BSD dataset is not guaranteed.
5 CONCLUSIONS
- Large, deep networks outperform previous state-of-the-art results from smaller networks on JPEG artifact removal.The authors combine residual learning, skip architecture, and symmetric weight initialization to train networks with eight layers.Source: §5 · p. 8
- Residual learning provides the best reconstruction results among direct mapping, residual learning, and edge-preserving objectives.Source: §5 · p. 8
- The networks, processed images, and implementations are made available online.Source: §5 · p. 8
- One network trained across several JPEG quality levels is reported as an acceptable trade-off.Source: §5 · p. 9
- Generalization is not guaranteed for L8 (100 K parameters) and larger networks trained on the small BSD dataset.The authors call for a larger common dataset for fair and consistent evaluation.Source: §5 · p. 9
- Future work proposes JPEG 2000, JPEG XR, and WebP, reconstruction from JPEG coefficients, and larger receptive fields or added context.The authors expect larger receptive fields or other context to enable higher reconstruction quality; JPEG-coefficient input should provide clues about which image elements are artifacts.Source: §5 · p. 9