Source-linked AI summary
DMCNN: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal
Xiaoshuai Zhang, Wenhan Yang, Yueyu Hu, Jiaying Liu
TL;DR
JPEG compression can introduce artifacts, while CNN-based restoration methods often use receptive fields too small to address large-scale banding. DMCNN combines large-receptive-field strategies with pixel- and DCT-domain processing, and the paper reports state-of-the-art JPEG artifact removal performance among CNN-based methods.
Problem
CNN-based JPEG restoration methods often use small receptive fields, limiting their access to image context when artifacts such as banding extend across large regions.
Source: §1 · pp. 1–2Method
DMCNN combines auto-encoder-style pixel- and DCT-domain networks with dilated convolutions and multi-scale loss to extract global information.
Source: §1 · p. 2, CONVOLUTIONAL NEURAL NETWORK (DMCNN) · p. 2, §4 · p. 4Results
DMCNN outperforms other methods across the evaluated datasets, quality factors, and metrics, and is reported as state of the art among CNN-based JPEG artifact removal networks.
Source: §1 · p. 2, §3.2 · p. 4Takeaways & Limitations
The large receptive fields help DMCNN handle quantization banding and recover lost details using regional patterns.
Source: §3.3 · p. 4
Abstract
from arXiv · showhide
JPEG is one of the most commonly used standards among lossy image compression methods. However, JPEG compression inevitably introduces various kinds of artifacts, especially at high compression rates, which could greatly affect the Quality of Experience (QoE). Recently, convolutional neural network (CNN) based methods have shown excellent performance for removing the JPEG artifacts. Lots of efforts have been made to deepen the CNNs and extract deeper features, while relatively few works pay attention to the receptive field of the network. In this paper, we illustrate that the quality of output images can be significantly improved by enlarging the receptive fields in many cases. One step further, we propose a Dual-domain Multi-scale CNN (DMCNN) to take full advantage of redundancies on both the pixel and DCT domains. Experiments show that DMCNN sets a new state-of-the-art for the task of JPEG artifact removal.
1. INTRODUCTION
The paper argues that small receptive fields limit CNN-based JPEG restoration, particularly for large-scale banding, and proposes DMCNN to use large receptive fields and features from both pixel and DCT domains. Evaluations on BSDS500 and LIVE1 are reported to establish state-of-the-art performance among CNN-based JPEG artifact removal networks.
1. INTRODUCTION
- JPEG quantization can produce blocking, ringing, and banding artifacts, which become more problematic as lossy compression rates increase.The paper links blocking to DCT block boundaries, ringing to sharp edges, and banding to visible image regions.Source: §1 · p. 1
- Earlier filter-based and sparse-coding methods can sharpen compressed images but are often slow and may introduce artifacts; CNN approaches also include pixel-only and dual-domain models.The paper notes that many CNN methods use only the pixel domain and incorporate little JPEG prior knowledge, while DDCN combines DCT-domain prior with CNNs.Source: §1 · p. 1
- Small receptive fields restrict CNN restorations to nearby pixels, while JPEG banding can span large image regions.The passage describes ARCNN as failing to recover sky banding because of its small receptive fields.Source: §1 · pp. 1–2
- DMCNN enlarges receptive fields with an auto-encoder architecture, dilated convolutions, and multi-scale loss, and adds a DCT-domain branch.The DCT branch is intended to exploit redundancies in that domain.Source: §1 · p. 2
- DMCNN is reported as state of the art among CNN-based JPEG artifact removal networks on BSDS500 and LIVE1.The proposed end-to-end CNN uses large receptive fields and dual-domain multi-scale features.Source: §1 · p. 2
CONVOLUTIONAL NEURAL NETWORK (DMCNN)
DMCNN combines pixel- and DCT-domain auto-encoders, with dilated convolutions and multi-scale losses to use broader context and features at different scales. Its DCT Rectify Unit constrains estimates to quantization-consistent values.
CONVOLUTIONAL NEURAL NETWORK (DMCNN)
- DMCNN uses similar auto-encoder networks in pixel and DCT domains, processes the DCT branch first, and forms the restoration as a weighted sum of the input and both branch estimates.Shortcuts between symmetric layers support local and global features, while residual learning is also employed.Source: CONVOLUTIONAL NEURAL NETWORK (DMCNN) · p. 2, §2.1 · p. 2, Figure 2 · p. 3
2.1. Auto-Encoder
- The auto-encoder treats the compressed image as input, extracts artifact-free features with encoder E, and restores the image from them with decoder D.The model also links symmetric layers with shortcuts and uses residual learning.Source: §2.1 · p. 2
2.2. Dilated Convolution
- 145 × 145 receptive fields are about 58 times larger than ARCNN’s 19 × 19 and 13 times larger than DnCNN’s and DDCN’s 41 × 41.The proposed model uses dilated convolutional layers with dilation factors 2, 4, and 8 in the auto-encoder middle layers.Source: §2.2 · pp. 2–3
2.3. DCT Rectify Unit (DRU)
- The DCT Rectify Unit crops DCT estimates outside the range allowed by the quantized coefficient, while values inside the range remain unchanged.Unlike, the unit drops the leaky slope α because no gain was observed with it.Source: §2.3 · p. 3
2.4. Multi-scale DCT-Embedded Loss
- Multi-scale loss guides the pixel decoder to reconstruct scaled images from features at different scales, and a separate DCT loss trains the DCT branch.The loss weights λ and θ are typically in the range [0, 1].Source: §2.4 · p. 3
3. EXPERIMENTS
DMCNN outperforms the compared methods across the evaluated datasets, quality factors, and metrics, and its LIVE1 restorations are described as more visually pleasing. Experiments train on ImageNet and evaluate on LIVE1 and BSDS500.
3.1. Implement Details
- ImageNet training used grayscale Y-channel images, with LIVE1 and BSDS500 testing; QF20 training expanded patches from 56×56 to 224×224 before QF10 fine-tuning.Training pairs were dynamically generated, and the QF10 model was trained from the QF20 model.Source: §3.1 · pp. 3–4
3.3. Subjective Comparisons
- On LIVE1 visual comparisons at QF=10, DMCNN appears more pleasing, handling quantization banding and recovering lost details through regional patterns.The figure compares different algorithms at QF=10.Source: §3.3 · p. 4, Figure 3 · p. 4
3.2. Objective Comparisons
- DMCNN outperforms the other methods on all evaluated datasets, QFs, and metrics, with gains over pixel-domain methods, general frameworks, and DDCN.The quantitative results are reported for LIVE1 and BSDS500.Source: Table 1 · p. 4, Table 2 · p. 4, §3.2 · p. 4
4. CONCLUSIONS
DMCNN is a dual-domain autoencoder network that uses dilated convolutional layers and multi-scale loss to extract global information and remove JPEG compression artifacts.