Source-linked AI summary
CAS-CNN: A Deep Convolutional Neural Network for Image Compression Artifact Suppression
Lukas Cavigelli, Pascal Hager, Luca Benini
TL;DR
JPEG compression reduces data size but introduces visual artifacts that degrade image quality and can affect low-level vision algorithms. This paper develops a 12-layer ConvNet with hierarchical skip connections and a multi-scale loss for artifact suppression. It reports gains over ordinary JPEG and prior ConvNets, including resilience across compression quality factors.
Problem
Lossy JPEG compression creates visible artifacts while serving storage and transmission constraints, motivating improved artifact suppression.
Method
The paper presents a 12-layer deep ConvNet with hierarchical skip connections and a multi-scale loss, trained using a larger ImageNet-derived dataset.
Results
The network improves PSNR by up to 1.79 dB over ordinary JPEG and up to 0.36 dB over the best previous ConvNet, while QF 60 training gains more than 1.5 dB from QF 40 to 76.
Takeaways & Limitations
A network trained for one JPEG quality factor can suppress artifacts across a wide range of input quality factors.
Abstract
from arXiv · showhide
Lossy image compression algorithms are pervasively used to reduce the size of images transmitted over the web and recorded on data storage media. However, we pay for their high compression rate with visual artifacts degrading the user experience. Deep convolutional neural networks have become a widespread tool to address high-level computer vision tasks very successfully. Recently, they have found their way into the areas of low-level computer vision and image processing to solve regression problems mostly with relatively shallow networks. We present a novel 12-layer deep convolutional network for image compression artifact suppression with hierarchical skip connections and a multi-scale loss function. We achieve a boost of up to 1.79 dB in PSNR over ordinary JPEG and an improvement of up to 0.36 dB over the best previous ConvNet result. We show that a network trained for a specific quality factor (QF) is resilient to the QF used to compress the input image - a single network trained for QF 60 provides a PSNR gain of more than 1.5 dB over the wide QF range from 40 to 76.
I. INTRODUCTION
Lossy compression reduces storage and transmission costs but introduces visible artifacts that can harm visual quality and low-level vision algorithms. The paper proposes a deep ConvNet for suppressing JPEG compression artifacts and evaluates it extensively.
- Lossy compression achieves higher compression ratios than lossless methods but produces non-invertible reconstructions whose quality depends on a size–quality trade-off.
- High compression rates create scheme-specific artifacts, including blocking and ringing, that degrade visual appearance and affect low-level vision algorithms.
- JPEG is widely deployed across consumer, archival, and evidentiary image storage, making artifact suppression broadly relevant.
- Compression is especially useful in power- and storage-constrained sensors and mobile devices because it reduces transmission energy and flash-memory usage.
- The paper presents a deep convolutional architecture, an adaptable training strategy, and extensive LIVE1 evaluations for JPEG compression artifact suppression.
II. RELATED WORK
Prior artifact-suppression methods include edge-aware denoising, sparse recovery, and ConvNets, with recent ConvNet approaches extending simple architectures using deeper or residual designs. Existing evaluations commonly train on BSDS500 and test on LIVE1.
- Traditional methods use edge-aware denoising or dictionary-based sparse recovery to address JPEG-specific ringing and blocking artifacts.
- Single-image super-resolution ConvNets provided inspiration because super-resolution can be viewed as compression artifact removal with simple sub-sampling compression.
- Recent ConvNet approaches include AR-CNN, which uses transfer learning for deeper networks and different quality factors, and a residual architecture described in.
- Earlier networks were trained on 400 BSDS500 images and evaluated on 100 validation images, with LIVE1 used for testing.
III. METHODOLOGY
The methodology builds on deep ConvNet regression for low-level vision while incorporating architectural and training elements intended to make deep regression networks trainable and effective.
- The approach starts from deep ConvNet regression methods developed for super-resolution and other low-level computer vision tasks.
- It incorporates residual architecture, edge-emphasized loss, symmetric initialization, and skip connections as strategies for training deep regression networks.
- The proposed design takes inspiration from deep neural network components while targeting compression artifact reduction.
A. Network Architecture
The proposed network is a 12-layer encoder–decoder-style ConvNet with hierarchical feature concatenation, multi-scale outputs, and training modifications designed to shorten gradient paths and support convergence.
- Network Architecture: The architecture contains blocks A–D with paired convolutional layers, increasing channels from 1 to 128 and then 256 while reducing resolution through 2 × 2 average pooling.
- Network Architecture: The main path uses full-convolution layers eD through eB followed by convolution layer ˆA, forming a 12-layer ConvNet.
- Network Architecture: Hierarchical skip connections concatenate earlier lower-level features with higher-resolution intermediate results at matching resolutions.
- Network Architecture: A multi-scale optimization criterion reconstructs low-resolution images from deep network layers instead of optimizing only the input-to-output mapping.
- Network Architecture: Full-convolution is used in the upsampling path, while PReLU activations follow convolutional and full-convolution layers.
- Network Architecture: Figure 1 distinguishes the main path, lower-level feature concatenations, multi-scale output paths, and reuse of multi-scale outputs by color coding.
- Network Architecture: The authors omit residual-to-input learning because it did not improve the proposed ConvNet's performance.
- Network Architecture: Batch normalization is avoided because batch-wise statistics can introduce jitter and prevent full convergence to maximum accuracy.
B. Performance Metrics
The paper evaluates reconstruction quality using fully referenced pixel-wise metrics, primarily MSE, PSNR, and IPSNR, alongside SSIM and PSNR-B. These measures are simple and widely used, although they do not always match perceived visual quality.
- MSE averages squared pixel-intensity differences between a distorted image and its reference.For the displayed formulation, X is the reference image, ˆX is the evaluated image, P indexes pixels, and e is the per-pixel error.
- PSNR normalizes MSE by the maximum possible signal value and is typically reported in decibels.
- IPSNR measures quality improvement as the PSNR difference from the baseline distorted image.The paper notes that IPSNR is more stable across different datasets.
- MSE and PSNR are fully referenced and converge to zero for perfect reconstruction, but can differ from perceived visual quality.
- SSIM assesses similarity through luminance, contrast, and structure over localized windows.The evaluation uses Matlab's implementation with K1 = 0.01, K2 = 0.03, and an 8×8 window of ones.
C. Loss Function
Training minimizes reconstruction error using a multi-scale loss that supervises full- and lower-resolution outputs, followed by fine-tuning on the full-resolution output alone. The setup excludes an edge-emphasized loss term to align optimization with the benchmark metrics and prioritize overall reconstruction.
- Multi-scale loss: The multi-scale loss gives equal weight to MSEs from the full-resolution output and three lower-resolution outputs.Lower-resolution references are computed by averaging across 4, 16, and 64 pixels, respectively.
- Fine-tuning: Fine-tuning removes the lower-resolution terms and continues optimization using only the full-resolution output loss.Training first proceeds to convergence with the multi-scale objective.
- Training schedule: Figure 2 separates training into a multi-scale-loss phase and a subsequent output-loss fine-tuning phase.The yellow training curve represents the multi-scale loss and is scaled by a factor of 3.
- Loss design: The authors omit an edge-emphasized loss because it adds a hyperparameter and differs from the quality measure used for benchmarking.
- Loss design: The setup prioritizes overall reconstruction quality, with high-quality deblocking treated as implied by that objective.The authors also state that a large training dataset removes the need for this regularization term.
D. Dataset
The dataset replaces the small BSDS500 training source used by previous methods with large-scale ImageNet-derived image patches. Training uses luminance-channel JPEG inputs, with expanded data for fine-tuning and evaluation on LIVE1.
- Dataset construction: ImageNet 2013 provides 396k training and 20k validation color images for generating 120 × 120 pixel cut-outs.The paper chose this dataset to avoid constraining network size by limited training data.
- Dataset construction: Unlike previous methods trained on 400 BSDS500 images, this work uses a substantially larger source dataset.
- Preprocessing: Color images are converted to YCbCr, after which only the luminance channel is retained.
- Preprocessing: The retained channel is compressed with the Matlab JPEG compressor using an 8-bit depth to form network inputs.
- Splits and evaluation: Training uses 50k cut-outs, validation uses 10k, and fine-tuning expands the training set to 150k cut-outs.
- Splits and evaluation: Testing is performed on the 29 images of the LIVE1 dataset.The main training uses quality factor 20 compressed inputs, with networks also trained for other quality factors.
IV. RESULTS & DISCUSSION
CAS-CNN improves restoration quality on LIVE1 across compression settings, with gains in PSNR, SSIM, and PSNR-B over JPEG and prior ConvNet methods. Its performance remains strong when the input quality factor differs from the training quality factor.
- 1.63 dB PSNR improvement over ordinary JPEG is achieved at QF 20, alongside an SSIM of 0.895.
- 1.67 dB PSNR improvement over ordinary JPEG is achieved at QF 10, exceeding the L4 and AR-CNN ConvNet baselines.
- 0.45 dB PSNR-B improvement over previous work is obtained at QF 10, although the L8 network remains better at lower compression rates.
- The method improves mean PSNR and SSIM and enhances each individual LIVE1 image across strong and weak compression.
- The visual comparison shows no perceptible blocking artifacts and strongly suppressed ringing without blurring the railing.
- More than 1.5 dB PSNR improvement is retained across input quality factors from 25 to almost 60 by a network trained at QF 60.
V. CONCLUSION
The paper presents a 12-layer JPEG artifact-suppression network with hierarchical skip connections and multi-scale training. It reports state-of-the-art ConvNet performance, quality-factor resilience, and qualitative improvements over existing ConvNets.
- The proposed network has 12 layers, hierarchical skip connections, and a multi-scale loss function.
- The network achieves up to 1.79 dB PSNR improvement over ordinary JPEG and up to 0.36 dB over the best previous ConvNet.
- A network trained at QF 60 provides more than 1.5 dB PSNR gain across input quality factors from 40 to 76.