PaperlayerView exact revision on arXiv

Source-linked AI summary

An End-to-End Compression Framework Based on Convolutional Neural Networks

Feng Jiang, Wen Tao, Shaohui Liu, Jie Ren, Xun Guo, Debin Zhao

arXiv:1708.00838v1cs.CV

TL;DR

Image compression needs high-quality reconstruction at low bit rates, but existing post-processing approaches can be computationally expensive and deep-learning methods may lack codec compatibility. The paper integrates ComCNN, an image codec, and RecCNN into an end-to-end framework trained with a unified method, and reports strong gains across JPEG, JPEG2000, and BPG settings.

  • Problem

    Existing post-processing methods can require computationally expensive iterative optimization, while deep-learning compression methods have ignored compatibility with existing image codecs.

  • Method

    ComCNN learns a compact structural representation for image-codec encoding, RecCNN reconstructs the decoded image, and both CNNs are trained simultaneously in an end-to-end pipeline.

  • Results

    The framework outperforms compared methods; for JPEG2000, it averages 3.06dB, 2.45dB, 1.34dB, and 1.09dB PSNR gains over 0.1–0.4bpp, respectively.

  • Takeaways & Limitations

    The framework combines improved compression performance with compatibility with JPEG, JPEG2000, and BPG, while being faster than most post-processing algorithms.

Abstract

from arXiv · show

Deep learning, e.g., convolutional neural networks (CNNs), has achieved great success in image processing and computer vision especially in high level vision applications such as recognition and understanding. However, it is rarely used to solve low-level vision problems such as image compression studied in this paper. Here, we move forward a step and propose a novel compression framework based on CNNs. To achieve high-quality image compression at low bit rates, two CNNs are seamlessly integrated into an end-to-end compression framework. The first CNN, named compact convolutional neural network (ComCNN), learns an optimal compact representation from an input image, which preserves the structural information and is then encoded using an image codec (e.g., JPEG, JPEG2000 or BPG). The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high-quality in the decoding end. To make two CNNs effectively collaborate, we develop a unified end-to-end learning algorithm to simultaneously learn ComCNN and RecCNN, which facilitates the accurate reconstruction of the decoded image using RecCNN. Such a design also makes the proposed compression framework compatible with existing image coding standards. Experimental results validate that the proposed compression framework greatly outperforms several compression frameworks that use existing image coding standards with state-of-the-art deblocking or denoising post-processing methods.

I. INTRODUCTION

Existing compression and post-processing methods face practical limitations, while this paper introduces an end-to-end framework that jointly optimizes compact representation, coding, and reconstruction using CNNs.

  • Traditional codecs reduce irrelevance and redundancy by distributing bits among quantized transform coefficients, lowering bits per pixel as compression increases.
  • Iterative nonlocal and low-rank post-processing methods can improve deblocking but are computationally expensive and difficult to apply in practical applications.
  • The proposed framework combines ComCNN, an image codec, and RecCNN to learn a compact structural representation and reconstruct decoded images end to end.
  • The unified learning algorithm simultaneously trains both CNNs despite the image codec’s nondifferentiable quantization rounding function.
  • Unlike prior methods that focus only on post-processing, the framework jointly optimizes encoder front-end and decoder back-end processing while remaining compatible with JPEG, JPEG2000, and BPG.

B. Image Super-Resolution Based on Deep Learning

Deep learning has been applied to super-resolution and image compression, achieving competitive results, but existing deep-learning compression methods often lack compatibility with established image codecs.

  • CNN-based super-resolution methods use architectures such as SRCNN and benefit from residual learning and gradient-based optimization for training deeper networks.
  • Deep-learning image compression has addressed both lossy and lossless settings, including variable-rate and arbitrary-size image compression with recurrent neural networks.
  • Prior deep-learning compression methods achieved competitive or state-of-the-art results but generally ignored compatibility with existing image codecs.
  • The proposed framework is illustrated as combining two CNNs with an image codec in an end-to-end compression pipeline.

A. Architecture of End-to-End Compression Framework

The framework combines ComCNN, an image codec, and RecCNN in an end-to-end pipeline: ComCNN creates a compact structural representation, while RecCNN enhances the decoded image.

  • A. Architecture of End-to-End Compression Framework: ComCNN generates a compact representation that preserves image structure for encoding with an image codec.The compact representation is produced from the input image before codec encoding.
  • A. Architecture of End-to-End Compression Framework: ComCNN uses three weight layers that maintain spatial structure and support RecCNN-based reconstruction.Its layers perform patch representation, downscaling and feature enhancement, then compact-representation construction.
  • A. Architecture of End-to-End Compression Framework: RecCNN contains 20 weight layers combining convolution, ReLU, batch normalization, and convolutional output reconstruction.Residual learning and batch normalization are used during training.
  • A. Architecture of End-to-End Compression Framework: RecCNN enhances the decoded image and reconstructs the high-quality output through residual learning.It predicts a residual image that is added to the upscaled decoded input.
  • A. Architecture of End-to-End Compression Framework: The end-to-end pipeline maps an original image through ComCNN, an image codec, and RecCNN to produce the reconstructed image.The codec may be JPEG, JPEG2000, or BPG.
  • A. Architecture of End-to-End Compression Framework: The codec’s rounding operation is nondifferentiable, so the framework uses an iterative optimization learning algorithm.This issue arises during backpropagation.

1) Updating the Parameters θ2 of RecCNN:

The RecCNN update introduces an auxiliary decoded compact representation and optimizes RecCNN using the reconstruction objective.

  • 1) Updating the Parameters θ2 of RecCNN:: An auxiliary variable represents the decoded compact representation of the original image.It is introduced according to the network topology before optimizing RecCNN.
  • 1) Updating the Parameters θ2 of RecCNN:: Combining the relevant equations yields the auxiliary-variable update used in the learning procedure.This update follows from combining Eq.(4) and Eq.(3).

2) Updating the Parameters θ1 of ComCNN:

ComCNN parameters are updated through an auxiliary representation and an approximation of the original objective, within an iterative alternating procedure.

  • 2) Updating the Parameters θ1 of ComCNN:: Optimizing ComCNN is difficult because the codec operation is inherently nondifferentiable during backpropagation.The method addresses this difficulty by defining an auxiliary variable.
  • 2) Updating the Parameters θ1 of ComCNN:: The auxiliary decoded representation is used as the optimal input of RecCNN in the ComCNN update.The construction relies on a monotonicity assumption concerning RecCNN.
  • 2) Updating the Parameters θ1 of ComCNN:: Because the codec is an encoder, the method derives a practical solution for the auxiliary-variable optimization.The derivation combines the codec property with the stated monotonicity assumption.
  • 2) Updating the Parameters θ1 of ComCNN:: Equation (13) approximates the original objective and is used to train ComCNN instead of the nondifferentiable formulation.The approximation is obtained under a reasonable assumption and accompanying derivations.
  • 2) Updating the Parameters θ1 of ComCNN:: The optimal θ1 and θ2 are obtained by iteratively optimizing their respective objectives in the proposed algorithm.Algorithm 1 initializes the parameters and alternates training of RecCNN and ComCNN.

C. Loss Functions

The framework uses mean squared error as the loss function for trained parameters and original images.

  • C. Loss Functions: Mean squared error is used as the loss function for training with original images and trained θ2 parameters.The loss is introduced for the subsequent optimization procedure.

1) For ComCNN training:

ComCNN training uses a batch-based optimization setup with trainable parameters θ1.

  • For ComCNN training:: The training formulation uses N as the batch size.
  • For ComCNN training:: θ1 denotes the trainable parameter in the ComCNN training formulation.
  • For ComCNN training:: The passage defines both batch size and trainable parameters for ComCNN training.

2) For RecCNN training:

RecCNN training uses MSE reconstruction loss and represents reconstruction through a learned residual formulation.

  • For RecCNN training:: MSE is used as the loss function for compact representations and original images.
  • For RecCNN training:: θ2 denotes the trainable parameter in the RecCNN loss formulation.
  • For RecCNN training:: The residual form in Eq. (15) is described as essentially identical to Eq. (5).res(·) represents the residual learned by RecCNN.

IV. EXPERIMENTS

The experiments compare the proposed framework with standard codecs and post-processing methods using separate training and testing setups.

  • IV. EXPERIMENTS: The framework is compared against JPEG, JPEG 2000, and BPG with state-of-the-art deblocking or denoising methods.
  • IV. EXPERIMENTS: Training uses 400 images of size 180 × 180, while testing uses 7 images excluded from the training dataset.
  • IV. EXPERIMENTS: A total of 204800 patches are sampled from training images and their flip and rotation augmentations.Each image contributes 8 augmentations and 64 patches of size 40×40.

B. Model Initialization

The initialized framework is evaluated at matched bit rates across JPEG, JPEG 2000, BPG, and multiple benchmark datasets, showing gains over comparison methods.

  • C. Experimental Results: At JPEG QF = 5, the framework gains 1.20dB PSNR and 0.0227 SSIM over Zhang’s.Zhang’s is identified as state-of-the-art among the compared methods.
  • C. Experimental Results: Removing either ComCNN or RecCNN produces worse performance than using both networks collaboratively.The ablation compares RecCNN alone with ComCNN alone plus bicubic interpolation.
  • C. Experimental Results: Across 0.1–0.4bpp, the framework gains average PSNR values of 3.06dB, 2.45dB, 1.34dB, and 1.09dB over JPEG2000.The corresponding average SSIM gains are 0.1047, 0.0709, 0.0525, and 0.0435.
  • C. Experimental Results: Against BPG, the framework gains average 0.99dB PSNR and 0.0218 SSIM while saving 5.22% bit-rates.The paper notes that improvements over BPG are less obvious because BPG is already a strong compression method.
  • C. Experimental Results: On Set5, Set14, LIVE1, and General-100, the proposed compression performance exceeds JPEG and JPEG2000 by a larger margin.

D. Running Time

The proposed framework is substantially faster than other post-processing methods, with measured CPU and GPU runtimes for 256 × 256 grayscale images.

  • Runtime comparison: The proposed compression framework runs faster than other post-processing methods.GPU runtimes could not be tested for all competing methods.
  • Overall outcome: The framework’s conclusion reports state-of-the-art performance together with substantially faster processing than most post-processing algorithms.This summarizes the paper’s reported quality and efficiency outcome without adding a new benchmark.
Loading 1708.00838v1…