Source-linked AI summary
Context-adaptive Entropy Model for End-to-end Optimized Image Compression
Jooyoung Lee, Seunghyun Cho, Seung-Kwon Beack
TL;DR
The paper addresses limited entropy-model capacity in end-to-end optimized image compression, where accurate latent-distribution estimation affects rate. It introduces a context-adaptive model using bit-consuming and bit-free contexts to estimate latent means and standard deviations, and reports better PSNR and MS-SSIM results than prior conventional and ANN-based approaches. The authors note remaining limits in Gaussian-based distribution models and computational complexity.
Problem
Entropy-model capacity limits distribution matching and rate estimation in end-to-end optimized image compression.
Method
The method extends entropy models with bit-consuming and bit-free contexts and estimates each latent distribution using both mean and standard-deviation parameters.
Results
The proposed method outperforms previous methods, including BPG and JPEG2000, in PSNR and MS-SSIM.
Takeaways & Limitations
Context adaptivity improves the reported compression performance of ANN-based image compression across conventional and previous ANN-based comparisons.
Takeaways & Limitations
Gaussian-based entropy models have limited expression power, and ANN-based approaches may still be unsatisfactory when their high complexity is considered.
Abstract
from arXiv · showhide
We propose a context-adaptive entropy model for use in end-to-end optimized image compression. Our model exploits two types of contexts, bit-consuming contexts and bit-free contexts, distinguished based upon whether additional bit allocation is required. Based on these contexts, we allow the model to more accurately estimate the distribution of each latent representation with a more generalized form of the approximation models, which accordingly leads to an enhanced compression performance. Based on the experimental results, the proposed method outperforms the traditional image codecs, such as BPG and JPEG2000, as well as other previous artificial-neural-network (ANN) based approaches, in terms of the peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM) index.
1 INTRODUCTION
End-to-end image compression depends heavily on entropy models that estimate latent distributions and minimize rate through close distribution matching. The proposed context-adaptive model uses bit-consuming and bit-free contexts to improve these estimates and reports gains over conventional and ANN-based codecs.
- Motivation: ANN-based image compression has progressed more slowly because image compression presents complicated target problems.Earlier ANN work often focused on reconstructed-image quality and artifact reduction.
- Entropy modeling: Entropy models estimate latent-representation distributions and required encoding bits, while rate minimization reduces approximation mismatch and latent entropy.The rate decomposes into actual entropy plus additional bits caused by mismatch between the true and estimated distributions.
- Entropy modeling: Compression performance depends on entropy-model capacity because rate is minimized when the estimated distribution matches the actual latent distribution.The paper motivates improving model capacity as a route to better compression.
- Contribution: The proposed entropy model exploits bit-consuming and bit-free contexts to estimate each latent distribution more accurately with a generalized approximation model.The contexts are distinguished by whether additional bit allocation is required, and the method aims to reduce spatial dependencies among adjacent latent representations.
- Contribution: The work reports outperforming BPG in PSNR and MS-SSIM and discusses improvements involving model capacity and context level.Its stated comparisons include conventional and ANN-based image-compression approaches.
2 END-TO-END OPTIMIZATION BASED ON CONTEXT-ADAPTIVE ENTROPY
The method extends entropy modeling for quantized latent representations by exploiting spatial dependencies through two context types. It estimates both means and standard deviations, while shared encoder-decoder processing ensures usable contexts during coding.
- Background: Earlier entropy models used non-parametric or Gaussian-mixture forms, while Ballé et al. (2018) input-adaptively estimated latent standard deviations to capture spatial dependencies.The proposed method builds on this entropy-modeling line of work.
- Spatial dependencies: The proposed model also estimates each latent representation’s mean from contexts, in addition to estimating its standard deviation.The paper motivates mean estimation by noting that neighboring latent values can be similar.
- Encoder-decoder structure: The model uses four transform functions for image analysis, image synthesis, redundancy analysis, and context generation.The context-generation transform produces contexts rather than directly estimating standard deviations.
- Training: Training uses noisy representations as entropy-model inputs, while quantized representations provide the conditions to better match training and testing contexts.The paper reports empirically better results when discrete representations are used in the conditions.
- Context-adaptive entropy model: The conditional entropy model uses a Gaussian form with both mean and standard-deviation parameters estimated deterministically from the two context types.The rate term estimates bits using conditional models for the latent representation and side information.
- Context types: Bit-free contexts are extracted from already known latent values, whereas bit-consuming contexts require additional bit allocation.The encoder and decoder can share the known subset because they process latent values in the same order.
- Side information: Side-information latents use a simpler zero-mean Gaussian model because they contribute a very small amount of the total bit-rate.This is an explicit modeling assumption for the auxiliary representation.
3 ENCODER-DECODER MODEL
The encoder transforms, quantizes, and entropy-codes latent representations, while the decoder entropy-decodes them and reconstructs the image using shared trained entropy models. The implementation uses convolutional networks and includes bit-consuming and bit-free context processing.
- The encoder transforms an input image into latent representations, quantizes them, and entropy-codes them using trained entropy models.
- The decoder entropy-decodes the latent representations with the same entropy models and reconstructs the image.
- Bit-consuming context c′i is treated as side information and requires additional bit allocation, whereas bit-free context c′′i is extracted without additional allocation.
- During training, entropy models receive noisy representations, while the encoder and decoder share the parameters of hs and the entropy models.
- The analysis and synthesis transforms and context-related functions are implemented with convolutional neural networks based on Ballé et al. (2018), using transposed convolution for scaling.
4 EXPERIMENTS
The experiments implement and evaluate the context-adaptive entropy model with convolutional components, context-based distribution estimation, and multiple rate–distortion configurations. On Kodak images, the method is compared with traditional and ANN-based codecs using BPP, PSNR, and MS-SSIM, outperforming the comparison methods in both quality metrics.
- 4.1 IMPLEMENTATION: The distribution estimator f receives context information and outputs estimated µi and σi for each latent representation.
- 4.1 IMPLEMENTATION: Contexts are extracted across spatially adjacent elements and known latent elements, with shared context features supporting estimation across channels.
- 4.1 IMPLEMENTATION: The model estimates all M distributions at one spatial position in a single step, shares f across positions, and uses random representatives for the training rate term.The base and hybrid models use 32 and 16 random spatial points per training step, respectively; distortion remains calculated over all images.
- 4.1 IMPLEMENTATION: A lightweight zero-mean Gaussian entropy model is combined with the proposed model for the top four higher-bit-rate λ configurations.
- 4.2 EXPERIMENTAL ENVIRONMENTS: 18 networks are trained and evaluated using both MSE and MS-SSIM distortion terms across nine λ configurations.
- 4.2 EXPERIMENTAL ENVIRONMENTS: Training uses 256×256 patches from 32,420 randomly selected YFCC100m images, while evaluation measures BPP, PSNR, and MS-SSIM on 24 Kodak PNG images.
- 4.3 EXPERIMENTAL RESULTS: The rate–distortion plots show PSNR and MS-SSIM as functions of BPP, with MS-SSIM converted to decibels for quality-level discrimination.
- 4.3 EXPERIMENTAL RESULTS: 34.08%, 11.97%, and 6.85% are the PSNR BD-rate gains over JPEG2000, Ballé et al. (2018), and BPG, respectively.
5 DISCUSSION
The proposed entropy model frames context adaptivity as a framework combining bit-free and bit-consuming contexts to improve latent-distribution estimation. The paper reports strong ANN-based compression results while identifying distribution expressiveness and context level as remaining opportunities.
- The framework combines bit-free and bit-consuming contexts to estimate latent-representation distributions more accurately using generalized entropy models.The authors distinguish contexts by whether additional bit allocation is required and emphasize that the contribution is the framework, not the contexts themselves.
- The proposed method achieved the best reported results in the ANN-based image-compression domain, but further performance improvements remain possible.
- Gaussian-based entropy models have limited expressive power, motivating combinations of context adaptivity with more elaborate non-parametric models.The authors also suggest higher-level contexts and larger network capacity could improve estimation accuracy.
- ANN-based compression may still be unsatisfactory when its high computational complexity is considered against traditional codecs’ accumulated complexity-reducing heuristics.
6 APPENDIX
The appendix describes a hybrid high-bit-rate network that combines the proposed context-adaptive entropy model with a lightweight model, then reports execution-time and representation-training evaluations. It also provides sample comparisons against BPG and Ballé et al. (2018).
- 6.1 HYBRID NETWORK FOR HIGHER BIT-RATE COMPRESSIONS: The hybrid network splits the latent representation channel-wise, applying the proposed model to ˆy1 and a side-information scale-only model to ˆy2.The lightweight model assumes zero-mean Gaussian distributions convolved with a standard uniform distribution.
- 6.1 HYBRID NETWORK FOR HIGHER BIT-RATE COMPRESSIONS: The hybrid objective retains rate and distortion terms, with rate contributions assigned separately to ˆy1, ˆy2, and ˆz.The quantized components are channel-wise concatenated for the distortion term.
- 6.1 HYBRID NETWORK FOR HIGHER BIT-RATE COMPRESSIONS: The hybrid models reduced execution time by 46.83% with N=320 and M=420, and by 57.28% with N=400 and M=600.The measurements were average per-image encoding and decoding times on the Kodak PhotoCD dataset under CPU execution, excluding entropy-coding time.
- 6.2 TEST RESULTS OF THE MODELS TRAINED USING DIFFERENT TYPES OF REPRESENTATIONS: The evaluation compares models trained with discrete versus noisy representations as inputs to the synthesis transforms.The discrete-representation model uses quantized ˆy and ˆz during training, whereas the comparison follows Ballé et al. (2018)’s noisy-representation process.
- 6.2 TEST RESULTS OF THE MODELS TRAINED USING DIFFERENT TYPES OF REPRESENTATIONS: Figure 7 compares execution time between the base and hybrid models, while Figure 8 evaluates models trained with noisy or discrete synthesis-transform inputs.
- 6.3 SUPPLEMENTAL TEST RESULTS: The appendix includes sample reconstructions for MSE-optimized and MS-SSIM-optimized versions of the proposed method alongside BPG and Ballé et al. (2018)’s approach.Figures 9–11 cover MSE optimization, while Figures 12–13 cover MS-SSIM optimization.