PaperlayerView exact revision on arXiv

Source-linked AI summary

Variable Rate Deep Image Compression With a Conditional Autoencoder

Yoojin Choi, Mostafa El-Khamy, Jungwon Lee

arXiv:1909.04802v1eess.IVcs.CV

TL;DR

Learned image compression commonly requires separate networks to cover different rates, making broad and fine-grained adaptation impractical. This paper uses one conditional autoencoder with Lagrange-multiplier and quantization-bin-size controls, and reports better performance than classical codecs plus comparable or sometimes better performance than multi-network learned methods. A reported limitation is that continuous Lagrange-multiplier conditioning did not produce good results in the authors’ trials.

  • Problem

    Separate networks are needed for rate adaptation across a broad, finely resolved rate-distortion curve, which is impractical when networks are large.

  • Method

    A conditional autoencoder uses the Lagrange multiplier for coarse rate control and mixed quantization bin sizes for fine rate adjustment in one model.

  • Results

    The model outperformed BPG in PSNR and MS-SSIM and showed comparable or sometimes better rate-distortion trade-offs than learned methods using multiple networks.

  • Takeaways & Limitations

    One conditional model can provide variable-rate image compression with two rate-control knobs while matching or improving on cited classical and learned baselines.

  • Takeaways & Limitations

    Using continuous Lagrange-multiplier values as conditioning variables did not produce good results in the authors’ trials.

Abstract

from arXiv · show

In this paper, we propose a novel variable-rate learned image compression framework with a conditional autoencoder. Previous learning-based image compression methods mostly require training separate networks for different compression rates so they can yield compressed images of varying quality. In contrast, we train and deploy only one variable-rate image compression network implemented with a conditional autoencoder. We provide two rate control parameters, i.e., the Lagrange multiplier and the quantization bin size, which are given as conditioning variables to the network. Coarse rate adaptation to a target is performed by changing the Lagrange multiplier, while the rate can be further fine-tuned by adjusting the bin size used in quantizing the encoded representation. Our experimental results show that the proposed scheme provides a better rate-distortion trade-off than the traditional variable-rate image compression codecs such as JPEG2000 and BPG. Our model also shows comparable and sometimes better performance than the state-of-the-art learned image compression models that deploy multiple networks trained for varying rates.

1. Introduction

Learned image compression has advanced beyond classical codecs, but conventional rate adaptation often requires separate networks. The paper proposes one conditional autoencoder with two rate-control knobs and reports competitive compression quality.

  • Background: Transform coding underlies practical codecs such as JPEG and JPEG2000 by converting images into latent representations suited to quantization and entropy coding.JPEG uses the discrete cosine transform, whereas JPEG2000 uses a discrete wavelet transform.
  • Background: Deep learned compression uses nonlinear transforms and has reached or exceeded classical codec performance, including BPG, on some PSNR evaluations.PSNR is based on mean squared error, and only a few learning-based schemes had reached BPG performance at the cited point.
  • Background: Autoencoder compression maps images to lower-dimensional latent representations and reconstructs them approximately, while optimizing rate-distortion performance through entropy estimates.Exact latent entropy is generally difficult to calculate and optimize, motivating learned density-estimation models.
  • Problem: Training separate networks for different Lagrange multipliers is impractical across a broad, finely resolved rate-distortion curve.The problem is aggravated when each network is large.
  • Proposed framework: The proposed conditional autoencoder uses the Lagrange multiplier for coarse rate adaptation and mixed quantization bin sizes for fine rate adjustment in one deployable model.The Lagrange multiplier is supplied as a conditioning input, while the quantization bin size is changed during latent quantization.
  • Proposed framework: The model avoids the progressive iterations required by RNN-based variable-rate methods because quality is controlled through conditioning variables.The reported conditioning variables are the Lagrange multiplier and quantization bin size.
  • Evaluation: On Kodak images, the model outperformed BPG on PSNR and MS-SSIM and achieved comparable or sometimes better rate-distortion trade-offs than learned models using multiple networks.Figure 2 reports a 0.3 dB PSNR gain over BPG in one example.

2. Preliminary

This section formulates image compression with an autoencoder, quantized latent representations, and rate-distortion optimization. It then introduces universal quantization as a differentiable training relaxation and motivates conditioning the network on the Lagrange multiplier.

  • Autoencoder formulation: An encoder fφ(x) maps an input image to a quantized latent representation z, which is decoded by gθ(z) and entropy-coded into a compressed bitstream.The quantization bin size is ∆, with z = round∆(fφ(x)); the preliminary setup fixes ∆ = 1.
  • Rate-distortion optimization: The rate-distortion objective minimizes a weighted sum of rate and squared L2 distortion, with λ selecting a specific trade-off point.The Lagrange multiplier λ controls the desired balance between compression rate and reconstructed-image quality.
  • Universal quantization: Universal quantization relaxes nondifferentiable quantization by dithering every encoder output element with one common uniform random variable.It uses z = round∆(fφ(x) + u) − u, where u repeats a uniform variable across dimensions.
  • Universal quantization: Universal quantization provides higher PSNR than additive uniform noise in experiments on 24 Kodak images.The comparison concerns networks trained with the two different quantization relaxations.
  • Differentiable rate-distortion cost: The relaxed rate and distortion use a tractable differentiable density qθ(z) to upper-bound the entropy-based rate estimate.The bound becomes tight when pφ(z) = qθ(z).
  • Conditional modeling: Conditional convolution lets network outputs depend on the input Lagrange multiplier λ, supporting multiple optimization objectives within one conditional network.This conditioning mechanism is used instead of training separate non-conditional networks for each objective.

3. Variable rate image compression

The paper develops a single conditional autoencoder for variable-rate compression, using the Lagrange multiplier for coarse adaptation and quantization bin size for continuous fine-tuning.

  • Motivation: Training and deploying multiple networks for different rate-distortion trade-offs is impractical, so the authors develop one deployable variable-rate model.The challenge is especially pronounced when covering a broad rate-distortion range with fine resolution.
  • Conditional autoencoder: The conditional autoencoder takes λ with the input image and produces rate-distortion behavior conditioned on that value.Conditional convolutions use λ-dependent channel scaling and additive bias terms.
  • Conditional autoencoder: The model optimizes multiple objectives with one conditional network instead of separate non-conditional networks for each objective.Its conditioning variable is the optimization hyper-parameter λ, which trades off quality against rate.
  • Rate adaptation: A finite set Λ of Lagrange multipliers provides discrete rate points, while mixed-bin-size training adds a continuous rate-control knob.The authors note that continuous λ conditioning did not produce good results in their trials.
  • Rate adaptation: Rates increase when λ or Δ decreases; changing λ gives large-scale adaptation, while adjusting Δ gives fine continuous adaptation at fixed λ.The reported mixed-bin-size range is Δ∈[0.5, 2], and λ and Δ are stored with the compressed bitstream for decoding.

4. Refined probabilistic model

The refined model introduces a secondary latent variable and autoregressive entropy models, yielding a hierarchical probabilistic compression structure represented in Figure 6.

  • Model refinement: The refinement introduces a secondary latent variable w that depends on x and z.Compression encodes z from x, then encodes w from z and x.
  • Probabilistic coding: The representations z and w are entropy-coded using qθ(z|w, λ, Δ) and qθ(w|λ, Δ), respectively.During decompression, decoded w helps compute the distribution used to decode z.
  • Model representation: Figure 6 presents a graph representation of the refined variable-rate image compression model.The refinement is described as orthogonal to the rate-adaptation schemes.
  • Autoregressive refinement: Autoregressive models are introduced for qθ(w|λ, Δ) and qθ(z|w, λ, Δ).The autoregressive parameterizations use preceding elements and remain conditioned on the rate-control variables.
  • Interpretation: The refined model can be viewed as a hierarchical autoencoder when its conditioning components are set aside.The paper relates it to prior hierarchical and probabilistic autoencoder models.

5. Experiments

Experiments evaluate the variable-rate model on Kodak images against learned and classical codecs using PSNR, MS-SSIM, visual quality, and rate-control behavior. The model uses conditional convolutions and two rate-control parameters while maintaining performance near fixed-rate networks.

  • Architecture: All convolution and masked-convolution blocks in the experimental architecture employ conditional convolutions.The architecture includes universal quantization, arithmetic encoding and decoding, and generalized divisive normalization blocks.
  • Experimental results: The evaluation compares the variable-rate model with learned compression models and BPG on the Kodak image set using PSNR and MS-SSIM.Five curves correspond to five Lagrange multiplier values, with quantization bin sizes ∆ varied from 0.5 to 2.
  • Experimental results: The model outperforms BPG in both PSNR and MS-SSIM and is comparable overall, sometimes better, than learned models using multiple rate-specific networks.These comparisons are reported for 24 Kodak images.
  • Comparison with RNN-based models: The model uses single-iteration encoding and decoding, keeping complexity fixed across rates, unlike the RNN-based model whose complexity increases with more iterations.The RNN-based model requires multiple iterations at high rates.
  • Fixed-rate comparison: The variable-rate network performs very near to separately trained fixed-rate networks across MSE, MS-SSIM, and combined MSE+MS-SSIM objectives.Figure 9 compares variable-rate and fixed-rate networks trained with these objectives.
  • Rate adaptation: Changing λ and ∆ changes the compressed outputs and the implicit arithmetic-coding bit allocation for latent representations z and w.Figure 10 visualizes code lengths assigned to selected latent channels.

6. Conclusion

The paper proposes one conditional autoencoder for variable-rate image compression instead of separate networks for different rates. Its two input controls support rate adaptation, while experiments report advantages over classical codecs and competitive performance against learned multi-network methods.

  • 6. Conclusion: The framework trains and deploys one variable-rate conditional autoencoder instead of multiple networks for different compression rates.The Lagrange multiplier and quantization bin size are supplied as conditioning inputs.
  • 6. Conclusion: The Lagrange multiplier and quantization bin size provide two knobs for controlling the compression rate.Both parameters are given as inputs to the conditional autoencoder.
  • 6. Conclusion: The proposed scheme performs better than classical codecs such as JPEG2000 and BPG, while matching or sometimes exceeding learned methods that use multiple rate-specific networks.The comparison concerns varying compression rates.
  • 6. Conclusion: The conditional neural-network approach is presented as applicable beyond image compression to optimization problems formulated with Lagrange multipliers.This is stated as a broader possible adoption of the proposed conditional-network idea.

A. Comparison of our refined probabilistic model to [16]

The refined probabilistic model differs from the prior model in its conditioning on λ and ∆ and in additional probabilistic-model components. The comparison is summarized in Table 1.

  • A. Comparison of our refined probabilistic model to [16]: The major difference from [16] is conditioning the model on the Lagrange multiplier λ and quantization bin size ∆.The conditioning design is identified as the principal architectural distinction.
  • A. Comparison of our refined probabilistic model to [16]: The paper also reports differences in the probabilistic model relative to [16], summarized in Table 1.The table highlights those differences in red.

B. More example images

Supplementary examples compare the proposed method with JPEG, JPEG2000, and BPG at matched rates on Kodak images. The reported comparisons favor the proposed method in objective metrics and visual quality.

  • B. More example images: The examples match the variable-rate network’s compression rate to BPG by adjusting λ and ∆.The comparisons include JPEG, JPEG2000, and BPG.
  • B. More example images: At the same bits per pixel, the proposed method outperforms BPG in both PSNR and MS-SSIM across the examples.The passage reports this outcome for the supplementary Kodak examples.
  • B. More example images: The proposed method provides better visual quality with fewer artifacts than the classical codecs in the compared examples.Orange-boxed regions in Figures 11 and 13 are magnified in Figures 12 and 14.
Loading 1909.04802v1…