PaperlayerView exact revision on arXiv

Source-linked AI summary

Nonlinear Transform Coding

Johannes Ballé, Philip A. Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agustsson, Sung Jin Hwang, George Toderici

arXiv:2007.03034v2cs.ITeess.IV

TL;DR

Nonlinear transform coding addresses the difficulty of using flexible nonlinear transforms for lossy compression, especially when sources are far from Gaussian. The paper reviews NTC methods, introduces VECVQ and a simplified rate–distortion parameterization, and finds that NTC can match optimal or near-optimal quantization on simple sources while supporting learned entropy models and perceptual distortion objectives.

  • Problem

    Nonlinear transforms can improve independence and source adaptation, but designing them for high-dimensional compression was difficult and empirical comparisons with vector-quantization optima were limited.

  • Method

    The paper reviews ANN-based NTC transforms, stochastic optimization, learned entropy models, and rate–distortion parameterizations, while introducing VECVQ as a stochastic variational entropy-constrained quantizer.

  • Results

    NTC matches VECVQ on the banana source, recovers the optimal entropy-constrained scalar quantizer on the Laplace source with an optimized offset, and can gain further rate–distortion improvements from learned entropy models.

  • Takeaways & Limitations

    NTC provides flexible source-adaptive quantization and can be optimized for differentiable distortion measures, including perceptual objectives such as MS-SSIM.

Abstract

from arXiv · show

We review a class of methods that can be collected under the name nonlinear transform coding (NTC), which over the past few years have become competitive with the best linear transform codecs for images, and have superseded them in terms of rate--distortion performance under established perceptual quality metrics such as MS-SSIM. We assess the empirical rate--distortion performance of NTC with the help of simple example sources, for which the optimal performance of a vector quantizer is easier to estimate than with natural data sources. To this end, we introduce a novel variant of entropy-constrained vector quantization. We provide an analysis of various forms of stochastic optimization techniques for NTC models; review architectures of transforms based on artificial neural networks, as well as learned entropy models; and provide a direct comparison of a number of methods to parameterize the rate--distortion trade-off of nonlinear transforms, introducing a simplified one.

I. INTRODUCTION

Transform coding separates source decorrelation from quantization and coding, but linear transforms impose lattice-like constraints on non-Gaussian sources. This paper reviews nonlinear transform coding, its optimization tools, architectures, entropy models, and rate–distortion parameterizations.

  • I. INTRODUCTION: Transform coding reduces compression complexity by mapping source vectors into a latent space before quantization and entropy coding.General vector quantization becomes computationally infeasible as dimensionality grows because codebooks and searches scale exponentially.
  • I. INTRODUCTION: Linear transform coding is well matched to Gaussian sources but can underadapt when the source distribution is strongly non-Gaussian.For the banana-shaped distribution, linear coding is limited to lattice quantization, whereas nonlinear coding can adapt more closely to the source.
  • I. INTRODUCTION: Artificial neural networks combined with stochastic optimization provide practical tools for learning nonlinear transforms for compression.These methods have become competitive with modern conventional image-compression methods such as HEVC.
  • I. INTRODUCTION: The paper evaluates NTC on Laplace and banana-shaped sources, introduces variational entropy-constrained vector quantization, and compares optimization, architecture, entropy-modeling, and rate–distortion parameterization methods.The review focuses on both capabilities and challenges of NTC.

II. STOCHASTIC RATE–DISTORTION OPTIMIZATION

The paper formulates lossy compression as a rate–distortion optimization over codebooks, entropy models, and reconstruction distortion. It illustrates the formulation with scalar and vector quantizers for Laplacian and banana-shaped sources.

  • II. STOCHASTIC RATE–DISTORTION OPTIMIZATION: Lossy compression maps each source vector to a codebook vector, then entropy-codes the resulting index into a bit sequence.The codebook contains reproduction vectors, and the encoder selects an index for each source vector.
  • II. STOCHASTIC RATE–DISTORTION OPTIMIZATION: The objective jointly minimizes index cross entropy and reconstruction distortion under an entropy model.The distortion is measured by d, while the entropy-coded index determines the rate.
  • II. STOCHASTIC RATE–DISTORTION OPTIMIZATION: The Lagrange multiplier λ in the relaxed objective determines the rate–distortion trade-off.This replaces a hard distortion constraint with a weighted distortion term.
  • II. STOCHASTIC RATE–DISTORTION OPTIMIZATION: The examples use squared-error quantization for a one-dimensional Laplacian source and a two-dimensional banana-shaped source.The Laplacian example displays source likelihood, codebook vectors, entropy-model probabilities, and quantization bins.

A. Variational entropy-constrained vector quantization

The paper introduces VECVQ, a stochastic-gradient variant of entropy-constrained vector quantization that jointly fits the entropy model and codebook through a variational objective. On the Laplace source, it recovers the known operational optimum and supplies an empirical benchmark for NTC.

  • A. Variational entropy-constrained vector quantization: VECVQ generates quantizers by directly minimizing the entropy-constrained vector-quantization objective with stochastic gradient descent.The algorithm parameterizes entropy-model and codebook variables and optimizes them from sampled source vectors.
  • A. Variational entropy-constrained vector quantization: The sample-gradient estimator is unbiased when derivatives are averaged over a batch of source samples.The encoder-selected code index determines each sample’s contribution to the loss.
  • A. Variational entropy-constrained vector quantization: VECVQ fits the entropy model to the marginal index distribution while adjusting codebook vectors to reduce distortion.The loss decomposes into a KL-divergence term for the entropy model and a distortion term for the codebook.
  • A. Variational entropy-constrained vector quantization: Because the entropy model can represent arbitrary distributions, the variational approximation can recover the true marginal as its KL divergence approaches zero.This makes VECVQ an empirical lower-bound benchmark for more constrained compression methods such as NTC.
  • A. Variational entropy-constrained vector quantization: VECVQ recovers the theoretical optimum of Sullivan’s entropy-constrained scalar quantizer on the Laplace source.The result supports using VECVQ as an empirical reference when no theoretical optimum is available for a more complex source.

III. NONLINEAR TRANSFORM CODING

NTC maps source vectors into a latent space, applies uniform scalar quantization, and maps quantized latents back through a learned synthesis transform. Compared with linear coding, its flexible transforms adapt quantization geometry to the source and can locally orthogonalize its density.

  • III. NONLINEAR TRANSFORM CODING: NTC closely matches VECVQ on the banana source, while the straight-through variant diverges at low rates and linear transform coding performs significantly worse.For the Laplace source, optimized-offset NTC and VECVQ recover the optimal entropy-constrained scalar quantizer; randomized-offset NTC is slightly suboptimal at lower rates.
  • III. NONLINEAR TRANSFORM CODING: NTC uses learned analysis and synthesis transforms to move between source and latent spaces, where quantization and compression occur.The analysis transform is y = ga(x), while reconstruction uses x̃ = gs(ỹ).
  • III. NONLINEAR TRANSFORM CODING: Uniform scalar quantization in the latent space produces an effective source-space quantizer whose bins and codevectors are determined separately by the two transforms.The analysis transform determines bin boundaries, while the synthesis transform determines codevector locations.
  • III. NONLINEAR TRANSFORM CODING: NTC generalizes companding because nonlinear transforms implement non-uniform source-space quantization while retaining a fixed latent quantization step.The transforms need not be exact inverses, allowing codevectors to lie anywhere within, or technically outside, corresponding bins.
  • III. NONLINEAR TRANSFORM CODING: NTC adapts quantization bins to high-probability regions, shifting codevectors toward them at the expense of accuracy in low-probability regions.Some low-probability codevectors in the nonlinear example lie outside their respective bins.
  • III. NONLINEAR TRANSFORM CODING: On the banana source, learned transforms are globally nonlinear but locally approximately orthogonalized in high-probability regions.This appears through approximate orthogonality of inverse-analysis and synthesis Jacobian columns.

A. Optimization and proxy rate–distortion loss

The paper analyzes stochastic optimization and proxy objectives for NTC, emphasizing how quantization offsets, dithering, and gradient approximations affect learned transforms. It also reviews soft quantization as a way to control proxy-loss behavior and reports instabilities of straight-through training at low rates.

  • Optimization and proxy objectives: Dithered quantization makes the NTC objective differentiable with respect to analysis-transform parameters by replacing quantization with additive uniform noise.The continuous entropy model fits the marginal distribution of noisy latents, while the distortion term follows from the equivalence between dithered quantization and additive uniform noise.
  • Optimization and proxy objectives: The dithered objective is not necessarily optimal for compression, so the quantization offset must be selected separately after proxy-loss optimization.Offsets can be searched explicitly or chosen heuristically to align a quantization-bin center with the entropy-model mode or median.
  • Optimization and proxy objectives: Optimizing the dithering proxy loss curves the transforms around the central quantization bin, producing an effective quantizer near the optimum despite a suboptimal offset.The analysis transform becomes flatter and the synthesis transform steeper around the mode, while other bins may not receive enough loss weight to develop the same behavior.
  • Optimization and proxy objectives: Explicit soft quantization transfers the required curvature from the ANNs into a temperature-controlled quantization function.This yields smoother ANN transforms and trades proxy-loss bias against gradient variance while retaining near-optimal performance across rates.
  • Optimization and proxy objectives: Straight-through gradient estimates perform reasonably at higher rates but can destabilize low-rate training, with oscillating analysis transforms creating effectively discontiguous bins.The instability is observed for both Laplace and banana distributions.

B. Nonlinear transforms

The paper reviews neural-network architectures for nonlinear transforms, including capacity, structural constraints, and generalized divisive normalization. Architectural benefits become most visible when networks approach their approximation limits, particularly at higher rates.

  • Nonlinear transform architectures: More complex sources and higher-rate operating points generally require transforms with greater complexity and neural networks with more parameters.The relevant capacity depends on the source distribution and rate–distortion setting.
  • Nonlinear transform architectures: NTC models for images, video, and audio commonly impose convolutionality, downsampling, and upsampling to reduce complexity and improve data efficiency.These choices give the transforms characteristics resembling multi-scale filterbanks.
  • Nonlinear transform architectures: GDN provides a spatially local normalization nonlinearity used to improve the trade-off between network size and rate–distortion performance in image compression.A simplified weighted ℓ1-like form can avoid costly exponentiations with negligible rate–distortion loss.
  • Nonlinear transform architectures: The expected high-rate collapse of transforms to identity functions is not observed for image-compression models at practically interesting rate–distortion trade-offs.The identity-function behavior is suggested only for extremely high rates.
  • Nonlinear transform architectures: At low rates, GDN and pointwise nonlinearities converge in performance because all networks have sufficient approximation capacity.At higher rates, smaller networks saturate earlier and GDN's performance benefit becomes measurable.

IV. LEARNED ENTROPY MODELS

Learned entropy models improve nonlinear transform coding by modeling dependencies among quantized latents, using forward and backward adaptation. These models yield substantial image-compression gains, though backward adaptation reduces computational parallelism and conditional models require deterministic decoding.

  • Backward adaptation: Backward adaptation conditions each latent probability on previously decoded dimensions without extra side information, but requires interleaving probability computation with decoding.Spatial autoregressive backward adaptation combined with forward adaptation produces further rate–distortion gains and outperforms BPG.
  • Learned adaptation: Conditional entropy models improve rate–distortion performance over factorized models by exploiting dependencies among quantized latent representations.Forward adaptation uses side information, while backward adaptation conditions probabilities on previously decoded latent dimensions.
  • Forward adaptation: Forward adaptation conditions the entropy model for ˆy on decoded side information ˆz, whose small entropy can be outweighed by the improved fit.The side information is produced by processing the analysis transform’s latent representation with an additional transform.
  • Practical considerations: Backward adaptation impedes computational parallelism because conditional-probability computation must alternate with entropy decoding.Channel-slice models were introduced as a more GPU-parallelizable alternative to spatial-location processing.
  • Practical considerations: Conditional entropy models can suffer catastrophic decoding failures when floating-point round-off creates probability discrepancies between sender and receiver.Practical deployment therefore requires numerical determinism across hardware platforms.
  • Image-compression performance: On the Kodak test set, sophisticated entropy modeling compares favorably with BPG in PSNR, while MS-SSIM optimization substantially outperforms BPG.The caption attributes the perceptual advantage to nonlinear transforms and the properties captured by MS-SSIM.

V. RD TRAVERSAL WITH λ-PARAMETERIZATION

The paper studies using λ-conditioned transforms to traverse rate–distortion trade-offs with one nonlinear-transform model. First-order spline parameterizations are easier to optimize than ANN parameterizations, while layer-wise conditioning approaches separate-model performance more closely than latent scaling alone.

  • Motivation: λ in the rate–distortion Lagrangian determines the trade-off between rate and distortion, motivating one transform family that spans multiple operating points.This follows the quantization-step parameterization used in many linear transform coders.
  • Parameterization: λ-conditioned ANN layers use affine transformations whose scale and bias are functions of λ, allowing transforms or entropy models to vary with the trade-off.The layer operation is w = h_f(λ) ⊙ v + h_b(λ).
  • Parameterization comparisons: ANN parameterizations perform consistently worse than first-order splines despite using more parameters, suggesting numerically harder optimization.The comparison is reported for the λ-parameterization evaluated in figure 13.
  • Parameterization comparisons: λ-parameterizing the entropy model is not crucial to rate–distortion performance and may not justify its implementation complexity.The entropy model also needs integer-arithmetic implementation for cross-platform stability.
  • Parameterization comparisons: Latent scaling or affine transformation alone does not match separate-model performance, whereas layer-wise parameterization comes close.The differences between parameterizations become more evident at high rates.
  • Parameterization comparisons: GDN reparameterization gives the best rate–distortion performance among the compared approaches, at the cost of slightly more model parameters.Its parameterization uses both length-B vectors and a B × B matrix.

VI. RELATED WORK

Related work traces NTC’s convergence with autoencoders, the growth of learned image and video compression, and extensions to other data types and tasks. It also covers architectural innovations, learned entropy models, perceptual objectives, and methods for traversing rate–distortion trade-offs.

  • Origins and scope: NTC emerged from a recent convergence between transform coding and autoencoders, alongside renewed interest in data-driven compression.The two areas had previously developed largely in separate communities.
  • Image compression: Early ANN-based image-compression work progressed from systems without entropy modeling to jointly optimized transforms and continuous entropy models.Other early systems used context-based adaptive entropy coding that was not jointly optimized with the transforms.
  • Architectures: Image NTC architectures incorporate multi-scale structure, attention mechanisms, and iterative processing to exploit properties of image distributions.These design choices extend beyond convolutional filtering, sampling, and specialized nonlinearities.
  • Applications: NTC has expanded beyond images to video, point clouds, volumetric data, ANN features, and ANN-parameter compression.Video work explores spatiotemporal convolutions, optical flow networks, and multi-scale linear filtering.
  • Entropy models: Learned entropy-model research includes forward adaptation, backward adaptation, and integer architectures for reliable decoding across hardware platforms.These methods address both probabilistic modeling and deployment concerns.
  • Rate–distortion traversal: Rate–distortion traversal work progressed from varying quantization step size to layer-wise parameterization, with this paper simplifying the parameterization and adapting it to GDN.The earlier quantization-step approach most closely corresponds to latent affine parameterization.
  • Perceptual objectives and extensions: Perceptual NTC research studies MS-SSIM, other perceptual distortion measures, and optimization methods for non-differentiable metrics.Related work also connects learned compression with content analysis, inpainting, super-resolution, quality enhancement, and encryption.

VII. CONCLUSION

The conclusion presents NTC as a flexible compression framework built on ANNs and stochastic rate–distortion optimization. It emphasizes adaptability to diverse sources and distortion measures, alongside new VECVQ and λ-parameterization contributions.

  • NTC combines ANN-based universal function approximation with stochastic optimization of the rate–distortion Lagrangian.
  • VECVQ is introduced as a stochastically optimized version of entropy-constrained vector quantization.
  • The paper introduces a novel λ-parameterization and directly compares different λ-parameterizations for image compression models.
  • NTC models can adapt to domain-specific imagery and new multimedia modalities through end-to-end optimization of many system parameters.The passage gives medical and astronomical imagery as examples and contrasts weeks of prototyping with years.
  • NTC can be optimized directly for differentiable distortion measures, including MS-SSIM and squared error.The cited comparison concerns models optimized for MS-SSIM versus squared error, with hybrid adversarial losses given as another example.
  • The authors believe increasing parallelized computation will fundamentally change practical data compression.

A. Local properties of nonlinear transforms

At high rates, nonlinear transform coding is locally governed by orthogonality and near-cubical quantization cells, while realistic rates couple cell geometry with entropy modeling. These conclusions extend locally to lower-dimensional manifolds and differentiable distortion measures under stated assumptions.

  • At high rates, an optimal nonlinear analysis transform has an inverse Jacobian whose columns tend toward orthogonality.The analysis assumes equal source and latent dimensions, local invertibility, and squared-error distortion.
  • Locally, quantization cells are parallelepipeds obtained by mapping a latent hypercube through the inverse analysis Jacobian.Their edge vectors are the inverse Jacobian’s columns, and their volume is det(∇g_a(x_k)^−1).
  • High-rate orthogonal transforms correspond to hypercubical quantization cells of uniform size and are near-optimal.This follows because the distortion term dominates the rate term as λ becomes large.
  • At realistic bit rates, entropy-model factorization couples the rate and distortion terms, allowing noncubical cells and nonuniform codeword density.As λ changes, cells evolve from hypercubes at high rates toward other shapes at lower rates, while local Jacobians remain biased toward orthogonality.
  • When latent dimension is not below the source’s essential reproduction dimension, the same local orthogonality argument applies on the source manifold.The reduced transform is formed by removing latent variables that consistently quantize to zero.
  • For twice-differentiable distortion losses, high-rate conclusions hold after a local coordinate transformation using the square root of the distortion Hessian.When the Hessian varies over space, nonlinear transforms can build the perceptual measure directly into the optimal transform.
Loading 2007.03034v2…