Variable Rate Deep Image Compression with Modulated Autoencoder

I. INTRODUCTION

• Training a separate deep image compression model for every rate-distortion tradeoff increases memory requirements.Deep image compression models are jointly optimized for rate and distortion at a particular tradeoff.Source: §I · p. 1
• Bottleneck scaling provides variable bitrate with one shared autoencoder, but performs worse at low bitrates and narrows the effective bitrate range.The approach scales the bottleneck representation before quantization, with the decoder inverting that scaling.Source: §I · p. 1
• A modulation network adapts representations of a shared autoencoder to each target rate-distortion tradeoff.The method formulates variable rate-distortion optimization and jointly trains the autoencoder and modulation network.Source: §I · p. 2
• MAEs achieve operational rate-distortion points almost matching independent models with fewer overall parameters.The parameter set consists of a shared autoencoder plus the modulation network overhead.Source: §I · p. 2

II. BACKGROUND

• Transform coding maps an input image to a representation, quantizes it, and reverses those steps to reconstruct the image.The quantized vector is binarized and serialized into a bitstream, whose statistical redundancy entropy coding exploits.Source: §II · p. 2
• Deep image compression replaces handcrafted transforms with learned convolutional-autoencoder encoders and decoders, optimized for rate and distortion at a fixed tradeoff.The Lagrange multiplier λ controls the tradeoff, and the stated problem is solved by gradient descent and backpropagation.Source: §II · p. 2
• During training, a differentiable proxy replaces quantization, while entropy is used to approximate bitstream length.The entropy estimate is a lower bound on actual bitstream length; the paper uses additive uniform noise as the quantization proxy, estimates entropy with an entropy model, and uses MSE distortion.Source: §II · p. 2
• The multi-rate formulation optimizes rate plus λ-weighted distortion across a set of tradeoffs, assuming uniform importance for those operating points.The formulation can be extended to a continuous range of tradeoffs, and an importance function could weight selected points differently.Source: A. Problem definition · p. 2

B. Bottleneck scaling

• Bottleneck scaling scales the latent representation before quantization and inversely scales it in the decoder for the target tradeoff.The scaling factor s(λ) depends on the rate-distortion tradeoff; other approaches keep the encoder and decoder fixed after optimizing for one tradeoff [1].Source: B. Bottleneck scaling · p. 2
• Scaling only bottleneck features and inversely scaling them in the decoder may limit adaptation across a large range of tradeoffs.The passage also notes that an autoencoder optimized for one tradeoff can have suboptimal parameters at distant tradeoffs.Source: B. Bottleneck scaling · p. 2
• Training the autoencoder and scaling factors separately may also limit bottleneck scaling; the paper proposes MAEs to address these limitations.Source: B. Bottleneck scaling · p. 2

C. Modulated autoencoders

• MAEs modulate encoder and decoder representations at multiple layers according to the target rate-distortion tradeoff.The modulated and demodulated representations are formed using tradeoff-dependent functions.Source: C. Modulated autoencoders · p. 3
• In the experiments, modulation is applied to convolutional-layer outputs in both the encoder and decoder.These outputs are the internal representations selected for modulation.Source: C. Modulated autoencoders · p. 3
• A modulating network and a demodulating network learn the tradeoff-dependent functions for the encoder and decoder.Their parameters are learned alongside the autoencoder parameters.Source: C. Modulated autoencoders · p. 3
• The MAE extends the rate-distortion objective with modulation networks, and all parameters are jointly learned by gradient descent and backpropagation.This enables joint optimization across the rate-distortion tradeoffs of interest.Source: C. Modulated autoencoders · p. 3

D. Modulating and demodulating networks

• A two-layer perceptron maps scalar λ to encoder modulation vectors using ReLU and exponential nonlinearities.The exponential nonlinearity ensures positive outputs, which the authors found beneficial in training.Source: D. Modulating and demodulating networks · p. 3
• A small hidden layer learns a nonlinear mapping between tradeoffs and modulation vectors, enabling more expressive interpolation than simple scaling factors.Tradeoffs are normalized by the maximum λ in the selected set.Source: D. Modulating and demodulating networks · p. 3
• The demodulating network follows an architecture similar to the encoder's modulating network.The encoder network's output is the vector of modulation functions.Source: D. Modulating and demodulating networks · p. 3

A. Experimental setup

• The evaluation uses 585 CLIC 2019 Professional training images and 226 test images, and also tests models on Kodak.The R-D curves include CLIC 2019 Professional and Kodak results.Source: A. Experimental setup · p. 3, Figure 3 · p. 4, Figure 4 · p. 4
• The independent-model baseline trains a separate model for each R-D tradeoff, providing optimal R-D performance at the cost of storing each model.The setup also includes bottleneck scaling, which freezes the autoencoder optimized for the highest tradeoff and learns scaling parameters for other tradeoffs.Source: A. Experimental setup · p. 3
• The R-D curves cover different methods on CLIC 2019 Professional and Kodak datasets.The captions identify the test dataset for each figure.Source: Figure 3 · p. 4, Figure 4 · p. 4

B. Results

• MAEs achieve R-D performance very close to corresponding independent models, while independent models provide the best R-D performance.The results cover CLIC 2019 Professional and Kodak; hyperprior models also have superior R-D performance.Source: B. Results · p. 4
• Shared-autoencoder methods use far fewer parameters than independent models, whose parameter requirements depend on the number of R-D tradeoffs.MAEs and bottleneck scaling incur small overheads for modulation networks or scaling factors.Source: B. Results · p. 4, Table I · p. 4
• MAE feature-map ratios vary across image regions, unlike the uniform ratio expected from bottleneck scaling.The authors state this more complex adaptation allows MAE to allocate bits more freely across tradeoffs, especially at low bitrates.Source: B. Results · p. 4, Figure 5 · p. 4

• MAEs use multi-layer feature modulation and jointly learned autoencoder parameters for variable-rate image compression.Source: §V · p. 4
• A single MAE model keeps performance close to the upper bound of independent models while requiring significantly less memory.Source: §V · p. 4
• MAEs outperform bottleneck scaling, especially at low bitrates.Source: §V · p. 4