Source-linked AI summary
Asymmetric Gained Deep Image Compression With Continuous Rate Adaptation
Ze Cui, Jing Wang, Shangyin Gao, Bo Bai, Tiansheng Guo, Yihui Feng
TL;DR
Continuous rate adaptation remains difficult because learned compression methods either require multiple models or incur complexity and performance costs in a single model. AG-VAE uses gain units with exponent interpolation and an asymmetric Gaussian entropy model, achieving comparable quantitative performance with state-of-the-art learned methods and better qualitative performance than classical codecs.
Problem
Continuous rate adaptation remains an open problem because existing learned methods use multiple networks or incur complexity and performance costs in one model.
Method
AG-VAE uses gain units for discrete rate control, exponent interpolation for continuous adaptation without extra training, and an asymmetric Gaussian entropy model for entropy estimation.
Results
AG-VAE achieves comparable quantitative performance with state-of-the-art learned image-compression methods and better qualitative performance than classical image codecs.
Takeaways & Limitations
Gain units provide a single-model approach to continuous rate adaptation while avoiding performance degradation and requiring negligible additional computation.
Abstract
from arXiv · showhide
With the development of deep learning techniques, the combination of deep learning with image compression has drawn lots of attention. Recently, learned image compression methods had exceeded their classical counterparts in terms of rate-distortion performance. However, continuous rate adaptation remains an open question. Some learned image compression methods use multiple networks for multiple rates, while others use one single model at the expense of computational complexity increase and performance degradation. In this paper, we propose a continuously rate adjustable learned image compression framework, Asymmetric Gained Variational Autoencoder (AG-VAE). AG-VAE utilizes a pair of gain units to achieve discrete rate adaptation in one single model with a negligible additional computation. Then, by using exponential interpolation, continuous rate adaptation is achieved without compromising performance. Besides, we propose the asymmetric Gaussian entropy model for more accurate entropy estimation. Exhaustive experiments show that our method achieves comparable quantitative performance with SOTA learned image compression methods and better qualitative performance than classical image codecs. In the ablation study, we confirm the usefulness and superiority of gain units and the asymmetric Gaussian entropy model.
1. INTRODUCTION
AG-VAE targets continuous rate adaptation in a single learned image-compression model, addressing the cost and performance limitations of prior approaches. It combines gain units, exponent interpolation, and an asymmetric Gaussian entropy model to control rate and improve entropy estimation.
- Prior VAE-based methods either train multiple fixed-rate models or use single-model schemes with added complexity, memory requirements, or rate-distortion degradation.
- AG-VAE uses gain units after the encoder to rescale latent representations and control information loss during quantization.
- AG-VAE achieves comparable rate-distortion performance with state-of-the-art learned methods and better qualitative visual quality than classical codecs.
- An inverse-gain unit is introduced to avoid performance degradation, while the asymmetric Gaussian entropy model targets more accurate entropy estimation for asymmetric latent distributions.
- Exponent interpolation generates gain vectors at arbitrary bit rates, extending adaptation from discrete points to a continuous range without extra training.
2. RELATED WORKS
Prior learned image compression work explored VAE-based coding, variable-rate adaptation, and entropy modeling, but existing approaches incurred fixed-rate, complexity, or rate-distortion limitations. Gain units with exponential interpolation were developed to support continuous rate adaptation in VAE-based methods.
- Learned Image Compression: VAE-based learned image compression uses an encoder, quantizer, entropy model, and decoder to transform images into discrete latent representations and reconstruct them.The encoder maps x to y, quantization produces discrete ˆy, and the framework is trained jointly.
- Learned Image Compression: The training objective combines expected code length and reconstruction distortion, with β specifying the rate-distortion tradeoff.Because β is fixed during training, VAE-based methods require multiple models trained at different β values for different compression rates.
- Variable-Rate Compression: Conditional autoencoders adjust compression using Lagrange multipliers and mixed bin sizes, but added fully connected layers increase complexity and bin-size choices affect rate-distortion performance.The approach also creates a dilemma in selecting the best bin-size combination.
- Variable-Rate Compression: Scale-based rate adaptation can degrade rate-distortion performance because scaling parameters are incompatible with the pretrained autoencoder, particularly at low rates.The cited approach first trains at high bit rate, then fixes the autoencoder while adding scale parameters.
- Entropy Estimation Model: Gain units with exponential interpolation were introduced as a generalizable approach for continuous rate adaptation in VAE-based image compression, while Gaussian entropy models estimate latent distributions for rate control.The authors report using gain units and optimization schemes in CLIC2020, and prior work used mean-and-scale Gaussian and fully factorized density models for entropy estimation.
3. PROPOSED METHOD
AG-VAE uses channel-wise gain units and paired inverse-gain units for discrete rate adaptation in one model, then applies exponent interpolation to extend adaptation continuously. It also extends gain-based scaling to the hyperprior and uses an asymmetric Gaussian entropy model for more accurate estimation.
- Gain Unit: AG-VAE introduces gain units that rescale latent channels before quantization, allowing channel-wise control of quantization loss and discrete rate adaptation in one model.Each channel has its own scale value, and the gain matrix is trained jointly with the autoencoder for compatibility.
- Gain Unit: The gain unit performs channel-wise multiplication, while quantization rounds the gained latent representation element-wise.The gained representation is passed through quantization before inverse rescaling.
- Discrete Variable Rate with Gain Units: Paired inverse-gain units map quantized representations back to the original numerical intervals, addressing reconstruction issues caused by quantization.Gain and inverse-gain vectors are paired with predefined Lagrange multipliers in the discrete variable-rate framework.
- Continuous Variable Rate: Exponent interpolation generates gain-vector pairs for arbitrary rates between adjacent discrete operating points without extra training or supplementary blocks.The interpolation coefficient is real-valued and controls the corresponding bit rate; the resulting CVR method extends coverage across the continuous R-D range without degraded performance.
- Variable Rate of Hyperprior: A second gain-unit pair scales the hyperprior representation, reducing its rate consumption without harming performance.The HCVR method adds these units to the hyperprior network within the CVR framework.
- Gaussian Entropy Model: The asymmetric Gaussian entropy model replaces the symmetric model to improve entropy estimation for latent representations with asymmetric distributions.The symmetric model is described as having insufficient degrees of freedom for some natural-image distributions.
4. EXPERIMENTS
Experiments evaluate AG-VAE across rate-distortion performance, variable-rate comparisons, computational overhead, generalization, and ablations of its entropy-modeling components.
- Implementation Details: AG-VAE adjusts rate continuously in one model by selecting discrete gain-vector indices and an interpolation coefficient.The bit rate increases as the index and interpolation coefficient increase.
- Rate-distortion Performance: On Kodak, AG-VAE achieves better PSNR rate-distortion performance than multiple-network learned codecs and better performance than other variable-rate methods.It is comparable to Cheng et al. in MS-SSIM, better than BPG in PSNR, and competitive with VTM.
- Comparison of Variable-Rate Methods: HCVR maintains good rate-distortion performance across the full rate range, while competing methods show degradation or substantially lower performance.Conditional Conv degrades at high rates, bottleneck scaling performs poorly at low rates, and the RNN method performs much worse than the others.
- Additional Computation and Parameters: HCVR’s additional FLOPs are nearly 100 times smaller than Conditional Conv’s, while its additional parameters are nearly seven times smaller.The comparison uses HCVR, bottleneck scaling, and Conditional Conv against fixed-rate models.
- Generalizability of Gain Unit: Gain units generalize to multiple VAE-based image-compression architectures and support continuous rate adaptation through exponent interpolation.The evaluated backbones include Ballé et al. and Minnen et al.
- Ablation Study: HCVR produces slightly better rate-distortion performance than CVR across the whole bit-rate range by flexibly adjusting hyperprior rate.The comparison uses AG-VAE and the Minnen et al. architecture as basic frameworks.
- Ablation Study: AG-VAE outperforms SG-VAE on both PSNR and MS-SSIM after replacing the symmetric Gaussian entropy model with the asymmetric model.The comparison focuses on rate-distortion curves from 0.4 to 0.6 bpp.
5. CONCLUSION
The paper concludes that AG-VAE provides continuously variable-rate compression with competitive quality and better qualitative results than classical codecs. Gain units enable this adaptation efficiently, while the asymmetric Gaussian model improves entropy estimation and can extend to other learned compression methods.
- Conclusion: AG-VAE achieves comparable quantitative performance with state-of-the-art learned image-compression methods and better qualitative performance than classical codecs.These conclusions are reported for the proposed continuously variable-rate framework.
- Conclusion: Gain units and exponent interpolation provide discrete and continuous rate adaptation without extra training or modules.The conclusion attributes the approach to uneven channel redundancy and reports effectiveness and efficiency experimentally.
- Conclusion: The asymmetric Gaussian entropy model enables flexible entropy estimation for raw images and can extend to other learned image-compression methods.The paper also reports gain units as a state-of-the-art rate-adaptation solution for VAE-based methods.