PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint Global and Local Hierarchical Priors for Learned Image Compression

Jun-Hyuk Kim, Byeongho Heo, Jong-Seok Lee

arXiv:2112.04487v2eess.IVcs.CV

TL;DR

CNN-based entropy models have limitations in capturing long-range dependencies and content-dependent nearby information. Informer combines global and local hyperpriors using attention, improving rate–distortion performance on Kodak and Tecnick over the baseline and global reference model while avoiding the latter’s quadratic complexity.

  • Problem

    CNN-based entropy models have limited global information from local receptive fields and process nearby elements together regardless of differing contents, limiting dependency modeling.

    Source: §1 · pp. 1–2

  • Method

    Informer uses attention with a global hyperprior extracted through cross-attention and a local hyperprior specialized for local information.

    Source: §1 · p. 2

  • Results

    Informer improves rate–distortion performance over the baseline entropy model [36] on Kodak and Tecnick, and outperforms global reference model [40] without its quadratic complexity.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Informer models global and local dependencies in a content-dependent manner, and the conclusion reports improved rate–distortion performance with computational efficiency.

    Source: §5 · p. 8

    Informer’s autoregressive decoding cannot be parallelized because it uses only previously decoded elements.

    Source: §6 · p. 8

Abstract

from arXiv · show

Recently, learned image compression methods have outperformed traditional hand-crafted ones including BPG. One of the keys to this success is learned entropy models that estimate the probability distribution of the quantized latent representation. Like other vision tasks, most recent learned entropy models are based on convolutional neural networks (CNNs). However, CNNs have a limitation in modeling long-range dependencies due to their nature of local connectivity, which can be a significant bottleneck in image compression where reducing spatial redundancy is a key point. To overcome this issue, we propose a novel entropy model called Information Transformer (Informer) that exploits both global and local information in a content-dependent manner using an attention mechanism. Our experiments show that Informer improves rate--distortion performance over the state-of-the-art methods on the Kodak and Tecnick datasets without the quadratic computational complexity problem. Our source code is available at https://github.com/naver-ai/informer.

1. Introduction

Learned entropy models estimate latent probabilities for coding, but CNN priors struggle with long-range dependencies and content-dependent selection of nearby information. Informer combines global and local hyperpriors using attention, improving rate–distortion performance on Kodak and Tecnick while avoiding the global reference model’s quadratic complexity.

1. Introduction

  • Learned image compression transforms an image into a quantized latent representation, then entropy-codes that representation into a bitstream.An entropy model supplies the prior probability model required for entropy coding.Source: §1 · p. 1
  • An entropy model estimates the joint distribution of quantized latents, and its accuracy matters because the bitstream length depends on it.Assuming complete independence limits compression efficiency because that assumption does not hold in most practical cases.Source: §1 · p. 1
  • CNN-based entropy models have limited global information from local receptive fields and cannot exclude nearby elements with different contents.These limitations affect modeling long-range dependencies and can combine distinct nearby content in the same receptive field.Source: §1 · pp. 1–2
  • Informer uses attention to capture global and local dependencies in a content-dependent manner, with separate global and local hyperpriors.The global hyperprior uses cross-attention; the local hyperprior uses 1×1 convolutional layers and prevents the global hyperprior from relying only on local information.Source: §1 · p. 2
  • Informer improves rate–distortion performance over the baseline entropy model [36] on Kodak and Tecnick, and outperforms the global reference model [40] without its quadratic complexity.The global reference model [40] targets global dependencies.Source: §1 · p. 2

2. Related work

Prior learned image-compression work developed nonlinear transforms, while attention research established tools for modeling long-range information across language and vision tasks.

2. Related work

  • Learned nonlinear transforms use deep networks, including GDN-based CNNs and approaches with residual networks, attention, invertible networks, or multi-scale back-projection.Traditional image codecs mostly assume linear transforms because hand-engineering nonlinear transforms for high-dimensional images is difficult.Source: §2 · p. 2
  • Attention handles global information in deep networks and has been applied across vision tasks, including classification, detection, segmentation, and image quality assessment.The Transformer architecture demonstrated attention performance in language tasks.Source: §2 · p. 2
  • Transformer can model long-range dependencies regardless of their distance in pixel space, motivating its use in learned entropy models.The passage contrasts this capability with existing learned entropy models.Source: §2 · p. 2

3. Joint global and local hyperpriors

The section develops Informer’s entropy model by combining global and local hyperpriors with context information to estimate the distribution of quantized latents. Its design uses attention for global dependencies and spatially preserving convolutions for local dependencies.

3.1. Learned image compression

  • The rate–distortion objective combines expected bitstream length, measured by cross-entropy under the learned entropy model, with expected reconstruction distortion.The Lagrange multiplier λ controls the rate–distortion trade-off; quantization is approximated with additive uniform noise for end-to-end training.Source: §3.1 · pp. 2–3

3.2. Learned entropy models

  • Conditional entropy models adapt Gaussian distribution parameters to capture dependencies among quantized latents, while context and hyperprior information provide complementary signals.The model uses Gaussian distributions convolved with a unit uniform distribution; context uses previously decoded adjacent elements, and hyperpriors provide additional encoded information.Source: §3.2 · p. 3

3.3. Decomposition of hyperpriors

  • Informer splits the hyperprior into global and local components that complement each other: attention models content-dependent global dependencies, while the local component models inter-channel dependencies at each spatial location.The global component has no spatial information; the local component preserves spatial resolution while reducing channels.Source: §3.3 · p. 4

3.4. Modeling of hyperpriors

  • The Global Hyper Encoder uses eight learned, fixed-size global tokens as attention queries over the latent representation, producing a fixed-size global hyperprior.The Global Hyper Decoder maps the quantized global hyperprior to features for subsequent modeling of inter-channel dependencies.Source: §3.4 · p. 4
  • The Local Hyper Encoder uses 1×1 convolutions to extract spatially preserved local hyperprior features for each location.The Local Hyper Decoder produces local features that are combined with context and global features to predict mean and scale parameters.Source: §3.4 · p. 4, §3.5 · p. 4

3.5. Prediction of distribution parameters

  • The Parameter Model combines context and global features with multi-head attention, then concatenates local features to predict the Gaussian mean and scale.It uses a following MLP and three 1×1 convolutional layers to generate the distribution parameters.Source: §3.5 · pp. 4–5

4. Experiments

The experiments compare Informer with other entropy models on rate–distortion performance and computational complexity, then analyze its hyperpriors, attention design, and global-context alternative.

4. Experiments

  • Experiments use 12 trained models and evaluate rate–distortion performance on Kodak and Tecnick, with MSE or MS-SSIM distortion metrics.Training uses Triplet images, and evaluation uses bpp with PSNR or MS-SSIM, depending on the distortion metric.Source: §4 · p. 5

4.1. Performance comparison

  • On Kodak and Tecnick, MSE Informer saves 11.84%–19.27% over BPG; on Kodak it beats Global Reference by 1.74%–2.60%, while MS-SSIM savings reach 54.32%.Figure 6 compares rate savings relative to BPG across quality levels; the reported comparisons cover both MSE- and MS-SSIM-optimized models.Source: Figure 6 · p. 6, §4.1 · p. 6
  • Informer’s computational complexity scales near-linearly with image resolution, whereas Global Reference’s GFLOPs increase quadratically and are 10 times larger at 4096×2304 than at 1920×1080.The paper attributes Informer’s scaling to cross-attention with a fixed number of queries.Source: Table 1 · p. 6, §4.1 · pp. 6–7

4.2. Model analysis

  • The full model performs best across both tested bpp regions; the global hyperprior improves on context alone, while the local hyperprior helps more when paired with the global one.The decomposition analysis also finds that adding a global hyperprior to the existing hyperprior can match or slightly underperform the baseline in the higher-bpp region, which the authors conjecture reflects overlapping roles and extra bit allocation.Source: Figure 8 · p. 7, §4.2 · p. 7
  • The proposed combining method outperforms the variant that updates the local hyperprior; four tokens work best at lower bpp, while eight work best at higher bpp.The authors suggest that more tokens capture detailed remaining dependencies in the higher-bpp region.Source: Figure 9 · p. 7, §4.2 · pp. 7–8, Figure 10 · p. 7

4.3. Global hyperprior vs. global context prior

  • Informer outperforms the Global Context Model, which uses masked self-attention to update context from previously decoded elements.The authors argue that self-attention alone is not a silver bullet for image compression and requires careful design.Source: Table 2 · p. 8, §4.3 · p. 8

5. Conclusion

Informer adds global and local hyperpriors to model remaining latent dependencies. The paper reports improved rate–distortion performance with computational efficiency, using attention to combine global and local information content-dependently.

5. Conclusion

  • Informer introduces separate hyperpriors for global and local dependencies in the quantized latent representation.The approach builds on joint autoregressive and hierarchical priors.Source: §5 · p. 8
  • The paper reports that Informer outperforms existing entropy models in rate–distortion performance with computational efficiency.Source: §5 · p. 8
  • Attention lets Informer use global and local information in a content-dependent manner beyond CNN-based localized dependency modeling.Source: §5 · p. 8

6. Limitation

Informer’s autoregressive decoding process cannot be parallelized because it uses only previously decoded elements.

6. Limitation

  • Informer decoding cannot be parallelized because its autoregressive prior uses only previously decoded elements.Source: §6 · p. 8

Related papers

Loading 2112.04487v2…