PaperlayerView exact revision on arXiv

Source-linked AI summary

ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding

Dailan He, Ziming Yang, Weikun Peng, Rui Ma, Hongwei Qin, Yan Wang

arXiv:2203.10886v2cs.CVeess.IV

TL;DR

Making learned image compression practical requires assessing both compression performance and running speed, especially decoding speed. ELIC combines uneven channel grouping with spatial context and an efficient transform, and is reported to surpass VVC on PSNR and MSSSIM while achieving state-of-the-art coding performance and running speed.

  • Problem

    Learned image compression architectures need thorough assessment of compression performance and running speed, especially decoding speed, to support practical applications.

    Source: §1 · pp. 1–2

  • Method

    ELIC combines information-compaction-guided uneven channel grouping with spatial context in SCCTX and uses residual-transform networks and a thumbnail network for preview decoding.

    Source: §1 · p. 2, §7 · p. 8

  • Results

    ELIC surpasses VVC on both PSNR and MSSSIM and is reported to achieve state-of-the-art coding performance and running speed.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ELIC better balances compression ability and running speed, while its thumbnail network improves preview-decoding utility.

    Source: §7 · p. 8

    The VVC comparison is scoped by VVC's design for YUV 4:2:0 rather than YUV 4:4:4, and the authors plan to investigate objective and subjective image quality.

    Source: §7 · p. 9

Abstract

from arXiv · show

Recently, learned image compression techniques have achieved remarkable performance, even surpassing the best manually designed lossy image coders. They are promising to be large-scale adopted. For the sake of practicality, a thorough investigation of the architecture design of learned image compression, regarding both compression performance and running speed, is essential. In this paper, we first propose uneven channel-conditional adaptive coding, motivated by the observation of energy compaction in learned image compression. Combining the proposed uneven grouping model with existing context models, we obtain a spatial-channel contextual adaptive model to improve the coding performance without damage to running speed. Then we study the structure of the main transform and propose an efficient model, ELIC, to achieve state-of-the-art speed and compression ability. With superior performance, the proposed model also supports extremely fast preview decoding and progressive decoding, which makes the coming application of learning-based image compression more promising.

1. Introduction · 2. Related works

The paper targets the speed bottlenecks of learned image compression with uneven channel grouping and spatial-channel context, then builds ELIC around an efficient transform. It also proposes fast preview generation while reviewing the rate-distortion and adaptive-entropy methods behind its approach.

1. Introduction

  • Learned compression has improved rate-distortion performance, with cited recent methods outperforming VVC on PSNR and MS-SSIM, but practical deployment requires attention to decoding speed.The introduction also reports usually better MS-SSIM than JPEG and BPG for learned methods.Source: §1 · p. 1
  • Because heavy main transforms can improve rate-distortion performance but hurt speed, the authors propose rebalancing computation toward adaptive entropy estimation to reduce latency.They motivate this rebalancing with stronger, faster entropy estimation using hyperprior and context models.Source: §1 · p. 2
  • SCCTX uses information compaction to enable uneven channel grouping and combines channel-conditional coding with spatial context for fast, effective bit-rate reduction.The authors identify the conventional 10-group channel-conditional approach as a speed bottleneck and propose uneven grouping to improve rate-distortion performance while maintaining fast inference.Source: §1 · pp. 1–2
  • ELIC uses stacked residual blocks instead of GDN layers and is reported to surpass VVC on PSNR and MS-SSIM while achieving state-of-the-art coding performance and speed.The introduction points to Figure 1 and Table 2 for the rate-speed comparison and results.Source: §1 · p. 2
  • The paper proposes an efficient method for generating preview images from compressed representations, which it describes as the first work to address very-fast learned-compression preview.The authors present fast preview generation as a separate contribution alongside SCCTX and ELIC.Source: §1 · p. 2

2.1. Learned lossy image compression

  • Learned lossy compression trains an analysis-synthesis pair to balance estimated rate and reconstruction distortion; Ballé et al. enabled differentiable rate estimation, and Ballé et al. introduced hyperprior side information.The rate-distortion objective uses a rate-controlling hyperparameter λ, and the hyperprior supplies entropy parameters as forward-adaptive information.Source: §2.1 · p. 2

2.2. Backward-adaptive entropy models

  • Backward-adaptive models reduce bits by conditioning on decoded symbols: spatial context can decode serially or use checkerboard anchors and non-anchors in parallel, while channel grouping trades speed for rate-distortion gains as K increases.The channel-wise model predicts each chunk from earlier decoded chunks; larger K improves rate-distortion performance but slows parameter estimation.Source: §2.2 · pp. 2–3
  • Prior multidimensional context proposals include 3D, multi-dimension, 2-chunk, and cross-channel models, which the authors characterize as still suffering from slow speed.The cited approaches use different context structures, including serial global-and-local coding and denser cross-channel references.Source: §2.2 · pp. 2–3

3. Parallel multi-dimension context modeling

The section identifies information compaction in learned transforms as a design prior, then uses uneven channel grouping and combined spatial-channel context to support efficient adaptive coding.

3.1. Information compaction property

  • Learned transforms show orderless information compaction across tested models [39] [15] [24]; under channel-conditional coding, earlier groups carry more entropy and image semantics.The authors treat this compaction as prior knowledge for model design, while noting that its theoretical explanation is beyond the paper’s scope.Source: §3.1 · pp. 3–4

3.2. Unevenly grouped channel-wise context model

  • The proposed five-chunk scheme uses 16, 16, 32, 64, and M-128 channels, requiring five parallel calculations to decode all slices.Fewer channels are assigned to earlier chunks and more to later ones, reflecting the lower information and predictive use of later channels.Source: §3.2 · p. 4

3.3. SCCTX: space-channel context model

  • SCCTX assumes spatial and channel redundancy are orthogonal, combining per-chunk spatial and channel context with hyperprior features to predict entropy parameters.By default, it uses parallel checkerboard spatial context only within each channel chunk [24].Source: §3.3 · p. 4

4. ELIC: efficient learned image compression with scalable residual nonlinearity

ELIC replaces GDN layers with stacked residual blocks to strengthen nonlinear transforms, then combines SCCTX entropy prediction with lighter stride-2 transform networks. The residual design also supports scalable profiling and is easier to extend to dynamic or slimmable inference.

4.1. Stacking residual blocks for nonlinearity

  • GDN [15] [22] [29] [50] applies point-wise nonlinear scaling across channels, motivating investigation of alternatives beyond treating GDN as an activation [14] [15].The paper characterizes GDN as more similar to point-wise attention than to linear affine normalization.Source: §4.1 · p. 5
  • Stacking residual bottleneck blocks instead of GDN improves rate-distortion performance as block count increases, even without attention.The authors attribute the improvement to stronger nonlinearity that better represents intermediate features.Source: §4.1 · p. 5
  • Residual blocks enable scalable model profiling and are easier to extend to dynamic or slimmable inference than GDN, which requires special handling.The authors note that architecture and loss-function search could also benefit, but are outside this work.Source: §4.1 · p. 5

4.2. Architecture of ELIC

  • ELIC uses SCCTX to predict mean-scale Gaussian entropy parameters Θ=(µ, σ), enabling lighter transforms than recent models [15] [22] [50] alongside stride-2 convolutions.The entropy model is Gaussian [39].Source: §4.2 · p. 5

5. Quickly decoding thumbnail-preview

ELIC adds a lightweight thumbnail synthesizer that uses early decoded channels to produce low-resolution previews, avoiding the heavy full-resolution synthesis when speed matters more than image quality.

5. Quickly decoding thumbnail-preview

  • Thumbnail previews support server image browsing and progressive decoding previews [40], where repeatedly invoking the heavy full-resolution synthesizer slows decoding and speed outweighs image quality.The heavy synthesis inference is the major decoding bottleneck; directly decoding full-resolution images is impractical for these previews.Source: §5 · pp. 5–6
  • The thumbnail synthesizer reconstructs previews from the first 4 SCCTX chunks (128 channels), where most semantic information is compacted.The method trains an additional tiny network to generate low-resolution thumbnail-preview images.Source: §5 · p. 6
  • After training the main models, the learned parameters are frozen, the main synthesizer is replaced with an initialized thumbnail synthesizer, and distortion optimization is restarted.Source: §5 · p. 6
  • A few microseconds suffice to decode a 768 × 512 image with the thumbnail synthesizer, more efficiently than down-sampling the fully reconstructed image.The comparison is specifically against obtaining the preview by down-sampling the entirely reconstructed full-resolution image.Source: §5 · p. 6

6. Experiments

ELIC combines strong rate-distortion performance with fast coding, while its uneven grouping, residual-block transforms, and preview and progressive decoding support efficient inference and flexible reconstruction.

6.1. Settings

  • Training uses ImageNet images and Kodak and CLIC Professional evaluation, with reported ELIC models finetuned for 2M iterations.The setup uses multiple λ quality presets and fixed channel counts N = 192 and M = 320.Source: §6.1 · p. 6

6.2. Quantitative results

  • BD-rate from PSNR-BPP curves is the quantitative metric, anchored to VVC or BPG at 0%.The supplementary material also reports BD-rates calculated on MS-SSIM.Source: §6.2 · p. 6

6.2.1 Comparison of coding performance and speed

  • ELIC outperforms VVC in PSNR and MS-SSIM and reaches state-of-the-art RD-latency Pareto performance among the tested approaches.The paper reports that learned models including ELIC can encode as fast as they decode; Figure 9 compares rate-distortion curves on Kodak.Source: Figure 9 · p. 6, §6.2.1 · p. 7, Table 2 · p. 7
  • ELIC-sm removes attention modules and uses fewer residual blocks, retaining remarkable RD performance while significantly reducing latency.Source: §6.2.1 · p. 7

Influence of uneven grouping and spatial adaption

  • Uneven grouping halves adaptive entropy-estimation latency, while adding spatial context improves RD performance and uneven+spatial[P] remains faster than 10-slice even grouping.The comparison uses matched main and hyper autoencoder architectures, varying only the backward-adaptive grouping model.Source: Table 3 · p. 7, Influence of uneven grouping and… · p. 8
  • Replacing GDN/IGDN with residual bottleneck blocks improves RD on Ballé2018 and Minnen2018 [39] at comparable latency, with further gains from stacking blocks.Point-wise residual blocks also improve BD-rate on both baselines; larger receptive fields help more without spatial context, while stacked GDN training fails.Source: Influence of uneven grouping and… · p. 8, Table 4 · p. 8

6.3. Qualitative results

  • ELIC supports thumbnail previews and progressive decoding: the thumbnail synthesizer takes about 3 microseconds, and early channel groups reveal structure before later groups add chrominance and high frequencies.The thumbnail preview has PSNR = 23.02 dB on full-resolution Kodak; progressive decoding zero-fills undecoded channel chunks, and uses the thumbnail synthesizer for partial reconstructions.Source: §6.3 · p. 8, Figure 10 · p. 8, Figure 11 · p. 8

7. Discussion and conclusion

ELIC combines SCCTX and residual transform networks to balance compression ability and running speed, while thumbnail-based preview decoding improves utility. The authors identify YUV 4:2:0 versus 4:4:4 and objective and subjective image quality as areas for further investigation.

7. Discussion and conclusion

  • SCCTX and residual transform networks produce ELIC, balancing compression ability and running speed; a thumbnail network enables preview decoding and improves utility.The authors plan to investigate information compaction further to improve the architecture.Source: §7 · p. 8
  • VVC is designed mainly for YUV 4:2:0 rather than YUV 4:4:4, and the authors plan to explore models with remarkable objective and subjective image quality, following [10] [18] [38] [43].Source: §7 · p. 9

Supplementary Material · 1. Detailed network architecture

The supplement specifies ELIC’s transform and spatial-channel context network architectures, including their residual-block layouts, convolutional context models, and feature dimensions.

1.1. Architecture of transform networks

  • ELIC’s main transforms adapt prior architectures, replace GDN/IGDN layers with residual blocks, and produce an H × W × M coding tensor.The architecture tables cover ELIC and ELIC-sm.Source: §1.1 · p. 11, Table 1 · p. 11, Table 2 · p. 11
  • Residual bottlenecks provide a larger receptive field while keeping calculation volume acceptable.Source: §1.1 · p. 11
  • ELIC stacks three residual blocks after each down/upsampling convolution, while ELIC-sm uses one; larger ELIC models also use attention modules.Source: §1.1 · p. 11
  • The hyper analyzer and synthesizer each have three layers, and the hyper-synthesizer outputs an H × W × (2M) tensor Ψ.Source: §1.1 · p. 11

1.2. Architecture of SCCTX networks

  • The SCCTX networks use 5 × 5 convolutions for cross-channel redundancy and a single-layer 5 × 5 masked convolution for spatial context.The spatial context model is autoregressive/checkerboard-masked.Source: §1.2 · p. 11
  • For channel group k, g_ch outputs H × W × (2M^(k)) features; concatenating Φ^(k) and Ψ yields H × W × (4M^(k)+2M), reduced to 2M^(k) dimensions.Source: §1.2 · p. 11

2. Detailed experimental settings

The section specifies the implementation and latency-testing protocol, explains why deterministic inference is disabled for speed tests, and describes visualizing symbol channels and their energy.

2. Detailed experimental settings

  • Models were implemented and evaluated in PyTorch 1.8.1 on an NVIDIA TITANXP, with synchronized latency tests excluding the first six images after warm-up.Previously proposed models were reproduced under the same running conditions; reparameterized weights were fixed before speed testing.Source: §2 · p. 11
  • Deterministic inference was disabled because engineering could address reproducibility and deterministic mode disproportionately slows transposed convolutions used by ELIC and Ballé et al. and Minnen et al. baselines.The authors considered deterministic floating-point inference unnecessary and deterministic-mode speed comparisons potentially unfair.Source: §2 · pp. 11–12

2.1. Visualization explanation

  • Symbol-channel visualizations rescale each channel’s magnitude, while energy is defined as its average squared value; figures show the largest-energy channels and each channel’s log energy.The lighthouse visualization appears in Figure 2 of the main text, and Figure 3-left presents log(e) energy by channel.Source: §2.1 · p. 12

3. More rate-distortion results · 4. Res-blocks v.s. GDN on CPU

The sections report additional ELIC rate-distortion evaluations, including MS-SSIM optimization and higher-bitrate tests, and compare CPU latency and computational cost for residual blocks versus GDN. They describe a favorable MS-SSIM comparison with VVC and on-par CPU speed after replacing GDN with residual blocks.

3. More rate-distortion results

  • Table 3 reports BD-rates over VVC for PSNR and MS-SSIM on Kodak data with BPP < 1, omitting baselines without MS-SSIM results for MSE-optimized models.The table distinguishes parallel checkerboard [P] from serial [S] context models.Source: §3 · p. 12, Table 3 · p. 12
  • ELIC models trained with λ = {0.08, 0.16} perform well from 1.0 to 1.5 BPP on Kodak, with results also reported for CLIC-Professional.The paper presents larger PSNR and MS-SSIM figures for the higher-bitrate Kodak results.Source: §3 · p. 12
  • On Kodak, ELIC optimized for MS-SSIM saves about half the bit-rate versus VVC at the same MS-SSIM.The loss uses 1 − MSSSIM(x), with λ = {3, 12, 40, 120}, following Cheng et al. (2020).Source: §3 · p. 12
  • For MS-SSIM-optimized ELIC, the loss replaces its distortion term with 1 − MSSSIM(x), using λ = {3, 12, 40, 120} as in Cheng et al. (2020).Figure 4 presents these results.Source: §3 · p. 12

4. Res-blocks v.s. GDN on CPU

  • On Intel Core i7-7700, Ballé2018 is evaluated on Kodak for CPU latency across different nonlinear layers, with parameter volumes and multiply-accumulate operations (MACs) also measured using a FLOPs profiler.Latency results are in Table 4; parameter volumes and MACs are in Table 5.Source: §4 · p. 12, Table 4 · p. 12, Table 5 · p. 12
  • Replacing GDN with RB×1 yields slightly larger MACs, while Ballé2018 encoding and decoding speeds remain on par with the original GDN model.The CPU result is described as similar to the GPU results in Table 4 of the main body.Source: §4 · p. 13

5. Image reconstruction results · 6. Progressive decoding

The reconstruction section compares ELIC with Cheng et al. (2020) at similar synthesis latency and includes VTM12.1 for reference. The progressive-decoding section presents thumbnail and full-synthesizer results with zero or mean filling.

5. Image reconstruction results

  • ELIC and Cheng et al. (2020) are compared in reconstruction examples because their synthesis latency is close, with VTM12.1 shown for reference.The examples show the lighthouse (kodim19) and sculpture (kodim17) images; the supplied passages do not report a quality winner.Source: §5 · p. 13, Figure 5 · p. 18, Figure 6 · p. 19

6. Progressive decoding

  • The paper presents progressive decoding with a thumbnail synthesizer and zero filling, and additionally shows full-synthesizer and/or mean-filling variants.The main-text setting decodes the first 4 groups with a thumbnail decoder and zero filling.Source: §6 · p. 13, Figure 7 · p. 20
  • All presented learning-based models decode the 512 × 768 Kodak image in 100 microseconds, and the figures include PSNR-BPP and MS-SSIM rate-distortion curves.The supplied captions identify Kodak curves for PSNR-BPP and MS-SSIM, including a Kodak MS-SSIM curve optimized for maximizing MS-SSIM.Source: Figure 1 · p. 14, Figure 2 · p. 15, Figure 4 · p. 17

5. Image reconstruction results

  • The reconstruction examples cover the lighthouse (kodim19) and sculpture (kodim17) images.The captions identify qualitative comparisons, but do not specify which method performs better.Source: Figure 5 · p. 18, Figure 6 · p. 19

6. Progressive decoding

  • For kodim05, the four shown settings combine thumbnail or full synthesis with zero or mean filling; the thumbnail settings decode the first 4 of 5 groups.Figure 7 labels these settings thumb-zero, thumb-mean, full-zero, and full-mean.Source: Figure 7 · p. 20
  • The house image (kodim24) also has progressive-decoding results, displayed in the same order as the kodim05 settings.The caption does not restate the settings, referring instead to Figure 7 for their order.Source: Figure 8 · p. 20, Figure 7 · p. 20

Related papers

Loading 2203.10886v2…