PaperlayerView exact revision on arXiv

Source-linked AI summary

ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding

Dailan He, Ziming Yang, Weikun Peng, Rui Ma, Hongwei Qin, Yan Wang

arXiv:2203.10886v2cs.CVeess.IV

TL;DR

Learned image compression has strong coding performance, but practical adoption requires better control of decoding and transform-inference speed. The paper introduces compaction-aware spatial-channel entropy modeling and residual transforms, yielding ELIC with state-of-the-art compression-speed balance and fast preview capabilities.

  • Problem

    Learned image compression requires improved running efficiency, especially decoding speed, alongside its strong rate-distortion performance.

  • Method

    The paper combines uneven channel-conditional grouping with spatial context in SCCTX, then builds ELIC with stacked residual-block transforms and thumbnail/progressive decoding methods.

  • Results

    ELIC achieves state-of-the-art coding performance and running speed, surpasses VVC on PSNR and MS-SSIM, and supports fast preview and progressive decoding.

  • Takeaways & Limitations

    The proposed model better balances compression ability and running speed while improving the utility of learned image compression through preview decoding.

  • Takeaways & Limitations

    The paper notes that VVC is mainly designed for YUV 4:2:0 rather than YUV 4:4:4, and leaves theoretical analysis of information compaction beyond scope.

Abstract

from arXiv · show

Recently, learned image compression techniques have achieved remarkable performance, even surpassing the best manually designed lossy image coders. They are promising to be large-scale adopted. For the sake of practicality, a thorough investigation of the architecture design of learned image compression, regarding both compression performance and running speed, is essential. In this paper, we first propose uneven channel-conditional adaptive coding, motivated by the observation of energy compaction in learned image compression. Combining the proposed uneven grouping model with existing context models, we obtain a spatial-channel contextual adaptive model to improve the coding performance without damage to running speed. Then we study the structure of the main transform and propose an efficient model, ELIC, to achieve state-of-the-art speed and compression ability. With superior performance, the proposed model also supports extremely fast preview decoding and progressive decoding, which makes the coming application of learning-based image compression more promising.

1. Introduction

Learned image compression offers strong rate-distortion performance but remains constrained by decoding and transform-inference speed. This paper addresses the efficiency trade-off through parallel contextual entropy modeling and lighter transform architectures.

  • Learned image compression can outperform conventional formats and recent VVC coding on reported PSNR and MS-SSIM, motivating practical deployment studies.
  • Joint adaptive entropy models reduce bit-rate by exploiting extra latent variables, but spatial autoregression and progressive channel grouping reduce decoding parallelism.Checkerboard spatial context and channel-wise context improve parallelism, though channel grouping still introduces sequential steps.
  • The paper proposes uneven channel grouping combined with spatial context as SCCTX, using information compaction to reduce bit-rate while preserving fast inference.The approach treats stronger early-channel activations as a useful inductive bias for channel-conditional entropy modeling.
  • ELIC replaces GDN layers with stacked residual blocks and reports state-of-the-art coding performance and running speed, surpassing VVC on PSNR and MS-SSIM.The proposed transform reallocates computation toward contextual entropy estimation while retaining a faster main transform.
  • The paper also proposes very-fast preview-image generation from compressed representations, addressing thumbnail and progressive-decoding use cases.Preview decoding avoids repeatedly running the heavy full-resolution synthesizer, while progressive decoding reconstructs structure before chrominance and high-frequency details.

3. Parallel multi-dimension context modeling

ELIC exploits information compaction across latent channels by using uneven channel groups, then combines channel-wise and spatial context to reduce redundancy while preserving parallel decoding.

  • 3.1. Information compaction property: Learned analysis transforms concentrate energy and entropy in earlier latent channels, which carry more information and are more frequently referenced by later groups.This pattern was observed across five tested models and aligns with progressive reconstruction from beginning channels.
  • 3.2. Unevenly grouped channel-wise context model: The proposed uneven grouping assigns fewer channels to early chunks and progressively more channels to later chunks, using five chunks with sizes 16, 16, 32, 64, and M−128.This allocation preserves finer granularity where entropy is higher and coarsens grouping where channels contain less information.
  • 3.2. Unevenly grouped channel-wise context model: Uneven grouping requires five parallel calculations to decode all channel slices, reducing the running time of channel-conditional coding.Larger later chunks reduce cross-group references while retaining finer early-channel conditioning.
  • 3.3. SCCTX: space-channel context model: SCCTX combines spatial and channel-wise context because the two axes are treated as orthogonal sources of redundancy.Spatial and channel branches are combined with the hyperprior representation to predict entropy parameters for each chunk.
  • 3.3. SCCTX: space-channel context model: The spatial context is applied within each channel chunk, while symbols in the same group can be processed in parallel.The default spatial component uses a parallel checkerboard model for efficiency.

4. ELIC: efficient learned image compression with scalable residual nonlinearity

ELIC replaces GDN-based transform layers with stacked residual bottleneck blocks and uses SCCTX to support lighter transforms and scalable model profiling.

  • 4.1. Stacking residual blocks for nonlinearity: ELIC replaces GDN and IGDN layers with stacks of residual bottleneck blocks to provide stronger nonlinear feature representation for rate-distortion optimization.The paper reports further performance improvement as the number of stacked blocks increases, even without an attention mechanism.
  • 4.1. Stacking residual blocks for nonlinearity: Residual-block stacking enables scalable model profiling and is easier to extend to dynamic or slimmable inference than GDN.The paper identifies architecture search and loss-function search as possible extensions outside this work.
  • 4.2. ELIC architecture: SCCTX predicts mean-scale Gaussian entropy parameters, allowing ELIC to adopt lighter main transform networks than recent prior models.The main transforms use stride-2 convolutions for changing feature-map sizes.
  • 4.2. ELIC architecture: The ELIC architecture combines SCCTX, residual blocks, attention modules, arithmetic coding, and transposed convolutions in its analysis and synthesis pipeline.The thumbnail synthesizer is shown separately as a lightweight reconstruction component.

5. Quickly decoding thumbnail-preview

ELIC addresses slow preview decoding by reconstructing thumbnails from early decoded channels with a tiny synthesizer instead of running the full-resolution synthesis network.

  • 5. Quickly decoding thumbnail-preview: Full-resolution synthesis is the main decoding bottleneck, making repeated reconstruction impractical for thumbnail browsing and progressive decoding.These use cases prioritize decoding speed over full-resolution image quality.
  • 5. Quickly decoding thumbnail-preview: The thumbnail synthesizer reconstructs low-resolution previews from the first four chunks, corresponding to the first 128 channels.The design relies on semantic information being compacted in earlier decoded channels under SCCTX.
  • 5. Quickly decoding thumbnail-preview: The thumbnail synthesizer requires only a few microseconds to decode 768 × 512 images and is more efficient than down-sampling fully reconstructed images.It is trained after the main model parameters are frozen.

6. Experiments

Experiments evaluate ELIC under standardized Kodak and CLIC Professional settings, comparing rate-distortion performance, latency, grouping schemes, nonlinear transforms, preview decoding, and progressive decoding. ELIC improves compression quality and speed, while its thumbnail synthesizer and channel-wise decoding support fast previews.

  • Overall comparison: ELIC outperforms VVC on PSNR-based rate-distortion performance and also achieves strong MS-SSIM performance.BD-rate is computed relative to VVC or BPG, while Figure 9 reports Kodak rate-distortion curves for MSE-optimized learned models.
  • Slim architecture: ELIC-sm removes attention modules and uses RB×1 blocks, retaining remarkable rate-distortion performance while significantly reducing latency.The slim model is adapted from ELIC for lower computational cost.
  • Overall comparison: ELIC achieves state-of-the-art performance on the rate-distortion–latency Pareto frontier, including learned models decoding Kodak images within 100 microseconds.The comparison considers both compression quality and coding speed.
  • Grouping and spatial context: Uneven grouping halves adaptive-entropy-estimation latency, while adding spatial context improves rate-distortion performance and remains faster than 10-slice even grouping.The comparison uses matched main and hyper autoencoder architectures, differing only in backward-adaptive grouping models.
  • Nonlinear transforms: Replacing GDN/IGDN with residual bottleneck blocks improves BD-rates on Ballé2018 and Minnen2018 while keeping inference latency on par with GDN.Stacking more residual blocks further reduces BD-rates, whereas stacking GDN layers becomes unstable and fails during training.
  • Preview and progressive decoding: The thumbnail synthesizer reconstructs low-resolution previews at PSNR 23.02 dB and takes about 3 microseconds, over 12 times faster than full synthesis.Progressive decoding reconstructs structural information first, followed by chrominance and high-frequency information from later channel groups.

7. Discussion and conclusion

The paper concludes that SCCTX and residual transform networks produce ELIC, which balances compression ability and running speed. It also identifies future work on information compaction and objective–subjective image quality.

  • Conclusion: SCCTX and residual transform networks yield ELIC, a state-of-the-art model balancing compression ability and running speed.The conclusion presents this balance as the principal outcome of the proposed architecture.

Supplementary Material

The paper is titled “ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding.”

  • The proposed method is named ELIC.
  • The paper concerns learned image compression.
  • The title highlights efficiency as a design objective.

1. Detailed network architecture

The supplementary architecture description details ELIC and ELIC-sm main transforms, residual bottleneck nonlinearities, hyper transforms, and spatial-channel context parameter aggregation. The designs use channel-grouped coding and bottlenecked residual structures to balance receptive field and computation.

  • Main transform architecture: ELIC and ELIC-sm main transforms are adapted from prior Ballé2018, Minnen2018, and Minnen2020 architectures.The coding symbol ŷ is an H × W × M tensor, with N intermediate-feature channels and M analyzer output channels.
  • Residual transform: Residual bottleneck structures provide a dimensional bottleneck that enlarges receptive fields while keeping computational volume acceptable.ELIC stacks three residual blocks after each downsampling or upsampling convolution, while ELIC-sm uses one.
  • Model variants: ELIC uses attention modules in larger models to further enhance nonlinearity, while ELIC-sm reduces the residual-block count.The architecture tables specify the corresponding main-transform designs.
  • Hyperprior architecture: The hyper analyzer and synthesizer use three layers, with the hyper-synthesizer output represented as an H × W × (2M) tensor Ψ.This hyperprior output supplies parameters used by the adaptive coding model.
  • Context modeling: Five-by-five convolutions analyze cross-channel redundancy, while a single-layer 5 × 5 masked convolution provides the spatial context model.The spatial-channel outputs are combined with Ψ and linearly reduced to 2M^(k) dimensions for parameter aggregation.

2. Detailed experimental settings

The experiments use a controlled PyTorch/GPU protocol for rate-distortion and speed comparisons, while analyzing information compaction across latent symbol channels. Channel energy is visualized through rescaled magnitudes and average squared values.

  • Experimental protocol: Models are implemented and evaluated under PyTorch 1.8.1 on an NVIDIA TITANXP, with reproduced baselines tested under identical conditions.Reparameterized weights are fixed before speed testing, and GPU synchronization is used for latency measurement.
  • Experimental protocol: Deterministic floating-point inference is disabled because it slows operators such as transposed convolutions and could make comparisons unfair.The authors suggest engineering solutions such as integer-only inference for deterministic deployment.
  • Experimental protocol: Prior rate-distortion results are collected by contacting original authors or accessing their released data.
  • Channel analysis: Each latent symbol channel is visualized as a grayscale image formed by rescaling the magnitude of that channel.
  • Channel analysis: Channel energy is defined as the average square value of the symbols in each channel, and logarithmic energy is plotted to examine compaction.The main-text visualizations emphasize channels with the largest energy values.

3. More rate-distortion results

Additional experiments evaluate ELIC across higher bit-rate ranges, MS-SSIM optimization, CPU efficiency, and complexity measures. The reported results include substantial MS-SSIM rate savings over VVC and comparable speed after replacing GDN blocks with residual blocks.

  • MS-SSIM results: Some MS-SSIM baseline results are omitted because the original authors did not provide them for MSE-optimized models.
  • Rate-distortion results: ELIC is evaluated from 1.0 to 1.5 BPP on Kodak, with additional results reported on CLIC-Professional.The high-rate models are trained with λ = {0.08, 0.16}.
  • MS-SSIM results: About half the bit-rate is saved over VVC at the same MS-SSIM on Kodak when ELIC is optimized for MS-SSIM.The models use 1 − MSSSIM(x) as the distortion term and λ = {3, 12, 40, 120}.
  • Rate-distortion results: Table 3 reports BD-rates over VVC for Kodak data points below 1 BPP using PSNR and MS-SSIM in dB.The table distinguishes parallel checkerboard and serial context models.
  • Efficiency analysis: CPU running speed, parameter volume, and MAC counts are separately evaluated for Ballé2018 variants using different nonlinear layers.
  • Efficiency analysis: Replacing GDN blocks with one residual block slightly increases MACs while keeping adapted Ballé2018 encoding and decoding speeds on par with the original GDN-based model.This CPU finding is reported as similar to the GPU results.

5. Image reconstruction results

The reconstruction experiments compare ELIC with Cheng et al. (2020) and VTM12.1, and examine progressive decoding under different synthesizer and filling choices. The figures cover rate-distortion curves, qualitative reconstructions, and progressive outputs.

  • Qualitative reconstruction: ELIC and Cheng et al. (2020) are compared on reconstructed lighthouse and sculpture images because their synthesis latencies are close.VTM12.1 decoded images are included as a reference.
  • Progressive decoding: Progressive decoding is evaluated with thumbnail or full synthesizers and with zero or mean filling across the displayed reconstruction groups.Figures 7 and 8 show results for kodim05 and kodim24, respectively.
  • Rate-distortion evaluation: The Kodak PSNR-BPP and MS-SSIM rate-distortion figures use models optimized for minimizing MSE, except the MS-SSIM-optimized curve.The displayed Kodak image is 512 × 768, and the figures state 100 microseconds decoding for presented learning-based models.
  • Rate-distortion evaluation: A PSNR-BPP curve is also reported on CLIC-Professional under MSE optimization.
Loading 2203.10886v2…