PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Generative Image Compression with Quantized Latents Refinement

Hao Cao, Yinhuan Huang, Wenqi Guo, Zhijin Qin, Jungong Han

arXiv:2610.29691v1cs.CV

TL;DR

Lightweight generative image compression offers faster inference than diffusion-based methods but lower reconstruction performance, while coarse low-bitrate quantization can leave fine details unspecified. GR-GIC combines entropy-guided latent refinement with reorganized P-RVQ; on Kodak it reports BD-rate savings over OneDC [41] across four metrics.

  • Problem

    High-performing GIC often relies on costly pretrained diffusion models, while lightweight methods are faster but leave reconstruction performance behind; coarse low-bitrate quantization can also collapse latents with different fine details.

    Source: §1 · p. 1

  • Method

    GR-GIC combines entropy-context-guided Gaussian refinement with adaptive training perturbations and reorganized P-RVQ using low-dimensional lookups.

    Source: §1 · p. 1, §3.3 · p. 4

  • Results

    On Kodak, versus OneDC [41], GR-GIC saves BD-rate 23.19% for LPIPS, 37.71% for DISTS, 22.30% for PSNR, and 22.14% for FID.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The optimized codec encodes and decodes in 4.60 and 5.98 ms on an NVIDIA A100, and has latency comparable to lightweight GIC methods on Jetson Orin NX.

    Source: §1 · p. 2

    Evaluation spans 0.0356–0.1426 BPP, and latency comparisons use official or released implementations with available acceleration rather than identical backends.

    Source: §5 · p. 8

Abstract

from arXiv · show

Generative image compression (GIC) delivers strong perceptual quality at low bitrates. However, GIC methods with strong perceptual quality often rely on pretrained diffusion models with high latency, while recent lightweight GIC methods reduce latency but deliver lower perceptual quality. We observe that, for lightweight GIC methods at low bitrates, coarse deterministic quantization can collapse nearby encoder latents corresponding to different fine details to the same point, making these details difficult to recover. Motivated by this, we propose GR-GIC, an efficient GIC that delivers competitive reconstruction performance with fast inference. For better performance, we introduce Guided Refinement (GR), which combines latent refinement guided by entropy context with adaptive perturbations for training regularization, jointly improving reconstruction at low bitrates. For fast inference, we introduce reorganized projected residual vector quantization (P-RVQ), which equivalently reorganizes residual updates into low-dimensional lookups to expose greater parallelism and reduce sequential computation. Experiments on Kodak show that for performance, compared with OneDC, GR-GIC achieves BD-rate savings of 23.19%, 37.71%, 22.30%, and 22.14% for LPIPS, DISTS, PSNR, and FID, respectively. For fast inference, the highly optimized implementation of GR-GIC encodes and decodes in 4.60 and 5.98 ms on an NVIDIA A100. Specifically, the reorganized P-RVQ implementation achieves a $3.14\times$ average speedup over the original implementation under the same FP32 setting. On an NVIDIA Jetson Orin NX, GR-GIC achieves latency comparable to lightweight GIC methods.

1. Introduction

GR-GIC addresses the trade-off between reconstruction quality and inference speed in lightweight generative image compression. It combines entropy-guided latent refinement with reorganized P-RVQ, and reports Kodak quality gains over OneDC alongside fast inference.

1. Introduction

  • At low bitrates, coarse deterministic quantization can map nearby latents with different fine details to the same point, leaving those details unspecified.High-performing diffusion-based methods can be costly to deploy, while lightweight methods have lower reconstruction performance, creating a performance–speed trade-off.Source: §1 · p. 1
  • Guided Refinement (GR) uses entropy context to set a Gaussian refinement distribution around the recovered latent, with adaptive perturbations for training regularization.The mean shifts the latent toward a refined center, while the variance controls perturbations; the two jointly improve low-bitrate reconstruction.Source: §1 · p. 1
  • Reorganized P-RVQ replaces serial high-dimensional residual updates with low-dimensional lookups, exposing more parallelism and reducing sequential computation.The method targets P-RVQ, identified as a major encoder latency bottleneck in EF-LIC [6].Source: §1 · pp. 1–2
  • On Kodak, versus OneDC [41], GR-GIC saves BD-rate 23.19% for LPIPS, 37.71% for DISTS, 22.30% for PSNR, and 22.14% for FID.The highly optimized implementation encodes and decodes in 4.60 and 5.98 ms on an NVIDIA A100; reorganized P-RVQ averages a 3.14× speedup over the original under the same FP32 setting.Source: §1 · p. 2

2. Related Work

Related work spans diffusion-based and VAE-based generative image compression, alongside lightweight approaches and methods that compensate for quantization loss. GR differs from prior latent corrections by using decoder-available entropy context to adapt a refinement distribution and by applying adaptive perturbations during training.

Related Work

  • Diffusion-based GIC uses generative priors to reconstruct details absent from compressed representations [8] [43], but model size and computation remain substantial for deployment.Recent methods pursue efficiency through fewer diffusion steps, one-step reconstruction, or lighter architectures [18] [22] [29] [41] [46].Source: §2 · p. 2
  • VAE-based GIC offers a more efficient alternative through adversarial priors, improved latent representations, and vector quantization [20] [23] [26] [31] [33].Lightweight examples include ProGIC with a depthwise-convolution backbone and RVQ [7], and EF-LIC, which models latent correlations [6].Source: §2 · p. 2
  • Prior latent compensation includes residual prediction, post-quantization filtering, adaptive quantization or correction, and latent or noise compensation [32] [48].Unlike approaches that use deterministic correction, encoder guidance, or context-independent stochastic compensation, GR uses decoder-available entropy context to adapt refinement to location and image content, without additional payload; adaptive perturbations regularize generative reconstruction.Source: §2 · pp. 2–3

3. Method

GR-GIC adds entropy-context-guided refinement after latent quantization and reorganizes P-RVQ to shorten its sequential computation path. The method targets low-bitrate reconstruction and faster inference while preserving P-RVQ’s greedy quantization process.

3.1. Overview

  • GR-GIC retains EF-LIC’s analysis transform, hyperprior, groupwise context, and P-RVQ structure, adding a refinement head before synthesis.The encoder produces the latent, while the hyper encoder and decoder provide context used to predict parameters for affine projections and P-RVQ.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2. Guided Refinement of Quantized Latents

  • Guided Refinement uses entropy context to shift the recovered latent toward a refined center and applies adaptive perturbations for training regularization.Its Gaussian refinement distribution is conditioned on the recovered latent and entropy context parameters already available after decoding.Source: §3.2 · pp. 3–4

3.3. Reorganized Projected Residual Vector Quantization

  • 3.14× average speedup over the original implementation across 16 evaluated settings; the reorganization moves high-dimensional reconstruction outside P-RVQ’s critical path.The algebraic reorganization accounts for 23.54% of total encoder latency reduction.Source: §3.3 · pp. 4–5, Figure 3 · p. 5

4. Experiments

Across four datasets, GR-GIC improves rate-quality curves over lightweight codecs while retaining low inference latency. The experiments also test Guided Refinement and deployment on edge hardware.

4.1. Experimental Setup

  • Evaluation covers Kodak, Tecnick, DIV2K validation, and CLIC 2020 Professional, using LPIPS, DISTS, PSNR, and patch-based FID; runtime is measured at batch size one on 512×768 images.Training uses two 1M-iteration ImageNet stages; the implementation adds a GR head before synthesis and reorganizes P-RVQ.Source: §4.1 · p. 5

4.2. Comparison with Previous Methods

  • 23.19%, 37.71%, 22.30%, and 22.14% BD-rate savings over OneDC on Kodak correspond to LPIPS, DISTS, PSNR, and FID, respectively.Figure 4 compares rate and reconstruction quality across four datasets.Source: §4.2 · pp. 5–6, Figure 4 · p. 6, Table 1 · p. 7
  • 4.60 and 5.98 ms are GR-GIC's encoding and decoding latencies on an A100; its rate-quality curves improve on previous lightweight codecs across reported metrics while retaining low latency.The optimized implementation is 16.73× and 34.19× faster than official OneDC for encoding and decoding, respectively.Source: §4.2 · p. 6, Table 1 · p. 7
  • At similar rates, qualitative comparisons show GR-GIC preserving a thin eyeglass frame and nose pad and reconstructing a small circular fixture with coherent local geometry.Figure 5 marks local details with boxes and reports LPIPS and BPP labels.Source: §4.2 · p. 6, Figure 5 · p. 7

4.3. Guided Refinement Configuration Study

  • M4 lowers LPIPS at every evaluated rate and FID by 4.2–7.8 versus M2 and 1.5–4.9 versus M3, despite M2 and M3 matching its parameter count and forward FLOPs.The study supports adaptive perturbations as training regularization and shows benefit from adapting their scale to spatial location and image content.Source: §4.3 · pp. 7–8, Table 2 · p. 8

4.4. Edge-Device Deployment

  • On Jetson Orin NX, GR-GIC has the lowest decoding latency at all four tested resolutions and the lowest encoding latency from 1080p to 4K.At 4K, OneDC and AEIC run out of memory, while CoD-Lite decoding exceeds 10 seconds.Source: §4.4 · p. 8, Table 3 · p. 8

5. Limitations

The evaluation spans 0.0356 to 0.1426 BPP, and the latency comparisons characterize the complete deployed codec rather than backbone-only FP32 inference under identical backends.

5. Limitations

  • The evaluated bitrate range is 0.0356 to 0.1426 BPP, with its lowest point below the 0.05 BPP ultra-low bitrate threshold.The paper notes that lower bitrates require greater decoder generative capacity as less fine detail is transmitted.Source: §5 · p. 8
  • Latency results characterize the complete deployed codec, including P-RVQ reorganization and CUDA acceleration.The paper does not claim the fastest backbone-only FP32 inference under identical backends.Source: §5 · p. 8
  • Comparisons retain official or released implementations and their available acceleration paths, reflecting system-level performance.Source: §5 · p. 8

6. Conclusion

GR-GIC combines guided latent refinement with reorganized P-RVQ for reconstruction gains and fast inference. The appendix establishes that the reorganization preserves greedy decisions while reducing sequential state-update work.

6. Conclusion

  • GR-GIC combines entropy-context latent refinement and adaptive training perturbations with reorganized P-RVQ; experiments report consistent LPIPS, DISTS, and FID gains over OneDC across four datasets and fast inference on A100 and Jetson Orin NX.The conclusion describes refinement of quantized latents and low-dimensional lookups that reduce sequential computation.Source: §6 · p. 8

A. Analysis of Reorganized P-RVQ

  • The appendix restores omitted projection biases, proves equivalence of the original and reorganized forms, analyzes computation and storage, and reports runtime results.Source: A. Analysis of Reorganized P-RVQ · p. 11

A.1. Complete P-RVQ Reorganization

  • The complete P-RVQ reorganization preserves the search vector at each stage, so the same greedy nearest-neighbor decisions yield the same indices and final reconstruction in real arithmetic.It avoids materializing high-dimensional partial reconstructions between search stages, keeping the greedy stage order while removing that reconstruction from the critical dependency path.Source: §A.1 · pp. 11–12

A.2. Complexity and Parallelism

  • For M = 4 and D = 16d, reorganized P-RVQ uses 6Ld rather than 48Ld state-update additions, an 8× reduction; projection and nearest-neighbor costs remain unchanged.The reduction applies to the sequential state-update path, not the complete P-RVQ runtime.Source: §A.2 · p. 12
  • The reorganized schedule evaluates all M source projections together and can apply up to M −1 later-stage low-dimensional updates concurrently, while retaining M ordered nearest-neighbor decisions.The original schedule exposes one source projection at each stage.Source: §A.2 · p. 12

A.3. Implementation and Runtime Evaluation

The implementation reorganizes P-RVQ search and fuses its ordered stages in CUDA kernels. Matched FP32 measurements show faster execution, while additional precision and kernel optimizations raise speedups further.

A.3. Implementation and Runtime Evaluation

  • A fused CUDA kernel performs candidate scoring, index selection, lookup, and later-state updates in one launch, then a second kernel gathers and sums selected reconstruction vectors when needed.The benchmark protocol uses an A100, batch size one, and weights converted from the same checkpoints.Source: §A.3 · p. 13
  • 3.14× average speedup: reorganized P-RVQ outperforms the original across 16 evaluated settings under the same Torch FP32 backend and precision.The matched comparison isolates the algebraic reorganization; Table 4 reports latency and speedup for each resolution and P-RVQ depth.Source: §A.3 · p. 13
  • 7.15× average speedup: the CUDA FP16 implementation exceeds the original FP32 implementation, combining reorganization, FP16 execution, and fused CUDA kernels.Source: §A.3 · p. 13
  • At the highest evaluated Kodak rate, reorganized Torch FP32 P-RVQ compresses in 3.2435 ms versus 11.2824 ms for original FP32, a 3.48× speedup.CUDA FP16 reaches 0.6001 ms, an 18.80× speedup at this operating point.Source: §A.3 · p. 13
  • The original and reorganized Torch FP32 implementations select identical indices, with a maximum absolute reconstruction difference of 4.77×10−7.Under FP16, reorganized Torch and CUDA implementations differ on 0.2293% of evaluated indices, mainly between nearby codewords, with little effect on final reconstruction.Source: §A.3 · p. 13

B. Architecture and Bitstream Details

The architecture follows a lightweight backbone and context pathway, then specifies fixed-width index streams, boundary padding, and the resulting nominal payload rates.

B.1. Backbone and Context Pathway

  • The codec follows EF-LIC’s lightweight backbone and context pathway; four quadtree groups are recovered using previously decoded groups, and the GR head predicts an offset and perturbation scale from the recovered latent and entropy context.Each P-RVQ depth is a separately trained operating point with its corresponding GR head.Source: §B.1 · pp. 13–14

B.2. Fixed-Width Index Packing

  • The hyper latent uses 10 bits per index, while the four main latent groups use 10, 9, 8, and 7 bits; the standalone bitstream stores image dimensions followed by five index streams.The 4-byte image-size header is excluded from reported payload BPP.Source: §B.2 · p. 14

B.3. Padding and Cropping

  • Inputs are padded on the right and bottom to dimensions divisible by 32; the decoder removes corresponding padding and returns the reconstruction at the original image size.The hyper-analysis path also pads the main latent to a multiple of four.Source: §B.3 · p. 14

B.4. Nominal Rate

  • Fixed index widths determine the nominal payload rate before boundary padding, which changes transmitted index counts and causes small deviations from those rates.The hyper latent contributes to the nominal rate alongside the four main latent groups.Source: §B.4 · pp. 14–15

C.1. Datasets and Objective Evaluation

The evaluation uses four image datasets and standardized objective metrics, with pairwise BD-rate comparisons over shared quality intervals. Runtime and model-complexity reporting follows defined implementation and accounting protocols.

C.1. Datasets and Objective Evaluation

  • Evaluation covers Kodak, Tecnick, DIV2K validation, and CLIC 2020 Professional; LPIPS, DISTS, PSNR, and patch-based FID provide standardized objective measurements.The datasets contain 24, 100, 100, and 250 images respectively, evaluated at original resolution; objective metrics are used for repeatable comparisons.Source: §C.1 · p. 15
  • BD-rate summarizes average rate differences at matched reconstruction quality over a common interval; pairs without a valid interval are left unreported.Negative values indicate bitrate savings over the common interval, and numerical tables report pairwise BD-rate.Source: §C.1 · p. 16

C.2. Runtime and Model Cost Protocol

  • Reported A100 encoding and decoding times cover the complete codec path, while latency comparisons retain each implementation’s precision, backend, and acceleration path.GR-GIC’s optimized implementation uses FP16, CUDA Graphs, and custom quantization and reconstruction kernels; its reported latency averages four independently trained operating points.Source: §C.2 · p. 16
  • The 79.389M parameter count represents one deployed GR-GIC checkpoint, not the combined storage of its four operating-point checkpoints; FLOPs use one batch-one FP32 encoder-decoder pass.Peak memory includes the loaded model and active codec buffers.Source: §C.2 · p. 16

C.3. Baseline Evaluation Protocol

The section specifies baseline, ablation, and stochastic-evaluation protocols, then reports efficiency and quality comparisons. GR-GIC performs strongly against lightweight codecs while retaining fast inference.

C.3. Baseline Evaluation Protocol

  • Baseline evaluations generally use released checkpoints on one A100 at batch size one, with implementation-specific exceptions for bitstream handling, high-resolution execution, and unavailable models.GLC receives a CompressAI bitstream integration for high-resolution comparisons; OSCAR uses CPU offloading when needed, while CADC and PICO are compared only using compatible published results. CoD-Lite timing uses released-code measurements rather than paper-reported values.Source: §C.3 · pp. 16–17

C.4. GR Ablation Protocol

  • The GR ablations compare direct synthesis, latent offsets, entropy-context refinement, and fixed versus spatially adaptive Gaussian perturbations under matched training conditions.Ablation models train for 1M iterations; the main operating points use separately trained models with a 2M-iteration schedule.Source: §C.4 · p. 17

C.5. Stochastic Evaluation Protocol

  • Main results use decoder seed 1926, and stochastic seed-stability tests vary only the decoder seed across ten runs while keeping transmitted indices and payload fixed.Perturbations are sampled only after deterministic RVQ index recovery.Source: §C.5 · p. 17

D.1. Efficiency across Operating Points

  • Across 512 × 768, 1080p, and 2K, increasing active P-RVQ depth changes runtime only modestly relative to the shared backbone.At 512 × 768, encoding latency ranges from 4.475 to 4.717 ms and decoding from 5.947 to 6.028 ms.Source: §D.1 · pp. 17–18, Table 7 · p. 18

D.2. GR-GIC Backend Comparison

  • At 512 × 768, eager PyTorch FP32 GR-GIC is 2.53× faster to encode and 4.57× faster to decode than official OneDC; CUDA FP16 further accelerates it.Across evaluated resolutions, CUDA FP16 accelerates encoding by 6.62×–8.18× and decoding by 7.47×–8.74× relative to PyTorch FP32.Source: §D.2 · p. 18

D.3. Additional Baseline Comparisons

  • At 0.03564 BPP, GR-GIC beats CADC by approximately 0.20 dB PSNR and 0.012 DISTS; on CLIC 2020 it records −54.01% PICO-anchored LPIPS BD-rate.CADC’s LPIPS uses AlexNet features rather than the main evaluation’s VGG features, so that comparison is restricted to PSNR and DISTS.Source: §D.3 · p. 18

D.4. Comparison with Lightweight Codecs

  • Across four datasets, GR-GIC has the strongest displayed lightweight-codec LPIPS, DISTS, PSNR, and FID curves at comparable rates; its 0.143-BPP point also beats EF-LIC’s fifth point on most metrics.At that rate, GR-GIC has higher PSNR and lower DISTS and FID on every dataset, with comparable or lower LPIPS.Source: §D.4 · p. 18

D.5. Additional Rate and Quality Results

GR-GIC shows favorable rate-quality results against OneDC across datasets and low latency at high image resolutions. The section also contextualizes quality results with model complexity.

D.5. Additional Rate and Quality Results

  • Across twelve LPIPS, DISTS, and FID entries, GR-GIC achieves a negative OneDC-relative BD-rate on every dataset.The rate-quality curves show this behavior over the evaluated operating ranges.Source: §D.5 · p. 19
  • AEIC uses approximately 11.2× GR-GIC’s parameters and 4.7× its FLOPs, while StableCodec uses approximately 13.4× the parameters and 14.9× the FLOPs.EF-LIC and ProGIC use fewer parameters and FLOPs than GR-GIC, while GR-GIC provides lower DISTS and FID over the reported comparisons.Source: §D.5 · p. 19

D.6. High-Resolution Runtime and Memory

  • The released GLC interface lacks a complete bitstream path, so its forward execution times are distinct from timings under the complete CompressAI integration.The integration reports encoding and decoding times at 1080p, 2K, and 4K.Source: §D.6 · p. 19
  • Among available implementations, GR-GIC has the lowest encoding latency, decoding latency, and peak memory at 1080p, 2K, and 4K.Relative to the second-lowest values, latency reductions are 1.26×–1.39× for encoding and 1.43×–1.66× for decoding.Source: §D.6 · p. 19

D.7. GR Inference Stability

Inference-stability tests find that most GR gains remain without sampled inference perturbations and that aggregate metrics vary little across decoder seeds. Additional visual comparisons annotate reconstructions with LPIPS and BPP.

D.7. GR Inference Stability

  • Removing sampled perturbation at inference changes LPIPS by only 0.000–0.002 and increases FID by 0.2–0.4 relative to complete M4.Without inference perturbation, M4 retains FID reductions relative to M2 and M3, and LPIPS remains equal to or lower than both at every evaluated rate.Source: §D.7 · p. 19
  • Across ten decoder seeds on Kodak, aggregate LPIPS varies by less than 10^-4 and FID by less than 10^-2, with identical transmitted indices and payload.Repeated FID computations with fixed reconstructions and patch extraction produce identical values.Source: §D.7 · pp. 19–20

E. Additional Visual Comparisons

  • Additional visual comparisons cover all four evaluation datasets, showing LPIPS and BPP labels for displayed reconstructions.The figures place comparisons with high-capacity diffusion codecs above those with lightweight baselines.Source: E. Additional Visual Comparisons · p. 20
Loading 2610.29691v1…