StableCodec: Taming One-Step Diffusion for Extreme Image Compression

• Traditional and learning-based codecs are often optimized for rate-distortion performance and can produce blurry or unrealistic reconstructions under severe bitrate constraints.Generative compression instead emphasizes perceptual quality and the rate-distortion-perception tradeoff [9] [10] [66] [67].Source: §1 · p. 2
• StableCodec combines SD-Turbo one-step denoising with a Deep Compression Latent Codec that transmits noisy latents at ultra-low bitrates.The approach addresses whether such latents can be compressed for effective single-step denoising.Source: §1 · pp. 2–3
• A Dual-Branch Coding Structure adds auxiliary encoder and decoder branches to improve reconstruction fidelity through semantic enhancement and structure apportionment.The auxiliary encoder embeds entropy-aware semantic information for coding decisions, while the decoder supports more consistent details during one-step denoising.Source: §1 · pp. 2–3
• End-to-end training uses a two-stage objective that jointly optimizes bitrate and pixel-level constraints.Source: §1 · p. 3
• StableCodec reports state-of-the-art FID, KID, and DISTS on CLIC 2020 and DIV2K, with high-fidelity, high-realism reconstructions down to 0.005 bpp and decoding speeds comparable to mainstream transform coding.The paper also reports experiments on Kodak and inference at arbitrary resolutions with memory consumption below 9 GB.Source: §1 · p. 3

2.1. Generative Image Compression

• Learned compression builds on nonlinear transforms and joint rate-distortion optimization, including the end-to-end framework [5], hyperprior [6], and context model [45].Subsequent work [13] [20] [21] [30] [42] [44] [49] addresses rate-distortion performance and practicality.Source: §2.1 · p. 3
• Generative image compression incorporates adversarial training into transform coding to improve perceptual quality, while diffusion-based methods target ultra-low bitrate compression but are constrained by fidelity and multi-step inference.The field evaluates this tradeoff through rate-distortion-perception criteria [9] [10] [66] [67]; GLC [28] applies transform coding in VQ-VAE's generative latent space.Source: §2.1 · p. 3

2.2. Generative Models and Few-Step Diffusions

• Diffusion models learn to reverse a noise perturbation process through a Markovian framework and have become a prominent image-generation approach.DDPM [25], DDIM [56], and LDM [52] are cited as advances that reduced computational complexity and improved image synthesis quality.Source: §2.2 · p. 3
• Few-step diffusion methods approximate the full denoising trajectory in one or a few steps to improve inference efficiency while maintaining generation quality.SD/SDXL-Turbo [54] demonstrates image generation in 1 to 4 steps with near-parity quality compared with multi-step models.Source: §2.2 · p. 3

3.1. Overview

• StableCodec uses SD-Turbo with a Deep Compression Latent Codec and auxiliary branches, then optimizes bitrate and pixel-level constraints end-to-end.The framework adapts SD-Turbo with LoRA in its VAE encoder and denoising UNet while leaving the VAE decoder unchanged.Source: §3.1 · pp. 3–4, Figure 3 · p. 4

3.2. Deep Compression Latent Codec

• Entropy coding operates on latents at 64× spatial compression and the hyperprior at 256×, then gs restores latents to 8× for denoising and decoding.The codec uses extreme analysis and synthesis transforms and a four-step autoregressive entropy model.Source: §3.2 · p. 4

3.3. Dual-Branch Coding Structure

• The dual-branch structure adds an auxiliary encoder and decoder to improve reconstruction fidelity alongside the latent codec.The encoder supplies semantic information, while the decoder apportions structure during decoding.Source: §3.3 · p. 4, §3.3.1 · pp. 4–5, §3.3.2 · p. 5

3.3.1. Entropy-Aware Semantic Enhancement

• The frozen ELIC analysis transform serves as EAux, adding entropy-aware semantic information that is aligned to 16× resolution and concatenated channel-wise.The SD-Turbo VAE is perceptually oriented and floating-point, limiting pixel-level fidelity and practical entropy coding; Table 1 contrasts its pixel information with ELIC's.Source: §3.3.1 · pp. 4–5, Table 1 · p. 5, Figure 4 · p. 5

3.3.2. Structure Apportionment

• DAux decodes basic structure directly from the bitstream, bypassing the UNet so gs can focus on guidance for realistic details.With DAux, energy in gs latents drops significantly, while more semantically aligned details are reconstructed.Source: §3.3.2 · p. 5, Figure 5 · p. 5

3.4. End-to-End Training Objective

• Training balances bitrate R and pixel-level distortion D with a Lagrange multiplier and uses two-stage implicit bitrate pruning to reach ultra-low target bitrates.The distortion includes MSE, VGG-feature LPIPS, and CLIP-embedding distance; GAN loss is added only in stage two.Source: §3.4 · pp. 5–6

4.1. Implementation

• Evaluation covers CLIC 2020 Test, DIV2K Val, and Kodak, emphasizing FID, KID, and DISTS because pixel-level metrics have limitations at ultra-low bitrates.Training uses DF2K and CLIC 2020 Professional; the second stage adds GAN training after a first stage of over 100k iterations.Source: §4.1 · pp. 6–7

4.2.1. Rate-Distortion-Perception Performance

• At bitrates as low as 0.005 bpp, StableCodec outperforms compared methods on FID, KID, and DISTS over CLIC 2020 Test and DIV2K Val.Compared with GLC [28], it shows superior and stable FID and KID performance; TACO [36] has the best LPIPS but high FID, KID, and DISTS scores.Source: §4.2.1 · p. 7

4.2.2. Qualitative Comparisons

• On Kodak, StableCodec produces visually aligned details and realistic textures, while competing methods show blurring, deviations from originals, or artifacts.The examples include teeth and murals; EGIC and TACO show noticeable artifacts in detailed areas.Source: §4.2.2 · pp. 7–8, Figure 7 · p. 7

4.2.3. Computational Complexity

• StableCodec achieves encoding and decoding speeds comparable to representative codecs, with significantly better performance at ultra-low bitrates.The comparison uses a single RTX 3090 GPU; Table 2 reports encoding and decoding seconds averaged on Kodak.Source: Table 2 · p. 7, §4.2.3 · p. 8

Auxiliary Encoder and Decoder

• Adding the auxiliary encoder saves more than 20% of bits at the same reconstruction quality, and the auxiliary decoder further improves performance, particularly perceptual quality.The purified gs latent provides better guidance for reconstruction consistency.Source: §5 · p. 8

LoRA Ranks

• Increasing LoRA ranks to 16/32 improves distortion and perception, while larger ranks can degrade performance as pre-trained priors may be corrupted.Adding LoRA to the VAE decoder improves PSNR and MS-SSIM at the cost of LPIPS and DISTS, so the decoder is left unchanged.Source: §5 · p. 8

Training Strategy

• Implicit bitrate pruning improves performance considerably, while GAN training introduces a significant distortion–perception trade-off and the CLIP distance alleviates distortion degradation.The base one-stage objective contains bitrate, MSE, and LPIPS; the ablations progressively add implicit bitrate pruning, adversarial training, and the CLIP distance term.Source: §5 · p. 8, Table 5 · p. 8

6. Conclusion

• StableCodec combines one-step diffusion with a Deep Compression Latent Codec and Dual-Branch Coding Structure to achieve extreme-bitrate compression with realism, fidelity, and coding efficiency.The paper reports FID, KID, and DISTS superiority at bitrates as low as 0.005 bpp, with speeds competitive with mainstream transform coding methods.Source: §6 · p. 8

A. Inference for Arbitrary Resolution

• Tiled VAE processing and Gaussian-weighted aggregation of overlapping latent patches support arbitrary-resolution inference with memory below 9 GB.The tiles are processed sequentially in the VAE encoder and decoder, and latent patches are processed individually before aggregation.Source: A. Inference for Arbitrary Resolution · p. 12
• Quantized adaptive instance normalization corrects observed high-resolution color shifts by aligning reconstructed-image channel means and variances with the originals.The correction adds 96 bits per image; quantizing the statistics to 16 bits does not significantly affect correction performance.Source: A. Inference for Arbitrary Resolution · p. 12

• The entropy model estimates the quantized latent’s distribution for arithmetic coding using a hyperprior module and an autoregressive context model.The hyperprior Φhyper is obtained from y using transforms ha and hs.Source: B. Network Structure · p. 12
• The 16-bit color fix brings clear visual refinement with negligible bitrate increase.Source: Figure 8 · p. 12
• A four-step quadtree-partitioned autoregressive process estimates Gaussian parameters for ˆy using the hyperprior and previously decoded groups.Latent residual prediction is also incorporated to alleviate quantization error.Source: B. Network Structure · p. 12

• Runtime analysis measures encoding and decoding modules on a single RTX 3090 GPU, averaged on Kodak images.The measured components include the VAE and auxiliary encoders and decoders, transforms, entropy coding, and one-step denoising.Source: C. Runtime Analysis · pp. 12–13, Table 6 · p. 13
• The entropy model uses a four-step autoregressive process with quadtree partitioning and latent residual prediction.Figure 9 illustrates the entropy model and four-step decoding process.Source: Figure 9 · p. 13
• StableCodec’s transform coding and entropy estimation are reported as much faster than ELIC’s, with coding speed comparable to mainstream neural codecs.The paper attributes the efficiency to transform coding at 16× and entropy estimation at 64×, compared with ELIC’s 16× transform coding and entropy estimation.Source: C. Runtime Analysis · pp. 13–14

D. User Study

• The user study compared StableCodec with seven generative codecs on Kodak at similar average bitrates.Each case presented the ground truth and eight reconstructions; participants chose the one most consistent with the ground truth.Source: D. User Study · p. 14
• StableCodec reconstructions were preferred in over 70% of user-study cases, based on 720 evaluations by 30 participants.Participants selected the reconstruction they found most consistent with the ground truth.Source: D. User Study · p. 14

E. Visual Performance

• On DIV2K, CLIC 2020, and USTC-TD images, StableCodec is reported to outperform competing methods in semantic consistency and textual realism while consuming fewer bits.The comparisons cover high-quality images at ultra-low bitrates.Source: E. Visual Performance · p. 14

• The paper describes PSNR and MS-SSIM comparisons on CLIC 2020 and DIV2K as supplemental to its main results.Source: F. Quantitative Results · p. 14
• The paper prioritizes FID, KID, and DISTS because pixel-level metrics are described as less suitable for assessing severely compressed images’ semantic consistency and textual realism.Source: F. Quantitative Results · p. 14
• Additional visual examples compare methods on 2K-resolution images from DIV2K.Source: Figure 12 · p. 15
• Additional visual examples compare methods on 2K-resolution images from CLIC 2020.Source: Figure 13 · p. 16
• Additional visual examples compare methods on 4K-resolution images from USTC-TD.Source: Figure 14 · p. 17, Figure 15 · p. 18