PaperlayerView exact revision on arXiv

Source-linked AI summary

WaveLLM: Wavelet-Domain LLM Entropy Modeling for Lossless Image Compression

Yue Yu, Bowei Kang, Changsheng Gao, Dong Liu, Xiaoyan Sun

arXiv:2610.22232v1cs.CV

TL;DR

Existing LLM-based lossless image compressors serialize pixels or residuals, while effective entropy modeling also depends on the predictability of the input representation. WaveLLM decomposes images into wavelet subbands and uses a shared prompted LLM with a numerical interface. It achieves 2.51, 2.24, and 2.27 bpsp on Kodak, CLIC.pro, and DIV2K, outperforming recent LLM-based compressors on all three benchmarks.

  • Problem

    Existing LLM-based lossless compressors serialize pixels or residuals, leaving heterogeneous signal patterns entangled in one prediction stream; entropy modeling also depends on input predictability.

    Source: §1 · p. 2

  • Method

    WaveLLM models wavelet coefficients with an LLM, using a numerical interface and task prompts to adapt a shared model across subbands.

    Source: §1 · p. 2

  • Results

    WaveLLM achieves 2.51, 2.24, and 2.27 bpsp on Kodak, CLIC.pro, and DIV2K, respectively, outperforming existing LLM-based compressors on all three benchmarks.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The reported results suggest wavelet subbands are more suitable for LLM probability modeling than the tested flat pixel and residual sequences.

    Source: §5 · p. 8

    Both transforms in the stated setup are lossless.

    Source: §3.2 · p. 3

Abstract

from arXiv · show

Lossless image compression fundamentally relies on accurate entropy modeling, driving a recent shift toward large language model (LLM)-based methods. However, existing approaches mainly emphasize the modeling capacity of LLMs while directly serializing images into pixel or residual sequences, leaving heterogeneous signal patterns entangled within a single prediction stream. In this paper, we argue that effective entropy modeling depends not only on model capacity, but also on the predictability of the input representation. Motivated by this insight, we propose WaveLLM, a wavelet-domain LLM-based entropy model for lossless image compression. By decomposing an image into multi-level wavelet subbands, WaveLLM converts a heterogeneous prediction problem into a set of more structured and statistically homogeneous subproblems that are more amenable to autoregressive probability estimation. To bridge numerical coding symbols with the LLM's native text token space, we introduce a dedicated numerical interface featuring value embeddings and a coefficient prediction head. Furthermore, a task-adaptive prompting strategy is designed to enable a single shared model to adapt to diverse subbands within a unified framework. Built on Llama-3.2-3B with LoRA adaptation, WaveLLM achieves 2.51, 2.24, and 2.27 bpsp on Kodak, CLIC.pro, and DIV2K, respectively, outperforming the PixelCNN-based codec using the same wavelet transform by 0.43, 0.36, and 0.46 bpsp, and surpassing recent LLM-based lossless compressors on all three benchmarks.

1 Introduction

WaveLLM argues that entropy modeling depends on input predictability as well as model capacity. It decomposes images into wavelet subbands and adapts a shared LLM to them, achieving lower coding rates than recent LLM-based compressors on three benchmarks.

1 Introduction

  • Entropy-model accuracy depends on input predictability as well as model capacity; entangled statistical regimes make accurate probability estimation more difficult.Source: §1 · p. 2
  • Wavelet decomposition separates coarse low-frequency structure from directional high-frequency details, making the prediction problem more structured and statistically homogeneous.This contrasts with pixel and residual streams that entangle heterogeneous patterns.Source: §1 · p. 2
  • A numerical interface maps wavelet coefficients into a compact local symbol space, embeds values, and predicts valid coefficient values with a dedicated head.The design lets a pretrained LLM operate on numerical wavelet symbols while preserving numerical validity.Source: §1 · p. 2
  • Task prompts condition one shared entropy model on subband identity and decoded context, adapting predictions to distinct subband distributions without separate models.Source: §1 · p. 2
  • WaveLLM achieves 2.51, 2.24, and 2.27 bpsp on Kodak, CLIC.pro, and DIV2K, respectively, and outperforms recent LLM-based compressors on all three benchmarks.Figure 1 compares lossless coding rates, for which lower is better.Source: §1 · pp. 2–3, Figure 1 · p. 2

2 Related Work

Prior work spans traditional, learned, and LLM-based lossless compression, while WaveLLM places a shared prompted LLM entropy model within a reversible wavelet-coding framework.

2.1 Lossless Image Compression

  • Classical and learned lossless codecs use varied prediction, latent-variable, flow, residual, and bit-plane approaches to estimate probabilities over discrete image signals.CALLIC reports strong learned-method results through content-adaptive per-image optimization.Source: §2.1 · p. 3

2.2 Wavelet-Domain Codecs

  • Wavelet codecs exploit reversible transforms and subband-dependent coefficient statistics; WaveLLM replaces PixelCNN-style entropy modeling with a shared prompted LLM over codec-defined subband tasks.Source: §2.2 · p. 3

2.3 LLMs for Lossless Image Compression

  • Recent LLM compressors code pixels [5] or residuals from lossy reconstructions [8], whereas WaveLLM uses decoded wavelet-codec context and a dedicated local value-ID interface.The cited pixel- and residual-domain methods report Llama-3-8B backbones; WaveLLM uses a LoRA-adapted 3B backbone.Source: §2.3 · p. 3

3 Method

WaveLLM applies a reversible wavelet transform and codes subbands in causal order, using task prompts and a numerical interface to produce coefficient probabilities. A shared LoRA-adapted LLM models the tasks, with patch-based training across subbands.

3.1 Overall Framework

  • A reversible YCoCg and four-level 5/3 wavelet pipeline produces 13 subbands, which are coded in a fixed order and exactly reconstructed through the inverse transform.The deepest low-frequency band is coded first, followed by high-frequency bands from deepest to shallowest.Source: §3.1 · p. 3

3.2 Wavelet-Domain Probability Factorization

  • LL4 is modeled first; at each level, HL_l, LH_l, and HH_l are predicted in sequence from already decoded same-level bands.For LL4, the MED predictor is applied first and the LLM models its residuals; high-frequency coefficients are modeled directly.Source: §3.2 · p. 4

3.3 Task Prompt Design

  • Each subband task uses a prompt naming its type, available context, and prediction objective, while serialized values provide the causal context and target.For LL4, the prompt is paired with target MED residuals; tasks with multiple context bands serialize them in causal order with delimiters.Source: §3.3 · pp. 4–5

3.4.1 Coding-Oriented Numerical Interface.

  • Wavelet coefficients are mapped to ordered local IDs, embedded through a trainable value pathway separate from frozen text embeddings, and predicted over the local value range.The fixed offset is 512 and the local range has K = 2048 values; a dedicated value prediction head replaces the full-vocabulary language-model head for target positions.Source: §3.4.1 · p. 5, Figure 4 · p. 5

3.4.2 Backbone Adaptation and Training.

  • A shared set of LoRA adapters on Llama-3.2-3B serves all subband tasks and four decomposition levels, with training sampling patches across the 13 tasks.Training images are transformed at full resolution before patches are cropped from wavelet-domain subbands.Source: §3.4.2 · p. 5

4.1 Experimental Settings

WaveLLM is trained on three natural-image datasets and evaluated on Kodak, CLIC.pro, and DIV2K. The experiments use a two-stage training schedule, specified implementation settings, and traditional, learned, and LLM-based baselines.

4.1.1 Datasets.

  • Training first uses the 607,715-image NIC corpus, then finetunes on DIV2K (800 images) and Flickr2K (2,650 images), with evaluation images held out.A separate validation split is used for model selection.Source: §4.1.1 · pp. 5–6
  • Evaluation uses Kodak (24 images), CLIC.pro (41 images), and DIV2K validation (100 images), with rates normalized by original dimensions before padding.Images are converted to YCoCg and padded to multiples of 16 for four-level wavelet decomposition.Source: §4.1.1 · p. 6

4.1.2 Training Details.

  • The implementation uses frozen Llama-3.2-3B weights with LoRA, a local value range of K=2048 and offset o=512, and training on eight RTX 3090 GPUs.Training uses batch size 1 with gradient accumulation 16, AdamW, and a 10^-5 learning rate.Source: §4.1.2 · p. 6

4.1.3 Baseline Codecs.

  • Baselines span traditional codecs, learned lossless codecs, and LLM-based methods.The groups include PNG, JPEG XL, L3C, iWave, CALLIC, Delétang et al., P2-LLM, and the visual-prompt method.Source: §4.1.3 · p. 6

4.2 Main Results

WaveLLM reports lower rates than recent LLM-based competitors across the three benchmarks and improves over strong non-LLM baselines on Kodak. Its runtime and computational-cost measurements are implementation- and hardware-dependent.

4.2.1 Comparison with Existing Methods.

  • With a 3B backbone, WaveLLM achieves 2.51 bpsp on Kodak versus 2.83 bpsp for both 8B LLM competitors, an 11.3% reduction; on CLIC.pro/DIV2K it reaches 2.24/2.27 bpsp versus 2.35/2.25.Source: §4.2.1 · p. 6
  • 0.36 and 0.43 bpsp: WaveLLM improves over JPEG XL and iWave, respectively, on Kodak in the matched 5/3 setting.Exact reconstruction was verified on all reported test sets, with no observed value-range overflow or out-of-range indices.Source: §4.2.1 · p. 6

4.2.2 Computational Cost.

  • 65.1 s to encode and 152.8 s to decode each Kodak image: WaveLLM’s measured runtime in the reported implementation.The implementation parallelizes 13 subband tasks across eight RTX 3090 GPUs and uses about 16.2 GB peak memory per card; runtime reduction remains future work.Source: §4.2.2 · p. 6
  • Runtime comparisons across papers are qualitative because hardware, implementation details, and model sizes differ.WaveLLM’s figures are measured in the authors’ environment, while some prior-work figures are quoted from other papers.Source: §4.2.2 · p. 6

4.3 Analysis of the Proposed Design

The analyses compare coding domains, entropy-model backbones, and subband distributions. They find that the wavelet formulation performs best among tested domains, improves on PixelCNN within the same wavelet codec, and uses one shared backbone to model distinct subband statistics.

4.3.1 Why the Wavelet Domain.

  • 0.21, 0.11, and 0.17 bpsp: the wavelet formulation beats the pixel-domain variant on Kodak, CLIC.pro, and DIV2K, respectively.Under the same Llama-3.2-3B next-token framework, the wavelet-domain formulation is strongest among the tested variants; the residual variant’s rate includes lossy-base and coded-residual bits.Source: §4.3.1 · p. 6

4.3.2 Wavelet-Domain Backbone-Scale Analysis.

  • Compared with PixelCNN, WaveLLM-3B reduces rates by 0.43, 0.36, and 0.46 bpsp on Kodak, CLIC.pro, and DIV2K, respectively, from 2.94 to 2.51, 2.60 to 2.24, and 2.73 to 2.27 bpsp.The reductions are 14.6%, 13.8%, and 16.8%, respectively. WaveLLM-1B also outperforms PixelCNN, and scaling from 1B to 3B yields further gains of 0.37/0.17/0.23 bpsp.Source: §4.3.2 · p. 7
  • Across subbands, the LLM provides positive local gain over iWave; shallow Level-1 bands dominate total bitrate because they contain most coded symbols.The figure shows local gain and each subband’s contribution to Kodak01 bitrate.Source: §4.3.2 · p. 7, Figure 5 · p. 8

4.3.3 Task-Specific Distribution Analysis.

  • Predicted peak-value distributions align with actual distributions for LL4, HL3, LH3, and HH3, while the shared backbone captures their distinct statistics.LL4 is broader; the three high-frequency bands are sharply peaked around zero, with differences in tails and spread.Source: §4.3.3 · pp. 7–8, Figure 6 · p. 8

4.4 Ablation Study

The ablations test task prompts, the numerical interface, context design, and MED residual coding for LL4. Removing prompts has the largest reported single-component degradation, while each tested design choice affects coding rates.

4.4.1 Task Identity and Prompt.

  • Removing text prompts increases rates by 0.11 bpsp on Kodak and 0.21 bpsp on CLIC.pro, the largest single-component degradation.Without task-specific instructions, the shared backbone tends toward an average distribution across the 13 subband tasks.Source: §4.4.1 · p. 8, §4.4 · p. 8, Table 6 · p. 8

4.4.2 Numerical Interface.

  • Replacing the value interface with Llama’s native vocabulary increases rates by 0.07 bpsp on Kodak and 0.08 bpsp on CLIC.pro.The paper attributes the interface’s benefit to avoiding distant token IDs for neighboring coefficients, language-trained embedding domain gap, and a less compact 128K probability space.Source: §4.4.2 · p. 8

4.4.3 Context Design.

  • Using LL-only context raises rates by 0.03/0.06 bpsp on Kodak/CLIC.pro, while adding parent-level HF bands raises them by 0.23/0.27 bpsp.The authors report that the results support the current design based on same-level decoded context.Source: §4.4.3 · p. 8

4.4.4 Low-Frequency MED Residuals.

  • Omitting MED residual coding for LL4 raises rates by 0.01 bpsp on Kodak and 0.03 bpsp on CLIC.pro.LL4 contributes about 0.016 bpsp to Kodak01’s total rate; MED makes its representation more concentrated and closer to high-frequency subbands.Source: §4.4.4 · p. 8

5 Conclusion

WaveLLM combines wavelet subbands, a numerical interface, prompts, and decoded context in a LoRA-adapted Llama-3.2-3B, with strong results on three benchmarks.

5 Conclusion

  • 2.51 bpsp on Kodak, 2.24 bpsp on CLIC.pro, and 2.27 bpsp on DIV2K; WaveLLM outperforms existing LLM-based compressors on all three benchmarks.It remains competitive with strong lossless codecs.Source: §5 · p. 8
  • WaveLLM combines a coding-oriented numerical interface, task prompts, and same-level decoded context in a LoRA-adapted Llama-3.2-3B backbone.Source: §5 · p. 8
  • The results suggest wavelet subbands are more suitable for LLM probability modeling than the tested flat pixel and residual sequences.The paper attributes more accurate entropy estimation to separating image content into subbands and guiding predictions with prompts and decoded context.Source: §5 · p. 8
  • Future work will consider distillation, quantization, and near-lossless coding.Source: §5 · p. 8

Related papers

Loading 2610.22232v1…