PaperlayerView exact revision on arXiv

Source-linked AI summary

NeuralZip: Reusable Setup for Fast Lossless Compression

Martín Bravo, Samuel Horváth, Gonzalo Navarro, Andrés Abeliuk

arXiv:2610.09916v1cs.LG

TL;DR

Lossless compression can reduce model-weight storage without changing floating-point values, but repeated analysis and code construction add overhead. NeuralZip reuses exponent statistics through shared Huffman codes and packed exponent tuples, and reports reusable setup transfer alongside faster exact compression.

  • Problem

    Lossless model compression seeks to reduce storage and movement while preserving exact weights, but existing procedures can incur substantial computational overhead.

    Source: §1 · p. 1

  • Method

    NeuralZip groups similar exponent distributions to share Huffman codes and packs recurring adjacent exponent tuples to exploit local redundancy.

    Source: §1 · p. 2

  • Results

    NeuralZip combines shared codebooks with exponent packing and provides setup transfer for compatible target models without target-side codebook construction.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Reusable statistical preparation can accelerate exact compression at a similar representation size, while deployment must account for setup construction and statistical compatibility.

    Source: §6 · p. 8

    Deployment must assess whether a precomputed setup is statistically compatible with target weights during training and transfer.

    Source: §6 · p. 8

Abstract

from arXiv · show

Lossless compression can reduce the storage and movement of model weights without changing their floating-point values, but repeated statistical analysis and code construction add computational overhead. We study whether the statistical structure of exponents can be prepared once and reused. For this, we introduce NeuralZip, which groups chunks with similar exponent distributions, shares Huffman codes, and selectively represents recurring exponent tuples using packed exponents, thereby achieving additional moderate compression ratios. A setup chooses these representations before subsequent encodings, while every encoding still processes the current tensor values. In floating-point model checkpoints, post-setup compression is 1.81-21.33$\times$ faster than the baselines and achieves exact bit-to-bit reconstruction. We show that this setup can be precomputed and transferred from another compatible architecture, preserving similar compression ratios and avoiding the need to amortize setup costs. Therefore, compression adaptation is transferable and reusable. Training checkpoints demonstrate continued reuse as the weights evolve. Finally, GPU experiments reduce active memory usage by up to 27.5$\%$ while reproducing the logits exactly.

1 INTRODUCTION

NeuralZip addresses the computation overhead of lossless model compression by reusing statistical structure in floating-point exponents. It combines shared codebooks and packed exponent tuples, and supports transferring setups to compatible architectures.

1 INTRODUCTION

  • ZipNN and DFloat11 preserve model parameters exactly, but repeated statistical analysis and code construction can make compression computationally expensive.Recomputing similar codebooks as if they represented different distributions motivates exploiting additional statistical regularity.Source: §1 · p. 2
  • Exponent distributions vary across parameter blocks yet form similar groups, making precomputed codebook sharing a potential way to accelerate compression.Sharing can increase the exponent storage ratio, motivating a complementary way to reduce storage.Source: §1 · p. 2
  • Packing adjacent exponent values into compound symbols exposes local regularities that can offset the storage cost of shared codebooks.The paper calls this property local exponent redundancy.Source: §1 · p. 2
  • NeuralZip combines Jensen–Shannon clustering for Huffman-code sharing with packed exponents, using both observed properties in evaluated architectures.The paper identifies different exponent-distribution groups and local redundancy in floating-point model weights.Source: §1 · p. 2
  • A setup-transfer mechanism lets compatible target models use compression setups without constructing codebooks on the target side.The paper also motivates fast, exact compression for model weights already available in model hubs and on end devices.Source: §1 · p. 2

2 RELATED WORK

Prior work establishes lossless compression and adaptive coding for model weights, while NeuralZip combines reusable distribution sharing with local exponent-pattern coding.

2 RELATED WORK

  • ZipNN and LMC adapt codes to input blocks, raising the question of when code construction can be shared instead of repeated.General-purpose compressors combine dictionary matching with entropy coding, while weight-aware methods expose floating-point structure explicitly.Source: Model compression · p. 2

Model compression.

  • Shannonic prepares reusable tables for quantized tensors, whereas NeuralZip discovers sharing groups for bit-exact floating-point weights and adds packed exponent symbols alongside singletons.MM-RePair exploits repeated patterns with grammars; NeuralZip uses a shared Huffman alphabet and retains exact reconstruction.Source: Model compression · p. 2
  • Prior compressed-execution work includes ANS-coded exponents, GPU Huffman reconstruction, FP8 exponent coding, and Huffman decompression on custom accelerators.These methods establish exponent compression and on-demand reconstruction on which NeuralZip's GPU inference builds.Source: Model compression · pp. 2–3

3 PRELIMINARIES

Exponent statistics in model weights are both skewed and locally structured, while chunk histograms also form similar groups. These properties motivate shared codebooks and packed exponent tuples, with ideal coding-cost bounds supporting the design.

Skewed Exponent field

  • BF16 exponent entropy is 2.691–3.345 bits, compared with 1 bit for sign and 6.97 bits for mantissa, making exponents the field with unused average capacity.BF16 allocates 1 sign bit, 8 exponent bits, and 7 mantissa bits; exponent values use about 2.7–3.4 of their 8 allocated bits on average.Source: §3 · p. 3

Heterogeneity of the exponents

  • 14.3–49.5% of sampled chunk comparisons fall below a Jensen–Shannon distance of 0.05, while the histograms also include widely separated pairs.The study sampled 48 chunks of size 256 KiB from the 12 largest tensors across six architectures.Source: §3 · pp. 3–4, Figure 2 · p. 4
  • Greedy clustering shares codebooks among compatible chunks whose exponent-histogram distance to a centroid is at most τ, set to 0.05.Sharing can increase the exponent storage ratio while enabling faster compression with precomputed tables.Source: §3 · p. 4

Local exponent redundancy

  • Packing two and four consecutive exponent values lowers empirical entropy by 0.007–0.235 and 0.016–0.429 bits per exponent, respectively.Non-overlapping tuples are represented as compound symbols; Figure 3 reports local exponent redundancy entropy.Source: §3 · p. 4, Figure 3 · p. 4

Ideal coding-cost bounds

  • The ideal zero-order coding cost does not increase when sequences are refined into chunks or when divisible-length sequences are packed into tuples.The combined bound excludes raw tails, escapes, integer code lengths, headers, metadata, fallbacks, and clustering costs, so practical size and time gains require measurement.Source: §3 · pp. 3–4

4 OUR METHOD: NEURALZIP

NeuralZip prepares and freezes shared Huffman codebooks and packed-exponent representations before compression, then applies them to current tensor values while preserving exact reconstruction.

4.1 A reusable setup

  • Compatible chunks are clustered by exponent histograms for shared Huffman codes, while selected adjacent-exponent pairs become packed symbols to reduce storage.The setup also prepares source-only codebook banks for transfer and selects tables before subsequent encodings.Source: §4.1 · pp. 4–5

4.2 Compression and exact decompression

  • Compression encodes current exponents with assigned shared tables, stores sign and mantissa bits verbatim, and uses raw fallback when coding is not worthwhile.Decompression restores singleton or packed exponents and recombines them with preserved bits to recover the original floating-point bit patterns.Source: §4.2 · p. 5

4.3 GPU Inference

  • The GPU path keeps coded exponents and lookup/restart metadata on-device, reconstructs exact weights in a reused workspace, and passes them to standard matrix multiplications.Weights are decompressed before each transformer block, and decompression is paid on each forward pass.Source: §4.3 · pp. 5–6

4.4 Setup transfer

  • Setup transfer reuses source tensor and global codebooks but not chunk assignments, selecting among frozen bank tables or singleton fallbacks using target samples.Target samples select among prebuilt tables; they do not build new tables or vocabularies, and raw fallback remains available.Source: §4.4 · p. 6

5 EXPERIMENTS

The experiments assess reusable compression speed, exponent packing, setup reuse across models and evolving weights, and compressed GPU inference. They benchmark compression and inference under specified hardware and timing procedures.

5.1 Experimental setup

  • Benchmarks use 12 CPU threads unless specified, report medians after warm-up, and measure setup separately; compression timing excludes loading, disk I/O, and bit-wise verification.CPU measurements use fixed pinned cores, while GPU measurements use four ASUS Ascent GX10 systems.Source: §5.1 · p. 6

5.3 Reusing setups

  • Transferred setups provide 1.77–16.65× compression-time gains for a 0.01–4.90 percentage-point increase in exponent storage ratio, but source compatibility matters.Targets avoid setup construction when a frozen compatible setup is available, although its source construction cost has already been incurred.Source: §5.3 · p. 6

5.2 Model compression

  • Compression is 1.81–21.33× faster than the fastest baseline, with exponent storage ratio changes of -0.22 to 1.49 percentage points.Speedups are higher in convolutional models, at 12.24–21.33×, and benefits vary with exponent heterogeneity and local redundancy.Source: §5.2 · p. 6, Table 2 · p. 7

5.4 Sharing and packing contribution

  • Combining sharing and packing gives the lowest exponent storage ratio among evaluated variants, while sharing reduces codebook overhead and packing reduces payload size.Sharing Only usually compresses faster than NeuralZip but has a higher ratio; Packing Only improves that ratio but costs more setup and compression time.Source: §5.4 · p. 7

5.5 Compressed GPU inference

  • GPU runs reduce live memory by up to 27.5%, while inference with NeuralZip’s own setups ranges from 11.8% faster to 8.3% slower than DFloat11.Weights and outputs are verified bit-by-bit against BF16.Source: §5.5 · p. 7

5.6 Training checkpoints storage

  • Across training checkpoints, NeuralZip is 1.33–3.74× faster than ZipNN with similar exponent storage ratios, though later distribution drift can worsen ratios.Each model family’s setup is built from its earliest checkpoint and kept frozen; the experiments do not establish a refresh schedule.Source: Figure 6 · p. 8, §5.6 · p. 8

6 DISCUSSION AND CONCLUSION

The conclusion presents reusable statistical preparation as a way to accelerate exact compression at similar representation size, with deployment requiring attention to setup construction and target compatibility.

6 DISCUSSION AND CONCLUSION

  • Exponent heterogeneity motivates codebook sharing, while local exponent redundancy offers size improvements in the proposed lossless compressor.The authors suggest these observations may motivate further work on reusable statistical structures for lossless compression.Source: §6 · p. 8

AI use statement

  • The work uses generative AI tools for implementation and writing-related tasks, with AI-assisted code checked through round-trip, bit-wise, integrity, and measurement validation.The authors state they reviewed the AI-assisted work and take responsibility for the final content.Source: AI use statement · p. 8

A ADDITIONAL EXPERIMENTS

Additional experiments extend the comparison to 18 models across vision, language, audio, diffusion, and mixture-of-experts architectures.

A ADDITIONAL EXPERIMENTS

  • Across 18 models spanning vision, language, audio, diffusion, and mixture-of-experts, NeuralZip is faster than baselines with setup available, while transfer ratios depend on the source model.The reported comparisons retain cases where setup transfer increases storage.Source: Table 3 · p. 12, A ADDITIONAL EXPERIMENTS · p. 12

B.1 Optimizers entropy.

Exponent entropy changes differently across optimizers, while JS-based grouping offers ratios close to or slightly better than per-tensor grouping with fewer tables.

B.1 Optimizers entropy.

  • Adam increases entropy more in the MLP and transformer experiments, while AdamW and Muon remain relatively stable.Entropy was tracked over 1,000 updates on BF16 copies of unique trainable parameters across MLP, ResNet-18, ViT-Base, and GPT-2.Source: §B.1 · p. 14

B.2 Level selection.

  • Global grouping has a low setup cost but the highest exponent ratio, while per-chunk grouping prepares many tables and takes substantially more setup time without the smallest archive.Source: §B.2 · p. 14
  • JS-based grouping reaches ratios close to, or slightly better than, per-tensor grouping while preparing fewer tables.The experiments compare global, per-tensor, per-chunk, and JS-based grouping.Source: §B.2 · p. 14

B.3 Sharing Only vs Packing Only trade-off.

Sharing and packing have different compression-time, ratio, setup, and metadata trade-offs; the default JS threshold balances setup time and ratio, while setup costs are concentrated in grouping.

B.3 Sharing Only vs Packing Only trade-off.

  • Combining sharing and packing gives the lowest exponent ratio across all six models.Sharing Only is fastest but has a higher exponent ratio; Packing Only improves the ratio relative to Sharing Only, at the cost of more setup work and larger codebook metadata.Source: §B.3 · p. 15
  • Approximately 6,283 KiB in Packing Only falls to 82 KiB in NeuralZip for Qwen 3.5 0.8B codebook size.Sharing reduces the metadata cost of using packed codebooks.Source: §B.3 · p. 15

B.4 Threshold sweep.

  • The default threshold τ = 0.05 is close to the best observed ratio for every tested model while avoiding the high setup cost of stricter thresholds.Very small thresholds require much more setup time and do not necessarily improve the final ratio; larger thresholds shorten setup but can worsen compression.Source: §B.4 · p. 15, Figure 8 · p. 16

B.5 Setup costs.

  • JS search and group creation account for approximately 49–77% of setup time across all six models, whereas Huffman construction and refitting take a much smaller fraction.The main setup cost is finding compatible groups; reusing a frozen setup avoids repeating this work for subsequent compression calls.Source: §B.5 · p. 16
  • Exponent extraction and histogram computation take O(N) time, while greedy codebook sharing costs O(CK) for a fixed exponent alphabet.Here N is the number of weights, C the number of chunks, and K the number of resulting clusters.Source: §B.5 · p. 16

Related papers

Loading 2610.09916v1…