PaperlayerView exact revision on arXiv

Source-linked AI summary

Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter

Pier-Jean Malandrino

arXiv:2609.35465v1cs.LG

TL;DR

Leech-lattice quantization offers strong quality at two bits per weight, but its enormous codebooks and the earlier kernel’s expansion of codes make serving costly. Tetra uses a trellis-based codebook and a full-model recipe to reduce memory, with Qwen3 files evaluated against AWQ and FP16.

  • Problem

    Leech-lattice codes can represent 24 weights in 48 bits, but their more than 10^14 points are too numerous for a lookup table, and the earlier kernel read 4.804 bits per weight for two bits of code.

    Source: §1 · p. 1

  • Method

    Tetra uses a Golay trellis and one shared table to decode lattice blocks, then combines retrained row scales, selected int4 matrices, and 4-bit embedding tables for complete models.

    Source: §1 · p. 1, §8 · p. 11

  • Results

    At about 2.7 bits per parameter, the Qwen3 files trail 4-bit AWQ by 4.76, 4.21, and 2.46 MMLU points at 4B, 8B, and 14B, respectively.

    Source: §8 · p. 11

  • Takeaways & Limitations

    With about as much weight memory or less, the 8B and 14B files score above AWQ’s 4B and 8B on MMLU and are not separated from them on GSM8K.

    Source: §8 · p. 11

    The int4 choices were made on MMLU test questions, creating an unmeasured selection bias in the MMLU scores; GSM8K is the only benchmark that chose nothing.

    Source: §7 · p. 11

Abstract

from arXiv · show

Leech-lattice quantization gives good quality at two bits per weight, but its codebooks hold more than 10^14 points, too many for a lookup table. Our earlier kernel expanded the codes at load time and read 4.804 bits per weight from GPU memory for 2 bits of code. We present Tetra, a new codebook on the same lattice. A 24-weight block still takes 48 bits, most of which index a 64-state trellis of the Golay code and one shared 16 KiB table. The kernel decodes a block with six table loads and two small lookups inside the matrix-vector product, and reads 2.148 bits per weight. For full models, we retrain one scale per matrix row, store the matrices that lose the most as 4-bit integers, and pay for them with 4-bit embedding tables. Our Qwen3-4B, 8B and 14B files hold 2.73, 2.70 and 2.73 bits per parameter over the whole model. They score 63.37, 69.58 and 75.66 on the full MMLU test set, 4.76, 4.21 and 2.46 points below 4-bit AWQ at 5.3 to 6.0 bits per parameter. They generate 113.8, 95.0 and 57.2 tokens per second in our engine. On GSM8K, through the served kernel, they lose 9.63, 4.62 and 3.26 points to FP16. At 4B our file scores 23.6 points above llama.cpp's IQ2_XXS (2.48 bits per parameter). Every number we measured for a table or figure comes from one NVIDIA L40S GPU. We preregistered the main experiments.

1 Introduction

Tetra addresses the mismatch between compact Leech-lattice codes and the larger memory reads required by the earlier kernel. It combines a new trellis-based codebook with changes for complete models, which are evaluated against AWQ and other baselines.

1 Introduction

  • A 14-billion-parameter model requires 29.5 GB for FP16 weights alone, exceeding the capacity of a 24 GB consumer GPU.At batch 1, each token reads every weight once, so weight size sets both memory use and speed.Source: §1 · p. 1
  • The earlier kernel expanded each 48-bit code at load time and read 4.804 bits per weight, more than the 4.179 bits per weight of 4-bit AWQ.The codebook contains more than 10^14 points, too many for a lookup table.Source: §1 · p. 1
  • Tetra decodes each 48-bit block through a 64-state Golay trellis and one 16 KiB table, reading 2.148 bits per weight.The kernel uses six table loads and two small lookups inside the matrix-vector product.Source: §1 · p. 1
  • Complete files combine retrained row scales, int4 storage for the projections that lose the most, and 4-bit embedding tables to pay for them.The recipe leaves the lattice codes unchanged and is measured at 4B, 8B, and 14B.Source: §1 · p. 1
  • With about as much weight memory or less, the 8B and 14B files score above AWQ’s 4B and 8B on MMLU and are not separated from them on GSM8K.This comparison was chosen after measuring.Source: §1 · p. 2
  • The files do not match AWQ on quality; at 14B, their MMLU gap to AWQ is about half the 4B gap, and they use about half its memory: 2.7 versus 5.3 to 6.0 bits per parameter.On GSM8K, run through the kernel, they lose more to FP16 than on MMLU when counted in errors, at every size.Source: §1 · p. 2

2 Background and related work

The background contrasts common weight-only quantization with multi-weight codebooks used below three bits. Prior Leech-lattice work achieves strong two-bit quality, while its earlier serving layout expands codes before the kernel reads them.

2 Background and related work

  • At batch 1, GPTQ quantizes one matrix column at a time and propagates rounding error using second-order statistics from calibration text; AWQ rescales input channels before 4-bit rounding.The passage identifies grouped-128 AWQ served by Marlin as the usual 4-bit baseline.Source: §2 · p. 2
  • Below three bits, vector-quantization methods encode several weights as a codebook choice, using lattices, trellises, learned tables, or lookup grids.QTIP combines a state-hashed 2 KiB table with computed codes; AQLM and VPTQ decode learned codebooks of at most 216 entries.Source: §2 · p. 2
  • Van der Ouderaa et al. [24] encode 24-weight blocks on Λ24 in 48 bits; their reported Qwen3-4B MMLU score is 60.7 without fine-tuning and 62.8 with it.Those model scores use several lattice shells, while their GPU kernel decodes only one shell.Source: §2 · p. 2
  • The earlier Planes14 kernel unfolded each code at load time into a 112-bit record, then read 4.804 bits per weight from GPU memory.The paper replaces this layout.Source: §2 · p. 2

3 The Tetra codebook

Tetra turns Golay-code paths into addresses for a Leech-lattice quantizer, using rank vectors to share a table across sections. Its compact decode trades some quantization quality for a six-load path, while the model-level comparison remains inconclusive.

3.1 The lattice and the quantizer

  • The encoder rotates weights and uses a GPTQ-style loop to choose a nearby lattice point under a length cap, searching non-exhaustively offline on a CPU.It uses two trial scales, keeps a few candidates per section, and joins the best across the trellis; each output row has one scale.Source: §3.1 · p. 3

3.2 Three octads and a trellis

  • The three disjoint octads split the 24 coordinates into sections, yielding a 64-state trellis whose 4,096 paths each encode one Golay codeword.No split of G24 into three sections of eight has fewer states at a cut.Source: §3.2 · p. 3

3.3 State, rank vectors, and one table for three sections

  • Rank vectors encode section coordinates in 32-bit rows; sorting by cost and class lets one table serve every section and recover parity from row position.The table holds the 2,048 cheapest vectors in each class; the middle section’s index also reveals its parity.Source: §3.3 · pp. 3–4

3.4 The word and the decode

  • Six loads decode the 24 values, with only one read waiting on another; any 48-bit word is legal and decodes to a Leech-lattice point.A random-word benchmark can therefore check every decode against a reference.Source: §3.4 · p. 4

3.5 What the construction costs

  • The rank-based split bounds each section separately, making the encoder search three 8-dimensional regions rather than Planes14’s single 24-dimensional ball.This split enables the six-load decode and is the format’s stated cost.Source: §3.5 · p. 5
  • 9.2% more mean squared error: Tetra reaches 88.80% retention versus Planes14’s 91.98% on Gaussian blocks; the Qwen3-0.6B first-layer gap is 3.2 points.The measured shaping-gain difference predicts 8.81% more mean squared error for the product region, though the actual regions are neither balls nor continuous.Source: §3.5 · p. 5
  • At equal point density, E8 has 9.0% more mean squared error than the Leech lattice, whose Golay and parity constraints the three-section construction retains.The reported normalized-second-moment ratio is 0.071682/0.065771 = 1.0899.Source: §3.5 · p. 5
  • On Qwen3-4B’s 2,280-question sample, bare Tetra is 2.11 MMLU points below Planes14, but the interval [−0.95, +5.17] and p = 0.11 cannot separate them.Each file came from one calibration draw, and three Planes14 draws spread MMLU over 5.83 points on this sample.Source: §3.5 · p. 5

4 From the codebook to a served file

The served-file recipe builds on an encoding that already stores v_proj in int4, while leaving the decoder and remaining lattice codes unchanged. It retrains row scales and combines selected int4 projections with 4-bit embedding tables to produce complete model files.

4.1 What a file holds, and how we count it

  • The files combine Tetra projections with int4 records for selected matrices, while keeping RMSNorm weights in f16 and row tails in the file.At 4B the output head is tied to the embedding; at 8B and 14B it is stored separately.Source: §4.1 · p. 5, §4.3 · p. 6
  • Memory is counted over the whole model, including embeddings, tails, scales, and int4 records, but excluding the KV cache and activations.For the authors’ files, the count uses the widths held on the GPU.Source: §4.1 · p. 5

4.2 Training the row scales

  • Retraining only the per-row scales with an FP16 teacher and KL loss adds 3.15, 3.29, and 1.67 MMLU points at 4B, 8B, and 14B, without adding bits.The lattice codes remain fixed; the 8B and 14B runs take 1.1 h and 1.8 h on one H200.Source: §4.2 · pp. 5–6
  • At 4B, moving v_proj from Tetra to int4 gained 1.73 MMLU points for 0.05 bits per parameter in an earlier encoding; the sealed files also use selected o_proj and down_proj matrices in int4.The sealed 8B file adds only down_proj, while 4B and 14B also retain o_proj in int4.Source: §4.3 · p. 6

4.4 4-bit embedding tables

  • Storing embedding tables in 4 rather than 8 bits frees 0.19, 0.62, and 0.78 GB at 4B, 8B, and 14B, paying for the int4 projections while reducing bits per parameter.The tables use groups of 64 with an f16 scale and offset per group.Source: §4.4 · p. 6

5 The kernel

The fused kernel decodes Tetra blocks inside the matrix-vector product without expanding their codes, reading fewer bits per weight than AWQ or the earlier layout. Its lower byte count does not translate into lower measured time than AWQ, and the tile sweep shows performance varies across configurations.

5.1 Decoding inside the matrix-vector product

  • One warp computes each output row, and the fused kernel decodes each block while multiplying, then applies the row scale and adds the unquantized tail.A block decode uses six table loads and two small lookups; the source count is about 150 instructions plus about 20 for gain and norm.Source: §5.1 · p. 6
  • The kernel reads the 48 bits stored per block without expanding the code at load time; loading only reorders words and pads rows.The sealed 4B file loads in 4.1 s, compared with 72.9 s for its dense f16 reconstruction.Source: §5.1 · p. 6

5.2 Bytes and time on the projections

  • 2.148 bits per weight for Tetra is about half the 4.179 for 4-bit AWQ and 4.804 for Planes14, but Tetra takes 3.424 ms versus AWQ’s 3.261 ms.Tetra’s time is a lower bound because its benchmark arm covers fewer matrices.Source: Table 2 · p. 7, §5.2 · p. 7

5.1 Decoding inside the matrix-vector product

  • The int4 projections use a separate kernel; additional kernels serve the 4-bit embedding tables, and another rotates activations online.The rotation is performed 96 times per token at 4B, once per group of projections that share a rotation.Source: §5.1 · pp. 6–7
  • Correctness checks matched every tested 4B output row against an f64 recomputation within tolerance, and the kernel and dense path generated the same 256 greedy tokens at 4B and 8B.At 14B, the paths first differ at token 78; the authors note that their preregistered rule treats divergence at or after token 32 as not a defect.Source: §5.1 · p. 7

5.2 Bytes and time on the projections

  • At tile sizes 128, 64, and 32, Tetra’s median pass took 4.078, 3.423, and 3.553 ms; the authors serve tile 64 on sm_89.They attribute Tetra’s larger timing range to L1 pressure but say they have not proven the cause.Source: §5.2 · p. 7

6 Experiments

The experiments evaluate sealed Qwen3 files against FP16, AWQ, and 2-bit formats, measuring quality, memory, speed, and served-kernel reasoning. They also test whether larger files fit at similar memory budgets and which construction steps improve quality.

6.1 Setup

  • The evaluation uses Qwen3-4B, 8B, and 14B encoded from the same 131,072 DCLM-Edu tokens, with MMLU paired over 14,042 questions and GSM8K over 1,319 problems.MMLU is scored on dense reconstructions; GSM8K runs through the served kernel, while speed and memory use specified engine-specific settings.Source: §6.1 · p. 8

6.2 Main results

  • The sealed files trail AWQ on MMLU by 4.76, 4.21, and 2.46 points at 4B, 8B, and 14B, while using 45–52% of AWQ’s bits per parameter.If AWQ also used 4-bit embedding tables, the files would use 64–65% of its bits per parameter.Source: §6.2 · p. 8, Table 3 · p. 9
  • At 4B, the file scores 23.6 MMLU points above IQ2_XXS for 0.25 more bits per parameter, while comparisons to cited lattice methods use different harnesses and rate conventions.The cited comparison places the 63.37 score only roughly, not as a ranking.Source: §6.2 · p. 8, Table 3 · p. 9
  • In the same engine, sealed-file speeds exceed dense-reconstruction speeds by 1.27, 1.47, and 1.61 times at 4B, 8B, and 14B when both use f16 embedding and output-head tables.The authors do not compare speeds across engines because that would mix format and engine effects.Source: §6.2 · pp. 8–9

6.3 Reasoning: GSM8K through the served kernel

  • On GSM8K, all six sealed-file versus baseline gaps are separated from zero, and the files make 2.22, 1.69, and 1.69 times as many errors as FP16 at 4B, 8B, and 14B.Compared in points, the GSM8K loss exceeds the MMLU loss only at 4B; at every size the error ratio is larger on GSM8K.Source: §6.3 · p. 9, Table 4 · p. 9

6.4 One size up for about as much memory

  • On MMLU, the 8B file scores 1.44 points above AWQ’s 4B with 0.10 GB more weights, and the 14B scores 1.87 points above AWQ’s 8B with 1.05 GB less.These same model pairs are not separated on GSM8K; the 14B file also scores 5.52 MMLU points above FP16’s 4B while using 5.04 GB versus 8.04 GB.Source: §6.4 · p. 10, Table 5 · p. 10

6.5 Where the gain comes from

  • Both construction steps improve MMLU at every size, and the 8B file scores 69.58 at 2.6953 bits per parameter with int4 on down_proj layers 10–26.That choice scores 0.77 points above the tested alternative; the 4B and 14B files retain int4 on o_proj, and the 8B choice was not tested at those sizes.Source: §6.5 · pp. 10–11, Table 6 · p. 10
  • The int4 choices carry unmeasured selection bias because they were made using MMLU test questions, while GSM8K selected none of the choices.At 8B, the better of two files was retained under a preregistered rule.Source: §6.5 · p. 11

7 Limitations •

The evaluation is limited to one GPU, one calibration draw per size, and short batch-1 requests; MMLU and GSM8K also use different inference paths.

7 Limitations •

  • Every measured table or figure number comes from one NVIDIA L40S; Tetra was not run on A100, and on sm_120 it ranged from 0.82 to 1.03 times Planes14 speed across tiles.The A100 result cited is for earlier lattice kernels, none of which ran faster than FP16 at tile 128.Source: §7 · p. 11
  • Each size uses one calibration draw, while three Planes14 draws at 4B varied MMLU more than any Table 6 gain; that spread is excluded from intervals for gaps to FP16 and AWQ.The fixed-file comparisons in Table 6 are unaffected by the draw-to-draw spread.Source: §7 · p. 11
  • MMLU uses dense reconstructions while GSM8K uses the served kernel; at 14B their greedy outputs first differ at token 78 for an unmeasured cause.The evaluation also uses MMLU test questions to select int4 choices, producing selection bias the authors did not measure.Source: §7 · p. 11
  • GSM8K is described as easy for these models, and the evaluation omits Qwen3 thinking mode and harder math benchmarks.The paper also reports no perplexity for sealed files.Source: §7 · p. 11
  • Serving measurements cover batch 1 and short contexts only, with one request at a time and no long prompts.Source: §7 · p. 11

8 Conclusion

Tetra reads Leech-lattice codes without expanding them, reducing GPU-memory traffic while retaining lattice codes. Complete Qwen3 files reach about 2.7 bits per parameter, with further evaluation and kernel work still planned.

8 Conclusion

  • Tetra reads Leech-lattice codes through a 64-state Golay trellis and one 16 KiB table, lowering GPU-memory reads from 4.804 to 2.148 bits per weight.The codebook bounds each third of a block separately rather than the whole block.Source: §8 · p. 11
  • About 2.7 bits per parameter, the Qwen3-4B, 8B and 14B files score 4.76, 4.21 and 2.46 MMLU points below AWQ and generate 113.8, 95.0 and 57.2 tokens per second.On GSM8K through the served kernel, the models lose more to FP16 than on MMLU; with about as much weight memory or less, the 8B and 14B score above AWQ’s 4B and 8B on MMLU and are not separated from them on GSM8K.Source: §8 · p. 11
  • The Qwen3 files use about half the weight memory of 4-bit AWQ, or about 65% if AWQ also stores embedding tables in 4-bit form.Three additional changes produce these files without changing the lattice codes.Source: §8 · p. 11
  • Further tests would vary which matrices use int4, add a second calibration draw and GPU, and address the L40S shared-memory limit for Qwen3-32B.Its widest activation requires 102,400 B, 1,024 B over the card’s limit.Source: §8 · p. 11

Availability

The paper provides its code, measurement records, preregistrations and source, along with commands and scripts for reproducing measurements and figures.

Availability

  • The code, measurement logs, preregistrations and paper source are available at https://github.com/pjmalandrino/llvq.The code and source are under MIT or Apache-2.0 licenses, and the paper is under CC BY 4.0.Source: Availability · p. 12
  • README commands rerun the main measurements, and scripts draw each data figure from CSV files and check tables against those CSVs.The paper’s provenance document names the source of each number.Source: Availability · p. 12
  • The QTIP kernel in Appendix A is not redistributed because it is GPL v3; the benchmark downloads it at a fixed commit.Source: Availability · p. 12

Use of generative AI

Claude, through Claude Code, contributed to the paper’s text, figures, code and statistical recomputation; the author designed the study and checked the results.

Use of generative AI

  • Claude, through Claude Code, drafted and revised the text and drew the figures.Source: Use of generative AI · p. 12
  • Claude wrote parts of the CUDA kernels, Rust host code and tests, and recomputed statistics from committed measurement logs.Source: Use of generative AI · p. 12
  • The author designed the study, authorized every measurement, checked each number against its source and is responsible for the paper.Source: Use of generative AI · p. 12

A The ten-arm kernel benchmark

Table 7 reports kernel timings on 4B matrix shapes from one L40S, including matched-protocol arms and separately sourced measurements.

A The ten-arm kernel benchmark

  • Each Table 7 row is one kernel timed on 4B matrix shapes on one L40S.Source: Table 7 · p. 14
  • The first ten rows comprise five Table 2 arms and five additional kernels timed in the same process at tile 64 with the same protocol.Source: Table 7 · p. 14
  • Rows marked “earlier” are Malandrino [19] layouts retimed here; the QTIP measurement is from another process and uses its own launch geometry.The QTIP run uses pseudo-random codes, reads no row scales or tail, and does not count its 2 KiB table.Source: Table 7 · p. 14

B Record of predictions

The paper records timestamped preregistrations for measurement campaigns and notes departures or exceptions. Table 8 compares selected predictions with measurements.

B Record of predictions

  • Before each campaign, preregistrations named the compared arms, decision rules, and prediction; files were not edited, and departures were recorded alongside them.Three results had no preregistration.Source: B Record of predictions · p. 14
  • Every measured GSM8K result for FP16 and AWQ fell within its preregistered prediction interval.Source: B Record of predictions · p. 14
  • The 14B row-scale training stopped at its preregistered “not improved” loss gauge, but the scales were folded anyway by a decision recorded before the result; this was Table 6’s +1.67.Source: B Record of predictions · p. 14
  • Table 8 presents selected preregistered predictions against measurements, drawing on preregistrations and measurement logs.Source: Table 8 · p. 15

Related papers

Loading 2609.35465v1…