PaperlayerView exact revision on arXiv

Source-linked AI summary

Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment

Anirudh Malik, M Sparsh Mehra, Poojith Devan

arXiv:2609.01962v1cs.AIcs.LG

TL;DR

A nominal low-bit label does not capture a model’s full stored representation or how much pretrained capability survives post-training conversion. This study ternarizes Qwen3-4B weights and finds uneven capability retention alongside a packed checkpoint reduced from 8.29 GiB to 3.96 GiB with perplexity preserved to measurement precision.

  • Problem

    A nominal 1.58-bit label does not specify complete model storage cost, while post-training quantization raises the distinct question of how much pretrained capability survives conversion.

    Source: §1 · p. 2

  • Method

    The authors post-training ternarize Qwen3-4B weights using KOTMS rotation, E2M-ATQ conversion, and GPTQ-style error compensation, with activations at 16-bit.

    Source: §3.1 · p. 3, §4.1 · p. 4

  • Results

    8.29 GiB to 3.96 GiB: lattice-aware packing reduced checkpoint size while preserving recorded perplexity to measurement precision.

    Source: §16 · p. 19

  • Takeaways & Limitations

    For this Qwen3-4B conversion, substantial capability remains on a subset of downstream tasks, but retention varies across task types.

    Source: §6.3 · p. 9, §16 · p. 18

    The packed artifact lacks an end-to-end task benchmark, leaving its packed-model capability unmeasured.

    Source: §15 · p. 18

Abstract

from arXiv · show

Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior. We study an end-to-end post-training conversion of Qwen, an instruction-tuned 4B-parameter model, using KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation from TWLA. The experiment is weight-only: activations remain at 16-bit precision, so ILA-AMP is omitted. We evaluate effective bit accounting, task capability retention, perplexity, calibration sensitivity, checkpoint composition, and deployment behavior. The final conversion uses 1.641 effective bits per weight for quantized linear weights, with 81.62% of model parameters targeted. Across ten scored capability comparisons, accuracy falls from 64.5% to 54.7%. Degradation is uneven: BoolQ retains 84.6% chance-corrected teacher performance, while ARC-Challenge retains 43.8%. Perplexity rises from 13.639 to 18.748 on WikiText-2, 24.700 to 31.992 on PTB, and 19.831 to 28.966 on C4. A subsequent packing run preserves the ternary planes and scales, reducing reported model size from 8.29 GiB to 3.96 GiB with essentially unchanged perplexity. A separate third-party packing attempt was lossy and is excluded from the primary artifact claim. The packed artifact has not been benchmarked end-to-end for task accuracy or generation throughput. A preliminary Triton GEMV microbenchmark is 4.6x slower than FP16 cuBLAS on one tested shape. We therefore do not claim that compression alone yields faster inference.

1 Introduction … 3 Model and Conversion Target

This paper characterizes post-training, weight-only ternarization of Qwen3-4B, emphasizing actual representation costs and deployment behavior rather than claiming a new ternarization algorithm. It uses prior TWLA components with activations left at 16-bit precision.

2. Representation: what is the effective bit budget after adaptive planes and metadata are considered?

  • An ideal ternary symbol carries log2 3 bits, but model storage also includes offsets, scales, residual planes, metadata, unquantized tensors, packing, and execution buffers.The study asks what effective bit budget remains after adaptive planes and metadata are counted, and what size reduction an actual artifact achieves over a fake-quantized FP16 checkpoint.Source: §1 · p. 2, §2 · p. 2

1 Introduction

  • Post-training conversion asks how much of a pretrained model’s linguistic structure, factual associations, task heuristics, and representations survive; this work empirically characterizes prior methods on Qwen3-4B.The stated contribution is representation accounting, capability diagnostics, packing analysis, and deployment measurements, not a new ternarization algorithm.Source: §1 · p. 2, §4 · p. 2

4. Execution: does the compact representation improve inference on the tested hardware?

  • A smaller representation is not automatically faster: unpacking, dequantization, rotation, scales, and kernel overhead can erase arithmetic gains, so the paper distinguishes storage compression from inference-speed advantage.The paper says its evidence supports a storage/compression result and a path toward hardware-aware execution, but does not establish a universal inference-speed or serving-cost advantage.Source: §1 · p. 2, §4 · p. 2

2.1 Native ternary training

  • Native ternary work places low-bit weights in the training formulation: BitNet and BitNet b1.58 provide conceptual foundations, but differ from this study’s post-hoc conversion setting.BitNet b1.58 popularized ternary weights {−1, 0, +1} and the practical relevance of approximately 1.58-bit models.Source: §2.1 · p. 2

2.2 General PTQ

  • General PTQ work shows the range of design choices: GPTQ uses approximate second-order information, AWQ protects salient weights, SmoothQuant enables W8A8 through scaling, and OmniQuant optimizes quantization parameters.Together these methods underscore the importance of calibration, representation design, and execution format.Source: §2.2 · p. 2

2.3 Ternary PTQ

  • Ternary PTQ includes structured planes in PTQTP, small-calibration conversion in CAT-Q, reasoning-trace calibration in ScaleQ-1.58, and TWLA’s combination of KOTMS, E2M-ATQ, and ILA-AMP.This study uses TWLA’s published components in W1.58A16, deliberately disabling activation quantization rather than reproducing W1.58A4.Source: §2.3 · p. 3

3.1 Base model

  • Qwen3-4B is an instruction-tuned dense 4B-parameter model with 36 layers; conversion targets 252 linear tensors, while embeddings and the untied language-model head remain BF16.Targets include MLP gate, up, and down projections and attention q, k, v, and o projections; activations, attention operations, and the KV cache remain FP16.Source: §3.1 · pp. 3–4

3.2 Terminology and representation boundary

  • 1.641 bits/weight is the measured average for the ternarized representation, calculated from observed use of one or two adaptive ternary planes rather than assigning every weight log2 3 bits.The report calls the mixed, adaptive configuration W1.58A16; “ternary” does not mean every parameter uses a single ternary plane.Source: §3.2 · p. 4

4 Method

The method converts Qwen3-4B through KOTMS rotation, E2M-ATQ weight ternarization, and GPTQ-style error compensation, with 16-bit activations. Hessian-salience masks allocate ternary planes unevenly, so the effective weight budget differs from both a single ternary symbol’s ideal information content and total checkpoint size.

4.1 End-to-end pipeline

  • The pipeline reuses a metadata-verified KOTMS-rotated checkpoint, applies E2M-ATQ to convert weights, then performs GPTQ-style error compensation; 16-bit activations make ILA-AMP unnecessary.The conversion sequence is FP16 Qwen3-4B → KOTMS → E2M-ATQ → GPTQ compensation → W1.58A16.Source: §4.1 · p. 4

4.2 KOTMS rotation

  • KOTMS transforms weights into a ternary-friendly basis with structured orthogonal matrices; inference applies the equivalent rotation to activations before the quantized linear operation.The rotation is norm-preserving in exact arithmetic; the experiment reused a pre-existing rotated checkpoint whose metadata was checked against the TWLA configuration.Source: §4.2 · pp. 4–5

4.3 E2M-ATQ: order-2 asymmetric residual ternarization

  • E2M-ATQ uses an asymmetric offset and two ternary planes with scales to approximate weights, then performs 15 refinement iterations while holding ternary supports fixed.The offset shifts the represented values from a centered grid to {μ−α, μ, μ+α}, so exact zeros are not expected to dominate the floating reconstruction.Source: §4.3 · p. 5

4.4 Salience masks and effective bit budget

  • The two most salient masks receive two ternary planes, while 96.5% of weights use one plane; measured effective accounting is distinct from ideal ternary information and checkpoint size.On the recorded layers.10.mlp.down_proj.weight, the first two masks cover 3.6% of weights.Source: §4.4 · p. 5

4.5 GPTQ error compensation

  • GPTQ-style compensation quantizes weights column-by-column and propagates error using a calibration-activation inverse Hessian; calibration used 64 WikiText-2 samples of length 2048.The configuration used block and group sizes of 128, order2_group=True, num_p=1, seed 0, and percdamp=0.1; a later 0.01 attempt did not complete.Source: §4.5 · p. 5

5 Experimental Setup

The experiments used a single-GPU setup, WikiText-2 calibration, and separate perplexity and zero-shot capability protocols. The section also documents a calibration memory leak and explains why capability retention was chance-corrected.

5.1 Hardware and software

  • Measurements ran on one NVIDIA RTX 5070 with 12,227 MiB VRAM under Windows, using lm-evaluation-harness 0.4.12 and TWLA commit 805cbb9.Source: §5.1 · p. 6

5.2 Calibration

  • WikiText-2 calibration used 64 samples of length 2048, while fixing a no_grad leak reduced per-layer memory needs from 10.45 GiB at sample 64 to below 3 GiB.The leak grew linearly from 1.99 GiB after the first sample to 10.45 GiB by sample 64; the sample count was constrained by the widest projection and practical VRAM ceiling.Source: §5.2 · p. 6

5.3 Perplexity evaluation

  • C4 perplexity was 19.831 → 28.966, with PTB and C4 evaluated after WikiText-2 to test whether WikiText-2 calibration and evaluation flattered the result.WikiText-2 evaluation used the full test split: 146 sequences of 2048 tokens.Source: §5.3 · p. 6

5.4 Capability evaluation

  • Zero-shot capability evaluation used about 1,500 examples per task, batch size 2, paired records, and 10,000-resample paired bootstrap intervals.Retention was chance-corrected using the larger of chance and majority-class floors; MMLU used answer-letter and continuation scoring because the protocols capture different output behaviors and answer-letter scoring can be distorted by degenerate policies. PIQA illustrates how raw accuracy ratios can overstate corrected retention.Source: §5.4 · p. 6

6 Main Results

The results show a lower descriptive task-level average and uneven degradation across evaluation corpora, capability families, and MMLU scoring protocols.

6.1 Language modelling quality

  • Perplexity degradation varies by corpus: WikiText-2 has the most favorable ratio and C4 the least, so the WikiText-2 headline alone is insufficient; this does not establish C4 as universally harder.The passages identify WikiText-2 as the calibration distribution and PTB and C4 as out-of-distribution checks.Source: Figure 2 · p. 7, Table 3 · p. 7, §6.1 · p. 7

6.2 Task-level capability

  • 64.5% versus 54.7% is the descriptive mean across ten task comparisons; no reported paired-bootstrap confidence interval crosses zero, but evaluation used one seed and finite limits.The mean is not an average over all language benchmarks or a chance-corrected aggregate retention score.Source: Table 4 · p. 8, §6.2 · p. 8

6.3 Capability stratification

  • BoolQ retains 84.6% of teacher performance; HellaSwag, PIQA, and WinoGrande retain 72–81%, while ARC-Easy retains 68.5%, MMLU continuation 50.0%, and ARC-Challenge 43.8%.This stratified pattern is consistent with greater damage to precise stored knowledge, but the small benchmark suite does not establish a causal law.Source: Figure 4 · p. 9, §6.3 · p. 9

6.4 MMLU scoring protocol

  • MMLU letter scoring: 68.50% FP16 versus 50.96% W1.58A16; continuation scoring: 45.10% versus 35.04%, so quote neither result without its protocol.The scoring gap differs substantially by protocol, and answer-format bias under extreme quantization can confound capability with decoding behavior.Source: §6.4 · pp. 9–10

7 Representation Analysis · 8 Calibration and Quality Sweep · 9 Storage and Packing

The analysis distinguishes the adaptive weight bit rate from whole-checkpoint storage, finds no demonstrated benefit from increasing calibration samples, and reports a packed artifact that preserved perplexity while excluding a lossy alternative.

7.1 From nominal 1.585 bits to 1.641 effective bits

  • 1.641 bits/weight is the measured effective budget versus the 1.585-bit theoretical lower bound; Hessian-based salience selects the 3.6% of weights that receive a second plane.The 1.641-bit figure describes the adaptive representation, not a simple one-plane-per-weight ternary encoding.Source: §7.1 · p. 10

7.2 Why the stored floating tensor can look “non-ternary”

  • A recorded row’s distinct reconstructed values fell from 1,169 in FP16 to 235 after conversion, although offsets, scales, ternary supports, salience masks, and GPTQ compensation can make the tensor look non-ternary.End-to-end perplexity was approximately 18.748 after conversion, compared with 13.639 for FP16, providing a check against evaluating a rotated or unconverted checkpoint.Source: §7.2 · p. 10

7.3 Representation versus complete checkpoint

  • The 1.641-bit rate applies to targeted weights, not the complete checkpoint: pre-pack accounting estimates approximately 2,271 MiB total, including 745 MiB ternary weights, 1,483 MiB embeddings/head, and 78 MiB rotation matrices.The embeddings and language-model head remain BF16; storage measurements therefore need to be reported separately from theoretical bit rate.Source: §7.3 · p. 10

8 Calibration and Quality Sweep

  • 18.749 perplexity with 64 calibration samples increased to 19.087 with 96 samples in the R1 run, a single-seed result interpreted as no improvement from more calibration data.Stage A and R1 used WikiText-2 calibration and percdamp=0.1, with calibration sample count as the changed variable; the reported increase was 1.8%.Source: Figure 6 · p. 11, §8 · p. 11
  • The presented configuration is the best completed sweep, not a proven optimum: FineWeb-Edu was not started, group_size=64 was not evaluated, and 128-sample calibration exceeded the approximately 10.98-GiB budget.Estimated memory for the widest projection was 6.39 GiB at 64 samples, 9.58 GiB at 96, 12.78 GiB at 128, and 25.56 GiB at 256.Source: §8 · pp. 11–12, Figure 7 · p. 12

9.1 Fake-quantized versus actually packed

  • The packed format persisted ternary planes and scales/offsets and reproduced the unpacked model’s perplexity within measurement precision, with 0.037% relative RMSE and a 50-second packing time.The earlier converted checkpoint stored values in FP16 tensors and therefore did not reduce disk footprint; quantization and serialization are distinct.Source: Table 6 · p. 12, §9.1 · p. 12

9.2 The excluded lossy packing attempt

  • The excluded third-party 2.659-GiB artifact discarded the second ternary plane and GPTQ residual, causing 0.305 maximum absolute weight error, and had no perplexity or task benchmark.The analysis concludes that post-hoc packing cannot restore information discarded at save time; the artifact was not used as evidence for the primary model claim.Source: §9.2 · p. 13

10 Deployment and Execution · 11 Failure Modes as Experimental Boundary Conditions

Deployment measurements reflect a reconstruction-oriented software path, while the preliminary Triton result identifies a kernel bottleneck rather than an intrinsic ternary limit. The reported failures also show why memory behavior, artifact representation, checkpoint recovery, and benchmark protocols required explicit checks.

10.1 Reconstruction-based execution

  • The measured path keeps weights in FP16 and adds an activation-side KOTMS matrix operation, so it measures a specific stack rather than native ternary execution.The comparison was benchmarked directly against FP16; the supplied passages do not provide the measured values.Source: Table 7 · p. 13, §10.1 · pp. 13–14

10.2 Preliminary packed-kernel microbenchmark

  • Figure 10 characterizes the preliminary GEMV measurement as a kernel bottleneck, not an intrinsic lower bound on ternary execution.The Triton kernel’s maximum absolute discrepancy from its encoded dense operation was approximately 0.00195; the third-party artifact’s 0.305 error came from encoding and re-quantization mismatch.Source: Figure 10 · p. 14, §10.2 · pp. 14–15

11.1 Autograd-induced memory leak

  • A checkpointing edit retained computation graphs, raising memory from 1.99 GiB after one sample to 10.45 GiB after 64; after the fix, per-layer memory stayed below 3 GiB.The incident motivated instrumenting VRAM constraints rather than inferring them from model size alone.Source: §11.1 · p. 15

11.2 Windows CUDA spill

  • Under the Windows configuration, CUDA allocations beyond physical VRAM spilled into system memory; a 0.92 per-process memory-fraction limit made over-allocation fail cleanly.Source: §11.2 · p. 15

11.3 False ternary artifact

  • Verification rejected an 8.5 GiB checkpoint that was not ternarized; asymmetric E2M-ATQ initially looked non-ternary because exact zeros were rare.The project resolved this by inspecting the quantizer’s asymmetric offset and enforcing end-to-end perplexity verification; low-bit experiments should verify both representation and forward path.Source: §11.3 · p. 15

11.4 Power loss and checkpoint recovery

  • A reboot about 15 minutes before a multi-hour run finished cost roughly 2.5 hours; per-layer checkpointing reduced the expected loss from interruptions to about one layer.Source: §11.4 · p. 15

11 Failure Modes as Experimental Boundary Conditions; 11.5 Benchmark protocol defects

  • The audit rejected raw accuracy ratios as retention, flagged default MMLU letter scoring, required per-item logs, and required both model arms per task for paired comparisons.These corrections were part of the experimental method, not merely editorial cleanup.Source: §11.5 · p. 15, §11 · p. 15

12 Comparison with the Previous OneBit AI Study · 13 Scientific Interpretation

The paper reports that post-training ternarization retained useful but uneven capability, approached ternary-level weight precision, and enabled lossless checkpoint compression, while runtime efficiency remained unproven. It presents these results as an empirical characterization of a 4B PTQ pipeline, not evidence that PTQ universally beats QAT.

12 Comparison with the Previous OneBit AI Study

  • The prior Cloe study converted Qwen3.5-0.8B with 72.4M QAT tokens and reported severe specialist-knowledge loss alongside measurable learnability and deployment benefits on its tested packed artifact [13].The present 4B conversion retained above-chance results on several knowledge-heavy tasks, but the evidence cannot isolate the causal effects of scale and conversion procedure.Source: §12 · p. 16
  • The present study replaces QAT with Hessian-aware PTQ for a larger 4B model, requiring no gradient training to test aggressive discretization without a large recovery-token budget.The comparison is not a controlled QAT/PTQ head-to-head on the same model and compute budget, and does not establish that PTQ universally beats QAT.Source: §12 · p. 16

13.1 Hypothesis assessment

  • BoolQ reached 81.4%, PIQA 68.1%, and WinoGrande 61.6%, while ARC-Challenge and MMLU still showed substantial losses.The capability hypothesis is supported only in part: performance remained above chance on several tasks, but degradation was not negligible.Source: §13.1 · p. 16
  • 1.641 bits/weight was the measured quantized weight budget, and 96.5% of weights in the recorded salience example used a single plane.The study treats the measured bit budget as close to the ternary information floor.Source: §13.1 · p. 16
  • 8.29 GiB became 3.96 GiB through the later lossless packing run, with essentially unchanged perplexity.This supports the storage component of the hypothesis, while deployment efficiency depends on serialization and execution.Source: §13.1 · p. 16
  • The execution-efficiency hypothesis remains unsupported: the reconstruction path and preliminary packed GEMV kernel were slower than FP16 and cuBLAS, respectively.The paper identifies faster execution as an engineering condition for turning representation efficiency into runtime efficiency.Source: §13.1 · pp. 16–17

13.2 What is genuinely new in this study

  • The defensible contribution is an end-to-end Qwen3-4B TWLA characterization spanning effective-bit accounting, task-level degradation, calibration sensitivity, lossless packing, and deployment limits.The paper describes these as empirical and system-oriented contributions, while noting that the underlying quantizer is prior art.Source: §13.2 · p. 17

14 Product and Deployment Implications

The evidence supports a materially smaller near-ternary model artifact, but not claims of faster or cheaper serving on arbitrary hardware. Deployment opportunities depend on native execution and targeted workloads, while broader performance and generalization claims remain unestablished.

14 Product and Deployment Implications

  • The current evidence does not establish universal inference speedup or serving-cost reduction; reconstruction and the tested Triton GEMV are slower than FP16.The commercial opportunity depends on a native execution stack rather than compression alone.Source: §14 · p. 17, §14.1 · p. 18
  • 3.96 GiB is materially smaller than the 8.29 GiB checkpoint, supporting model distribution and storage.The study positions compression as the demonstrated product benefit, not acceleration.Source: §14 · p. 17
  • Further quantizing BF16 embedding/head tensors could reduce the footprint, while native kernels could consume ternary planes directly and avoid reconstruction overhead.The low-cardinality arithmetic structure is compatible with specialized low-bit kernels and accelerators, as demonstrated conceptually by prior BitNet systems [6].Source: §14 · p. 17
  • The capability profile does not justify broad general-purpose replacement, though targeted workloads may offer a more favorable quality-efficiency trade-off.Source: §14 · p. 17

14.1 What the current evidence does not establish

  • The study does not establish equivalence to a native 1.58-bit model, state-of-the-art capability among ternary models, or superiority over every alternative PTQ method.Source: §14.1 · p. 18
  • Production robustness across hardware and general scaling laws across model sizes remain unestablished; these are future research questions, not settled conclusions.Source: §14.1 · p. 18

15 Limitations and Required Next Experiments

The section identifies seven limitations spanning reproduction, experimental variability, tuning, packed-model validation, kernel maturity, parameter coverage, and benchmark breadth. It prioritizes experiments to address these gaps and extend evaluation across calibration data, model scales, and deployment settings.

Limitations and Required Next Experiments

  • Qwen3-4B lacks an independent reproduction gate, and the source record says it is absent from the TWLA paper.The highest-value next experiment is to complete the published-method reproduction gate.Source: §15 · p. 18
  • Capability results use a single seed and therefore do not quantify run-to-run variation.Repeat the capability suite across seeds.Source: §15 · p. 18
  • Tuning is incomplete: the percdamp=0.01 run did not complete, and alternative calibration corpora and representation parameters were not evaluated.Rerun with a broader calibration corpus such as FineWeb-Edu.Source: §15 · p. 18
  • The available packing record shows perplexity preservation but contains no end-to-end packed-model task benchmark.Benchmark the exact packed artifact end-to-end.Source: §15 · p. 18
  • The Triton GEMV measurement is preliminary and should not be treated as a lower bound on native ternary performance.Persist the discrete lattice during quantization and build a lossless native kernel.Source: §15 · p. 18
  • Embeddings and the LM head remain BF16 and account for a substantial fraction of the remaining artifact.Evaluate embedding and LM-head quantization.Source: §15 · p. 18
  • The final task suite is informative but smaller than comprehensive LLM evaluations.Evaluate the same conversion procedure on at least one additional model scale.Source: §15 · p. 18

16 Conclusion … C Implementation and Reproducibility Checklist

The conversion reaches a measured 1.641-bit/weight representation while retaining capability unevenly, and lattice-aware packing makes the checkpoint substantially smaller without changing recorded perplexity. The runtime path remains slower than FP16, so the results support a smaller artifact—not a demonstrated inference-speed advantage.

16 Conclusion

  • 1.641-bit/weight quantized representation retains substantial capability on some tasks, but the mean accuracy cost across ten comparisons is 9.8 percentage points.Contextual and commonsense tasks are more robust than knowledge-heavy tasks.Source: §16 · p. 18
  • 8.29 GiB → 3.96 GiB: lattice-aware packing preserves recorded perplexity to measurement precision while producing a smaller checkpoint.The lattice-aware serialization result is also recorded in the experimental ledger.Source: §16 · p. 19, A Complete Experimental Ledger · p. 19

Reproducibility Note

A Complete Experimental Ledger

  • 19.087 ppl was the negative R2 WikiText-2 result for 64 samples and pd=0.01; the ledger says it did not improve on the 18.749 ppl best completed configuration.The alternative-calibration R3 experiment was not started.Source: A Complete Experimental Ledger · p. 19

16 Conclusion

  • 4.6× slower than FP16 cuBLAS: the preliminary packed Triton GEMV took 0.2082 ms on one tested shape, so current software is not yet hardware-efficient.The conclusion cautions against generalizing this measurement to ternary arithmetic itself and calls for native packing, kernels, and broader validation.Source: A Complete Experimental Ledger · p. 19, §16 · p. 19

C Implementation and Reproducibility Checklist

  • The completed checklist configuration used instruction-tuned Qwen3-4B with TWLA E2M-ATQ, KOTMS rotation, GPTQ compensation, WikiText-2 calibration, and damping 0.1.Calibration used 64 samples of 2048 tokens; block/group size was 128 and order-2 salience grouping was enabled.Source: C Implementation and Reproducibility… · p. 20
  • Evaluation used the full WikiText-2 test split for perplexity and a zero-shot capability suite, with paired bootstrap over 10,000 resamples.The checklist records num_p=1, an NVIDIA RTX 5070 with 12,227 MiB on Windows, and the listed PyTorch, Transformers, Datasets, lm-eval, and Python versions.Source: C Implementation and Reproducibility… · p. 20

D Interpretation Guardrails

Interpret the work as an adaptive ternary weight representation with a measured effective budget of 1.641 bits/weight, uneven task retention, and smaller packed storage—not as evidence of faster execution or broader superiority.

D Interpretation Guardrails

  • 1.641 bits/weight is the measured effective budget for the model’s adaptive ternary weight representation.This does not mean the whole checkpoint is 1.58 bits/parameter.Source: D Interpretation Guardrails · p. 20
  • The completed conversion retains substantial performance on several commonsense and contextual tasks, while knowledge-heavy tasks degrade more strongly.Source: D Interpretation Guardrails · p. 20
  • 8.29 GiB to 3.96 GiB is the recorded checkpoint-size reduction from a later lattice-aware packing run, with essentially unchanged perplexity.Source: D Interpretation Guardrails · p. 20
  • Current execution is slower than FP16; native hardware-efficient execution remains future work.Source: D Interpretation Guardrails · p. 20
  • These experiments do not establish that the packed model is faster, the method is state of the art, 64 calibration samples outperform 96, or PTQ beats QAT.The packed model lacks a full end-to-end benchmark; the 64-versus-96 claim relies on a single R1 comparison, and the PTQ-versus-QAT claim relies only on comparison with the previous Cloe study.Source: D Interpretation Guardrails · p. 20

Related papers

Loading 2609.01962v1…