PaperlayerView exact revision on arXiv

Source-linked AI summary

Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference

Sunghyun Wee, Geunjae Choi, Hyeonjin Kim, Suyoung Kim, Nojun Kwak

arXiv:2610.21257v1cs.LGcs.AI

TL;DR

Coarse PoT quantization grids make multiplier-free LLM inference difficult to achieve without accuracy loss. FPoT uses a half-octave weight grid with uniform integer activations and calibration refinements. It reports near-lossless accuracy across multiple models and quantization settings while enabling multiplier-free shift-add inference.

  • Problem

    Standard PoT’s coarse factor-2 grid creates a worst-case relative quantization error of 41.4%, an accuracy bottleneck for LLM weight quantization.

    Source: §3 · p. 3

  • Method

    FPoT uses a half-octave weight grid with uniform INT activations and calibration-time refinements to enable integer shift-add inference.

    Source: §7 · p. 9

  • Results

    20/21 configurations recover zero-shot accuracy to within 1.0%p of the uniform INT baseline, with a mean gap of −0.17%p across seven models and three quantization settings.

    Source: §7 · p. 10

  • Takeaways & Limitations

    FPoT integrates with state-of-the-art LLM PTQ frameworks at comparable overhead and achieves near-lossless accuracy while enabling multiplier-free shift-add inference.

    Source: §7 · pp. 9–10

    End-to-end RTL/ASIC validation remains future work; efficiency projections use analytical cost models anchored to published RTL synthesis.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) demand efficient inference, where the multiply-accumulate (MAC) operations in linear projections dominate compute and energy cost. Post-training quantization (PTQ) reduces this cost by mapping weights and activations to low-bit integers. Power-of-Two (PoT) weight quantization promises efficient multiplier-free LLM inference via shift-add logic, but suffers significant accuracy degradation at low bitwidths due to its coarse base-2 quantization grid. We introduce Fractional PoT (FPoT), a base-$\sqrt{2}$ (half-octave) fractional PoT grid that achieves practical 4-bit weight quantization while preserving a fully integer shift-add datapath. The method combines (i) the base-$\sqrt{2}$ weight grid with uniform integer activations, (ii) a calibration-time grid-alignment refinement absorbed into the static per-channel scale, and (iii) a lightweight 1-adder $\sqrt{2}$ approximation whose error is absorbed during weight calibration via approximate-grid quantization, yielding a processing element (PE) with significantly fewer gates than a 4-bit integer multiplier. On Llama-3-8B with 4-bit weight and activation quantization, FPoT improves zero-shot commonsense reasoning performance by 8.72% relative to the PoT baseline, while maintaining performance close to the uniform integer baseline, with only a negligible 0.4%p degradation. These results hold across various configurations spanning seven models and three bit-widths. Furthermore, FPoT can be integrated with state-of-the-art LLM PTQ methods, demonstrating framework-agnostic applicability.

1 Introduction

PoT enables multiplier-free inference but its coarse grid causes substantial accuracy loss for LLMs. FPoT introduces a finer weight grid and reports near-INT performance with a gain over PoT.

1 Introduction

  • MAC operations in Transformer linear projections account for virtually all parameters and a majority of inference energy.The introduction identifies this cost as a challenge for deploying LLMs on resource-constrained edge devices.Source: §1 · p. 1
  • Standard PoT’s factor-2 grid has 41.4% worst-case rounding error and yields +1.43 PPL degradation even when restricted to weights.Applying PoT to both weights and activations yields PPL degradation of +9.5 to +107.5.Source: §1 · p. 2
  • 8.72% relative gain over standard PoT, with only 0.4%p 0-shot degradation from the uniform INT baseline, is reported for 4-bit Llama-3-8B.The comparison applies FPoT to weights.Source: §1 · p. 2
  • FPoT uses a base-2 integer-resolution grid at 4-bit, halving standard PoT’s worst-case relative error from 41.4% to 18.9%.A 1-adder approximation and approximate-grid quantization support a PE with significantly fewer gates than a 4-bit integer multiplier.Source: §1 · p. 2
  • FPoT integrates with state-of-the-art LLM PTQ frameworks at comparable overhead and achieves performance on par with uniform INT baselines.The paper defines near-lossless as PPL increase within +0.5 and downstream accuracy drop within 1.0%p of the uniform INT baseline.Source: §1 · p. 2

2 Related Work

Recent PTQ methods support low-bit uniform quantization, while prior shift-based methods leave a gap between multiplier-free computation and fully integer LLM inference. FPoT combines logarithmic weights with uniform INT activations and recent PTQ methods.

2.1 Post-Training Quantization (PTQ) for LLMs

  • Recent PTQ approaches address activation outliers through scaling or rotations and collectively enable near-lossless W4A4KV4 uniform quantization.The cited approaches include SmoothQuant, QuaRot, SpinQuant, OSTQuant, and DuQuant.Source: §2.1 · p. 2
  • Applying logarithmic weight quantization to state-of-the-art uniform PTQ methods reaches comparable performance with better efficiency.The passage frames this as the contribution of the work relative to methods focused mainly on uniform grids.Source: §2.1 · p. 2

2.2 Power-of-Two (PoT) and Shift-based Quantization

  • Prior PoT methods target CNNs, require multiple sequential shift-adds, or retain FP16 activations, unlike FPoT’s fully integer shift-add PE.The cited LLM methods PoT-PTQ and LogART retain FP16 activations.Source: §2.2 · p. 3
  • FPoT targets the gap between uniform INT methods that retain integer multipliers and PoT methods that target CNNs or retain FP activations.It pairs base-2 weight quantization with state-of-the-art uniform INT activation quantization and composes with recent PTQ frameworks.Source: §2.2 · p. 3

3 Methodology

FPoT refines the PoT weight grid to half-octave spacing while keeping activations and the KV cache uniformly quantized. Calibration aligns scales and absorbs the approximation used by the shift-add hardware.

3 Methodology

  • FPoT applies logarithmic quantization to static linear-projection weights while retaining uniform INT quantization for activations and the KV cache.Preprocessing redistributes weight distributions before GPTQ calibration; a calibration-time grid-alignment refinement adjusts the per-channel scale.Source: §3 · p. 3
  • Standard PoT’s factor-2 grid has 41.4% worst-case relative error, motivating a finer grid for LLM weight quantization.The methodology introduces fractional exponent resolution k to refine the PoT grid.Source: §3 · p. 3
  • At k=2, FPoT’s half-octave levels are spaced by approximately 1.414 and reduce worst-case relative error to 18.9%.For k=1, the formulation reduces to standard PoT.Source: §3 · p. 3
  • For k=2, weights with magnitude below approximately 0.841 times the per-channel scale map to zero.The paper analyzes the resulting zero fraction and accuracy implications in its grid-matching analysis.Source: §3 · pp. 3–4
  • The hardware decomposition replaces integer multipliers with barrel shifters and a lightweight unit for odd exponents, adding one adder and a MUX.The FPoT PE shares a barrel shifter with PoT; activations remain uniform INT.Source: §3 · p. 4, Figure 1 · p. 4
  • Approximate-grid quantization rounds weights to the actual hardware grid so calibration absorbs the 1-adder approximation instead of adding it as post-hoc distortion.The hardware uses 1.5 for odd exponents instead of approximately 1.414, a 6.1% difference.Source: §3 · p. 4
  • Grid alignment tests candidate scales per output channel and selects the one minimizing reconstruction MSE, with the winning scale fixed for inference.This refinement applies to both PoT and FPoT grids and adds no inference overhead.Source: §3 · p. 4

4 Analysis

FPoT’s half-octave grid is motivated by preprocessed LLM weight statistics and analyzed for both quantization quality and hardware cost. The analysis finds k=2 well matched to 4-bit weights, with a shift-add implementation that costs less per weight MAC than INT multiplication.

4.1 Grid-Matching Analysis

  • The analysis evaluates FPoT on preprocessed weights using a zero-mean symmetric distribution with unimodal log-magnitude, taking Gaussian weights as a tractable proxy.The preprocessed distributions of all seven studied models closely match the proxy prediction.Source: §4.1 · p. 5
  • Under a Gaussian proxy, k=2 gives a 128× dynamic range aligned to the ∼4-octave spread, and achieves MSE = 1.27× INT4 versus 3.21× for PoT.The numerically optimal effective k is ≈1.8; the k=2 choice is specifically motivated by the 4-bit constraint.Source: §4.1 · p. 5
  • FPoT achieves NBE = 0.9364 under the Gaussian proxy, above uniform INT4 at 0.9026 and PoT at 0.8773, as its half-octave grid densifies levels near the weight mode.Empirical bin occupancy on real LLM weights closely matches the Gaussian-proxy prediction.Source: §4.1 · p. 6
  • At k=2, the zero threshold maps ∼22% of preprocessed weights to zero, enabling their MACs to be skipped through zero-gating.The finer grid resolution compensates for this implicit magnitude pruning.Source: §4.1 · p. 6

4.2 Hardware Cost Analysis

  • FPoT converts the 6.98 B weight MACs per token in linear projections into shift-accumulate operations; its reported hardware ratios are per weight MAC, not end-to-end inference savings.End-to-end energy reduction depends on the share of total energy attributable to weight MACs, which is largest during compute-bound prefill and diminishes during memory-bound decode.Source: §4.2 · p. 6
  • At W4A4, FPoT costs ∼0.50× an INT multiplier; its 1-adder adds ∼0.12× over PoT while reducing PPL overhead by 67%–94%.The reported ratios are per weight MAC; the 1-adder is active for ∼50% of MACs.Source: §4.2 · p. 6
  • FPoT’s 0.42–0.46× energy ratio at W4A4 corresponds to a 2.2–2.4× reduction per weight multiply versus INT, anchored to published RTL synthesis.The estimates use RTL synthesis results for PoT and account for the FPoT approximation unit’s small overhead.Source: §4.2 · p. 6

5 Experimental Results

Across seven models and three bit-width configurations, FPoT retains near-lossless performance by the paper’s stated criteria. On Llama-3-8B W4A4KV4, it substantially reduces PoT’s perplexity and downstream-accuracy penalties.

5.1 Experimental Setup

  • Evaluation uses WikiText-2 PPL and mean zero-shot accuracy across nine benchmarks, with the main comparison reporting mean±std over three calibration seeds.The setup uses OSTQuant preprocessing, and the default FPoT configuration uses grid alignment and 1-adder approximate-grid quantization.Source: §5.1 · pp. 6–7, §5.2 · p. 7

5.2 Main Results

  • On Llama-3-8B W4A4KV4, FPoT reduces PPL overhead from +1.43 (PoT) to +0.24 and accuracy degradation from −5.6%p to −0.4%p.The PPL overhead reduction is 83%.Source: §5.2 · p. 7
  • Across 14 W4A4KV4 and W4A8KV8 configurations spanning seven models, FPoT’s ∆PPL ranges from +0.05 to +0.26, below +0.3 at the designed operating point.The reported range runs from Qwen2.5-14B W4A4KV4 to Qwen2.5-1.5B W4A8KV8.Source: §5.2 · p. 7
  • Across all 21 configurations, FPoT reduces PoT’s PPL overhead by 67%–94%; W8A8KV8 remains within the +0.5 near-lossless PPL threshold despite being outside the 4-bit design range.At W8A8KV8, ∆PPL is up to +0.42.Source: §5.2 · p. 7

2 Approximation Quality √

Approximate-grid quantization preserves the accuracy of a lightweight 1-adder approximation by absorbing its error during weight calibration. Additional ablations identify preprocessing and uniform integer activations as important conditions for the reported performance.

2 Approximation Quality √

  • The 1-adder approximate-grid strategy achieves PPL = 7.574, within +0.01 of exact √2, while the 3-adder alternative matches INT4 cost at 1.00×.The 1-adder implementation approximates √2 as 1.5; naive post-hoc substitution instead degrades PPL by +0.17, whereas calibration absorbs the error into the quantization grid.Source: §2 · pp. 7–8, Table 2 · p. 8

5.4 Additional Ablations

  • Preprocessing is necessary: without a transform FPoT yields PPL = 181.3, while learned rotation plus per-channel scale reaches 7.56.The full combination is within +0.12 PPL of the non-PoT baseline.Source: §5.4 · p. 9
  • Applying the logarithmic grid to both weights and activations degrades accuracy, with PPL ranging from 8.21–52.49 at k=2 versus 7.56 for weight-only.Uniform INT activations remain the practical choice.Source: §5.4 · p. 9
  • Grid alignment reduces PPL by up to −0.051 at k=1 with zero inference cost, while at k=2 its effect is within |∆PPL| ≤0.034.At k=2, the finer grid self-corrects.Source: §5.4 · p. 9
  • Changing the zero-threshold τ or log-versus-linear rounding mode has negligible impact, with ∆PPL < 0.03.Source: §5.4 · p. 9
  • Calibration remains robust across n ∈ {128, 256, 512}, with |∆PPL| ≤0.065 and only +0.06–+0.10 shift from WikiText-2 to C4.The passage attributes this robustness to FPoT’s data-independent grid boundaries.Source: §5.4 · p. 9
  • With another PTQ framework, FPoT has comparable PPL overhead of +0.05–+0.34; LogART, by contrast, has PPL > 3900 after post-hoc INT4 activation without PTQ preprocessing.The LogART comparison is in the weight-only W4A16 setting.Source: §5.4 · p. 9

6 Discussion

FPoT combines a regular geometric weight grid, uniform integer activations, and a 4-bit-resolution focus. The discussion also notes that its hardware efficiency claims are projected rather than validated end to end.

6 Discussion

  • FPoT uses a regular geometric grid suited to GPTQ, uniform INT activations that avoid runtime log2 conversion, and a 4-bit grid-resolution target.The discussion contrasts these choices with APoT’s irregular additive decompositions and PoT-PTQ’s 2–3-bit scale-calibration focus.Source: §6 · p. 9
  • The paper links multiplier-free LLM inference to potential energy and carbon-footprint reductions, while noting that cheaper inference may lower barriers to harmful applications.It states that the work applies to existing public models and does not introduce new capabilities.Source: §6 · p. 9
  • End-to-end RTL/ASIC validation remains future work; preprocessing training requires float32 weight copies, limiting single-GPU 80 GB experiments to ≤14B parameters.Compatibility with AWQ or RTN is untested, and the k=2 optimum is 4-bit specific.Source: §6 · p. 9

7 Conclusion

FPoT combines a half-octave weight grid with uniform integer activations and PTQ preprocessing to support multiplier-free shift-add inference. It achieves near-lossless accuracy across tested models and settings, with compute-unit cost about half that of an INT4 multiplier.

7 Conclusion

  • Approximate-grid quantization absorbs the 1-adder √2 approximation error during weight calibration.The resulting compute unit uses a shifter and adder rather than an INT4 multiplier.Source: §7 · p. 9, §E.1 · p. 22
  • 20/21 configurations recover zero-shot accuracy to within 1.0%p of the uniform INT baseline, with a mean gap of −0.17%p across seven models and three settings.All 14 W4 configurations are within 0.5%p.Source: §7 · p. 10
  • The FPoT compute unit is ∼0.50× the INT4 multiplier gate count, compared with ∼0.38× for standard PoT.The conclusion reports near-lossless accuracy alongside this compute-unit cost.Source: §7 · pp. 9–10

B.3 MSE and NBE Across Integer k

  • At 8-bit, with 127 magnitudes per side, abundant levels permit finer resolution without dynamic-range starvation, shifting the optimal grid to k⋆≈19; thus FPoTk=2 does not generalize as preferred at higher bit-widths.Source: §B.3 · p. 14

C Empirical Bin Occupancy

  • Across model families and scales, empirical occupancy matches the Gaussian prediction; FPoT has more uniform non-zero occupancy while mapping ∼20–22% of weights to zero.The reported zero-bin share follows FPoT’s k-adaptive threshold.Source: C Empirical Bin Occupancy · pp. 15–16

C.2 Effect of PTQ Preprocessing on Bin Occupancy

  • PTQ preprocessing raises NBE for uniform INT4 and FPoT but slightly lowers it for PoT, whose 64:1 grid cannot exploit the redistributed ∼4-octave weight spread.The effect is most pronounced for Qwen2.5 models with stronger raw-weight outlier concentration.Source: §C.2 · p. 18

D.2 Per-Projection FPoT Sensitivity

  • Applying FPoT to all seven projections adds +0.12 PPL and reduces 0-shot9 accuracy by 0.7%p; the degradation is sub-additive across projections.For the Value projection, full FPoT incurs ∆PPL = +0.30 versus +0.04 for weight-only FPoT; Output is the safest future activation-quantization candidate, followed by FFN projections.Source: §D.2 · p. 21, §D.3 · pp. 21–22

E.1 Naive Component-Based Cost Model

  • The cost analysis assumes simplified component costs and omits synthesis optimization and energy modeling; scaling to larger models, higher bit-widths, and end-to-end RTL/ASIC validation remains future work.The model treats gate counts as an area proxy and is technology-node independent.Source: §7 · p. 10, §E.1 · p. 23

E.2 Open-source Synthesis Results

Open-source synthesis supports the structural cost ordering INT > FPoT > PoT, while the per-model trade-offs show FPoT approaching Uniform INT accuracy at substantially lower compute-unit cost. The energy comparison is anchored to published RTL synthesis and applies to the per-weight MAC arithmetic unit.

E.2 Open-source Synthesis Results

  • INT > FPoT > PoT holds across all synthesized configurations, and scaling W4A4 to W8A8 grows INT compute-unit gates 5.2× versus 3.8× for FPoT.The Yosys + ABC gate counts exclude sign negation and zero-gating in the accumulator control path.Source: §E.2 · p. 23, Table 7 · p. 23
  • At W4A4, synthesized FPoT costs 0.57–0.59× the INT compute unit, versus 0.50× in the analytical model; the 0.988 cost-vector cosine similarity supports structural agreement.The generic-gate output-width MUX makes the synthesized FPoT ratio higher than the analytical estimate.Source: §E.2 · p. 23, Table 7 · p. 23

E.3 MAC Energy Ratio: Published Synthesis Anchor

  • Published Samsung 5 nm RTL synthesis reports a 0.42× PoT-to-INT energy ratio at W4A8, an arithmetic-unit comparison rather than total LLM inference energy.The per-MAC ratio is presented as an upper bound on weight-arithmetic energy savings; system energy also includes memory, accumulator, and activation-path contributions.Source: §E.3 · pp. 23–24

E.4 Per-Model Accuracy–Cost Trade-offs

  • Across seven models, FPoT approaches Uniform INT accuracy at 0.50× compute-unit cost, while full-MAC cost is 0.67× at W4A4.The accuracy–cost curves vary by model in accuracy, while their hardware-cost coordinates are shared.Source: §E.4 · p. 24, Figure 13 · p. 24, Figure 14 · p. 25

F Fused Triton Kernel for FPoT Training

A fused Triton implementation reduces FPoT training’s intermediate-memory peak and retains close numerical agreement with the PyTorch reference. The accompanying sensitivity analyses examine calibration, grid alignment, activation quantization, zero-threshold choice, and rounding mode.

F Fused Triton Kernel for FPoT Training

  • The fused Triton quantize–dequantize–STE kernel cuts peak intermediate memory from 1.07 GB to 0.19 GB, enabling 14B-model training on one A100-80GB at batch size 4.It keeps intermediate values in registers rather than materializing global-memory tensors; 99.6% of full-size tensor elements exactly match the PyTorch reference.Source: F Fused Triton Kernel for FPoT Training · p. 28

G.1 Calibration Sensitivity

  • Across a 4× calibration-sample range, PPL varies by at most 0.065 within either dataset; switching WikiText-2 to C4 changes PPL by +0.06–+0.10 with downstream accuracy virtually unchanged.The tested default, WikiText-2 with n=128, is described as near-optimal across tested configurations.Source: §G.1 · p. 30, Table 8 · p. 30

G.2 Weight Grid-Alignment

  • At k=1, grid alignment reduces PPL by up to −0.051; at k=2, the finer grid self-corrects and |∆PPL| stays at or below 0.034.The table caption identifies G as the number of phase-shift candidates and compares against plain FPoT.Source: Table 9 · p. 30, §G.2 · p. 30

G.3 Weight-Only vs. Full PoT Quantization

  • Applying PoT or FPoT to both weights and activations yields PPL 16.9–114.1 at k=1 and 8.21–52.49 for FPoT k=2, while weight-only quantization remains usable.The reported dominant bottleneck is the coarse logarithmic grid on post-softmax activations.Source: §G.3 · p. 30, Table 10 · p. 31

G.4 k-Adaptive Zero-Threshold Sensitivity

  • Sweeping the zero-threshold parameter from α=−0.5 to −100 changes PPL by only 0.033 and 0-shot9 by 0.5%p, making the threshold non-critical in this setup.The default α=−0.5 maps values below τ=0.841 to zero in the reported sweep.Source: §G.4 · p. 31, Table 11 · p. 31

G.5 Rounding Mode

  • Log-space and linear-space rounding differ by ∆PPL = −0.001 and ∆0-shot9 = +0.1%p; log-space rounding is retained for computational simplicity.Their geometric- versus arithmetic-mean decision boundaries differ by only about 1.5%.Source: §G.5 · pp. 31–32, Table 12 · p. 32

G.6 Preprocessing Ablation

The preprocessing ablation varies rotation quality and per-channel scaling within OSTQuant for FPoT weight quantization. The full learned-rotation-plus-scale combination gives the best reported PPL among the tested components.

G.6 Preprocessing Ablation

  • The ablation varies rotation quality from none to fixed Hadamard to learned rotation, and tests whether per-channel scale is present.It evaluates FPoT weight quantization on Llama-3-8B W4A4KV4 with uniform INT4 activations and KV-cache.Source: §G.6 · p. 32, Table 13 · p. 32
  • PPL is 181.3 without a transform, 8.30 with fixed Hadamard alone, 7.78 with learned rotation alone, and 9.78 with scale alone.The tested full combination reaches PPL 7.56.Source: §G.6 · p. 32, Table 13 · p. 32
  • Only learned rotation plus scale reaches PPL 7.56, within +0.12 of the non-PoT baseline; the passage attributes the roles to outlier suppression and per-channel amplitude normalization.The reported result supports the design criterion used by OSTQuant and DuQuant.Source: §G.6 · p. 32, Table 13 · p. 32

H Per-Task Downstream Results

Across seven models and three bit-width configurations, FPoT generally tracks Uniform INT performance more closely than standard PoT. The patterns vary by model family and task, and the gap to the unquantized reference narrows at higher bit-widths for most models.

H Per-Task Downstream Results

  • Across seven models and three bit-widths, FPoT with 1-adder approximate-grid quantization maintains competitive per-task performance against FP and Uniform INT, usually within ±2–3%p of Uniform INT.The appendix reports normalized accuracy for ARC-C, ARC-E, HellaSwag, PIQA, and OpenBookQA, and accuracy for the other tasks.Source: H Per-Task Downstream Results · p. 32
  • At W4A4KV4, FPoT AVG meets or exceeds Uniform INT for all three Qwen2.5 models by +0.7–+1.2%p and for Mistral-7B by +0.1%p, while Llama models trail by −0.1 to −0.8%p.The reported FPoT-versus-Uniform difference is model-family dependent.Source: H Per-Task Downstream Results · p. 33
  • Standard PoT degrades consistently by −2 to −8%p across tasks; WinoGrande improves under FPoT at W4A4KV4 for five of seven models.LAMBADA results differ by model: FPoT beats Uniform on Llama-2 and Mistral-7B, while quantized methods are near 0% for Qwen2.5-1.5B.Source: H Per-Task Downstream Results · pp. 32–33
  • At W8A8KV8, FPoT AVG is within about 1%p of the FP reference for six of seven models; Qwen2.5-1.5B is the exception at −18.7%p.The reported W4A4KV4 gap from FP is largest across all models.Source: H Per-Task Downstream Results · p. 33

I Comparison with LogART

The LogART comparison distinguishes weight-only results from joint W4A4KV4 behavior, while the DuQuant comparison tests whether FPoT’s quality overhead transfers across PTQ frameworks.

I Comparison with LogART

  • With post-hoc INT4 activations, LogART’s weight-quantized model reaches PPL > 3900 or ∞ and 0-shot9 ≈34%, while FPoT maintains PPL ≤7.60 and 0-shot9 ≥61.9%.These W4A4KV4 results analyze activation quantization without LogART PTQ preprocessing, not a direct LogART benchmark.Source: Table 21 · p. 36, I Comparison with LogART · p. 36
  • In W4A16, LogART beats FPoT by −0.24 PPL / +3.3%p 0-shot on LLaMA-2-7B and −0.35 PPL / +2.2%p 0-shot on LLaMA-3-8B.The passage attributes this likely to dynamic per-channel base allocation and learned rounding; Table 21 uses single-seed entries for these comparisons.Source: I Comparison with LogART · p. 36, Table 21 · p. 36

J Cross-Framework Portability

  • DuQuant integration swaps only the weight grid for FPoT; activations and KV-cache remain uniformly quantized, with column-wise weight calibration replacing round-to-nearest.The calibration backend performs an independent per-channel MSE clip-ratio search.Source: J Cross-Framework Portability · p. 37, Table 22 · p. 37
  • In DuQuant, FPoT raises perplexity by +0.05–0.34 versus uniform INT weights, comparable to OSTQuant’s +0.08–0.42; nine-task mean accuracy drops at most 1.1%p.DuQuant comparisons use one calibration seed, while OSTQuant uses a three-seed mean.Source: J Cross-Framework Portability · p. 37, Table 22 · p. 37
  • FPoT’s near-lossless multiplier-free property transfers to PTQ pipelines with channel-level weight calibration, including DuQuant and OSTQuant.The reported FPoT quality overhead is small in both frameworks.Source: J Cross-Framework Portability · p. 37

Related papers

Loading 2610.21257v1…