Transforms for LLM Quantization: The Great Inversion and Format Co-Design

Ehsan Jokar

Source-linked AI summary

Transforms for LLM Quantization: The Great Inversion and Format Co-Design

Ehsan Jokar

arXiv:2608.25188v1Aug 25, 2026cs.LGcs.IT

TL;DR

The paper addresses the lack of a dedicated account of the transform stage in low-bit LLM quantization and its relationship to earlier transform-coding theory. It surveys and organizes the literature while proving the Great Inversion between allocation-flexible concentration and shared-scale flattening. It concludes that optimality guarantees do not transfer across regimes and that transform choice depends jointly on quantization format and deployment setting.

Problem
The transform stage in low-bit LLM quantization lacks a dedicated survey, while its literature re-derives transform-coding ideas under a different shared-scale regime.
Method
The paper synthesizes classical and LLM transform methods, proves their opposing objectives under within-group majorization, and analyzes formats, rounding, deployment, and related tensors.
Results
The Great Inversion shows that allocation-flexible coding favors concentration whereas uniform-grid shared-scale quantization favors flattening, with no optimality guarantee transferring for a generic spectrum.
Takeaways & Limitations
Transform selection must account jointly for allocation regime, number format, and deployability rather than importing an optimum from another quantization setting.
Takeaways & Limitations
No surveyed result establishes a single transform as jointly optimal across number formats and rounding stages for deployed settings.

Abstract

from arXiv · show

Most competitive 4-bit LLM research pipelines now open the same way: apply a linear, function-preserving transform (rotation, scaling, permutation, non-orthogonal affine) so the outlier mass sits more favorably against the group scales, and only then round. Yet we are aware of no survey dedicated to this transform stage, and its literature is quietly re-deriving an older theory. We identify and formalize the principle that organizes it, the Great Inversion: allocation-flexible coding rewards energy concentration, whereas the grouped shared-scale quantization a deployed matrix instruction performs rewards within-group flattening. Classical transform coding (1963: decorrelate, allocate bits, quantize) spends different bits per coordinate at a fixed total rate; for a Gaussian source at high rate the Karhunen-Loeve transform's concentration minimizes distortion. A deployed operand tile instead carries one absolute-maximum scale per group and equal bits everywhere, with no allocation; on a uniform grid that objective rewards flattening, approached by Hadamard incoherence. We prove that opposition under within-group majorization: the prescriptions point in opposite directions, each backed by a proof against its own objective, and for a generic spectrum no optimality guarantee transfers. A second axis is the number format: the non-uniform FP4 grid makes flattening buy less, MXFP4's power-of-two block scale still rewards a rotation confined to that block, and NVFP4's mantissa-carrying scale largely removes that pull, so the target pole depends jointly on allocation regime and format. We survey 200 works to a June 2026 cutoff; classify 43 transform methods by structure, data-awareness, searched-versus-constructed, and runtime cost; record, where reported, how they compose with GPTQ rounding; distill a first-choice guide by deployment regime; and close with the open problems it exposes.

1 Introduction

Low-bit LLM deployment is constrained by shared-scale outliers, motivating function-preserving transforms before rounding. The survey connects this practice to transform coding, formalizes the Great Inversion between concentration and flattening, and maps methods, formats, deployment costs, and open problems.

Motivation: Autoregressive decoding is memory-bound, so four-bit PTQ reduces weight traffic and targets latency and energy costs.W4A4 additionally addresses compute through low-precision tensor cores.
Motivation: Shared group scales are pinned to extreme values, making crest factor rather than typical magnitude the central obstacle at four bits.Better rounding alone cannot remove representation-limited dynamic-range penalties.
Transform stage: Function-preserving transforms redistribute outliers before rounding, while folding or online execution determines their deployment cost.Randomized Hadamard transforms can reduce crest factor from O(d) to O(√log d).
Great Inversion: Allocation-flexible quantizers reward energy concentration, whereas shared-scale quantizers with equal bits reward within-group flattening on uniform grids.The KLT represents the concentration pole; Hadamard incoherence approaches the flattening pole.
Great Inversion: The Great Inversion formalizes these opposing prescriptions under within-group majorization, with proofs tied to separate objectives and no generic optimality transfer.The synthesis distinguishes established component results from the survey’s organizing thesis.
Survey scope: The survey covers 200 works and classifies 43 transform methods by structure, data awareness, search strategy, runtime cost, and rounding composition.It also examines empirical comparisons, shipped releases, format interactions, and four open problems.

2 Foundations and Background

Low-bit LLM quantization uses function-preserving transforms before rounding to reshape outliers and shared-scale error, while preserving the layer’s computation. This section defines the linear-layer setting, quantization proxies, transform absorption costs, and the crest-factor penalty that makes outliers especially damaging at four bits.

Notation: A linear layer computes Y = XW ⊤, with calibration tokens arranged as rows of X and weights represented by W.The survey uses Frobenius, spectral, entrywise maximum, and ℓp norms, plus recurring second-moment matrices.
Function-preserving transforms: Function-preserving transforms redistribute error between weight- and activation-side proxies while leaving Y = XW ⊤ unchanged.Their usefulness depends on both quantization benefit and whether the transformed activation side is cheap to compute.
Grouped quantization: A deployed kernel quantizes coordinate groups separately, giving each group one scale; therefore layer error is assembled group by group.The running example is a sixteen-value group containing fifteen ordinary values and one large outlier.
Scale selection: AbsMax pins the scale to the group maximum, allowing outliers to stretch the grid and worsen ordinary-value resolution.MSE-optimal scaling may sacrifice rare large values, but the paper emphasizes that outliers carry output-critical signal.
Crest-factor law: The 6 dB/bit rule makes every doubling of crest factor cost 6 dB of SQNR under high-resolution symmetric AbsMax quantization.The paper reports that a CF = 8 group incurs an 18 dB tax, consuming much more of a 4-bit than an 8-bit SQNR budget.

2.5 Rate regimes, rate–distortion, and majorization

The paper distinguishes allocation-flexible coding from deployed shared-scale quantization through the freedoms to allocate bits and assign scales. Majorization proves that these regimes reward opposite within-group energy arrangements, while flattening guarantees require explicit assumptions about group RMS and transform randomness.

Rate regimes: Shared-scale uniform quantization uses equal bits and one scale per group, whereas classical rate–distortion theory allows coordinate-wise allocation.The distinction concerns both bit allocation and per-coordinate scaling, not simply fixed- versus variable-length coding.
Rate regimes: Allocation-flexible coding may spend different numbers of bits across coordinates under a fixed total-rate budget, unlike deployed shared-scale GEMM kernels.Allocation can use deterministic unequal bit assignment, entropy coding, or both; entropy coding is optional to the KLT result.
Rate–distortion: Reverse water-filling allocates more bits to higher-variance Gaussian coordinates and none to coordinates below the water level.This is the rate–distortion mechanism underlying the concentration preference of classical transform coding.
Rate–distortion: The coding gain is the arithmetic-to-geometric mean ratio of transformed diagonal variances and is maximized by concentration, achieved by the KLT.AM–GM gives equality when transformed variances are equal, while Schur–Horn establishes that the KLT diagonal is maximally concentrated.
Majorization: Under within-group majorization, allocation-flexible and shared-scale objectives are extremized at opposite poles: concentration versus flattening.The shared-scale surrogate depends on group maxima and grouping, so global permutation order alone does not characterize it.
Majorization: The Great Inversion states that classical coding wants concentrated energy, while shared-scale kernels want within-group flattening.Each prescription is proved against its own objective, and for a generic spectrum neither optimality guarantee transfers to the other regime.
Incoherence: Randomized Hadamard or Haar rotations reduce worst-case crest factor to O(√log d), but per-group guarantees require a lower bound on each group’s realized RMS.A deterministic Hadamard does not guarantee flattening for arbitrary vectors, and the RMS premise is not automatic.
Incoherence: When the group-RMS condition holds, the crest factor becomes O(√log G), and a reduction by factor F is worth log2 F bits of AbsMax precision.This quantifies the benefit of incoherence against the O(d log d) cost of a multiplier-free Hadamard transform.

2.7 Why rounding is not independent: an error-feedback primer

Error-feedback rounding improves the coupled layer objective by compensating earlier rounding errors, but it cannot remove the shared-scale dynamic-range wall. Transform usefulness therefore depends on both rounding overlap and the inference cost of applying the transform.

Error-feedback mechanism: RTN rounds coordinates independently, whereas the Hessian-weighted layer objective couples their errors; error feedback exploits this coupling by compensating later coordinates.The update is posed as constrained least squares and applied greedily in Optimal Brain Compression and GPTQ.
Composition with transforms: Transform benefits can overlap with error-feedback gains, so evaluating a transform after GPTQ may overstate its independent contribution.The passage identifies quantifying this overlap as a central composition-analysis task and evaluation pitfall.
Dynamic-range boundary: With fixed scales, error feedback remains tied to the shared AbsMax peak and clips excursions beyond the frozen range rather than rescaling them.GPTQ’s default instead refits each group’s scale on compensated weights, still using the AbsMax rule.
Absorption and runtime cost: A foldable transform has zero inference cost, a structured orthogonal map costs O(d log d) additions per token, and a dense data-dependent map costs O(d^2) per token.The cost depends on whether the transform is absorbed into weights, implemented as a butterfly network, or applied as a dense matmul.
Computational invariance: For RMSNorm residual streams, a global orthogonal rotation preserves the function when reading and writing weights and stream biases are transformed consistently.Norm preservation under RMSNorm enables folding the rotation into surrounding weights at zero inference cost; mean subtraction would invalidate this argument.
Placement constraints: Orthogonal transforms at mid-block sites remain online unless both sides land on static weights, while non-orthogonal maps must justify their online cost.The residual-stream site is the free orthogonal case; other placements depend on where the transform attaches.
Format dependence: INT4, MXFP4, and NVFP4 all use fixed-rate shared scales, but their scale types and element grids change how strongly flattening improves error.MXFP4 uses an E8M0 power-of-two block scale, while NVFP4 uses finer G=16 blocks with a richer E4M3 scale.

3 The Classical Inheritance: Transform Coding, 1963–2021

Classical transform coding establishes the concentration pole: variable-rate coding benefits from decorrelation, energy concentration, and allocating bits toward high-variance coordinates. Its guarantees rely on high-rate assumptions and, for KLT distortion optimality, Gaussian-shaped rotated marginals.

Transform coding inheritance: Classical high-rate transform coding proves a concentration strategy: the KLT packs signal energy into as few coordinates as possible before bit allocation.The paper frames this as one pole of the later concentrate-versus-flatten inversion.
Fixed versus variable length: Fixed-length and entropy-constrained quantization optimize different point densities: the former uses λ⋆(x) ∝ p(x)^1/3, while the latter makes λ⋆ constant.Entropy coding replaces point-count accounting with index entropy, allowing shorter codewords for more frequent cells.
High-rate performance: Entropy-coded uniform scalar quantization is within 0.255 bits/sample of the Shannon lower bound at high resolution under standard regularity conditions.The gap is described as a geometric constant and corresponds to the scalar quantizer’s space-filling inefficiency.
Scope of KLT optimality: The KLT distortion guarantee requires Gaussian sources with positive-definite covariance and common quantizer-shape constants across rotated marginals.For non-Gaussian sources, the KLT can be strictly suboptimal among orthogonal transforms, even though its coding-gain statement is distribution-free.
Rate allocation: Water-filling then assigns the variable-rate budget to coordinates according to their transformed variances, requiring per-coordinate rate allocation.This allocation privilege is precisely what fixed-rate hardware removes.
Fixed proxies: The KLT is source-dependent and lacks a fast algorithm, motivating fixed cosine or Hadamard proxies when signal-independent, efficient transforms are preferred.For AR-1 sources, Toeplitz eigenvectors approach cosines as correlation ρ →1; the paper connects this precedent to fixed Hadamard transforms.
Beyond linear-Gaussian assumptions: Linear transform-coding optimality is a high-rate, second-order result, while nonlinear transforms can exploit higher-order structure on curved source manifolds.The limitation matters when applying classical Gaussian theory to real, non-Gaussian data.

4 The Great Inversion and the Optimality Theory

The Great Inversion formalizes opposing transform objectives: variable-rate coding favors energy concentration, while fixed-rate shared-scale quantization on uniform grids favors within-group flattening. The paper proves this opposition under within-group majorization, then shows that number format, grouping, reachability, and model assumptions constrain how directly these surrogate optima apply.

Opposed objectives: Variable-rate coding concentrates energy, whereas deployed shared-scale quantization rewards within-group flattening on a uniform grid.The former follows the geometric-mean coding surrogate; the latter follows the group-maximum objective induced by one shared scale and equal bits.
Format dependence: Number format changes the fixed-rate target: MXFP4 retains an AbsMax-like penalty, whereas NVFP4’s mantissa-carrying scale weakens flattening’s incentive.The format-dependent measure is derived for an ideal FP grid and has not been evaluated against deployed formats.
Fixed-rate optimum: At fixed total energy, the fixed-rate surrogate is minimized exactly when every group is internally uniform, while concentration can raise it to its upper bound.The lower bound E/G is attained by internally uniform groups, and the upper bound E occurs when each group’s mass sits on one coordinate.
Reachability and guarantees: The reversal is a pole structure, not a proof that KLT or Hadamard is universally optimal: admissible transforms jointly constrain all rows, columns, and groups.Exact within-group flatness is generally unattainable, and the fixed-rate objective depends on grouping and is not permutation-symmetric or globally majorization-monotone.
Majorization reversal: Within-group majorization moves the two surrogates in opposite directions: concentration lowers the geometric-mean objective but never lowers the shared-scale objective.The opposition is strict for the geometric mean when all coordinates remain positive and strict for the fixed-rate objective when a group maximum rises.
Scope of the theory: The theory forbids transferring optimality proofs across regimes but licenses no transform as jointly optimal across formats and rounding stages.The paper identifies the absence of a joint deployed optimum as the current frontier rather than a measured guarantee for every instance.

5 What Is Proven, and What Is Not

The ledger separates results by regime and objective: concentration is proved for allocation-flexible coding, while flattening is supported for shared-scale quantization under stated assumptions. Format-specific extensions and deployed-map evidence remain incomplete, so no optimality guarantee transfers across the boundary.

Table 1 records optimality results against each regime’s own objective, and none transfers across the concentration–flattening divide.The ledger distinguishes allocation-flexible coding, shared-scale quantization, format-dependent results, and regime-neutral rounding machinery.
Format-dependent shared-scale results: On an ideal FP grid, rotation helps only when the untransformed joint concentration ΔFP exceeds approximately 1; its measured benefit is matrix-specific.The cited result reports ΔFP=1 as the isotropic target, post-rotation expectation below 3, and measured values near 1 after a Hadamard.
Format-dependent shared-scale results: On deployed MXFP4, a block-confined rotation is measured to improve the coarse power-of-two-scale setting, but the corresponding mechanism is not yet explained.The global-versus-block-confined contrast was measured, while the AbsMax surrogate was not computed on the two deployed maps.
Format-dependent shared-scale results: A closed-form data-aware blockwise transform is model-optimal for FP-AbsMax and O(1/d)-near-optimal for INT-AbsMax under a stochastic-noise model.The guarantee applies to zero-mean tail-bounded Gaussian or Laplacian data and arbitrary invertible block-diagonal transforms, with d denoting block size.
Regime-neutral machinery: GPTQ equals LDLQ and is optimal among H-dependent, W-independent linear-feedback rounders under the stated integer-rounding proxy; Babai’s bound additionally requires an unbounded grid.The rounding ledger also identifies back-to-front GPTQ with Babai’s nearest-plane method under full-column-rank and matched-domain conditions.
1 bit?
What is not proven: The sharpest unresolved gap is the unexplained global-versus-block-confined MXFP4 contrast, and no result proves one transform optimal across all deployed formats and rounding stages.The survey identifies evaluating the deployed-map surrogate and reconciling published mechanistic accounts as open problems.

6 A Taxonomy of Transforms

The taxonomy spans diagonal, permutation, orthogonal, learned-orthogonal, and affine transforms, progressing from redistribution and migration toward whitening and greater expressiveness. Across these families, fixed rotations offer cheap near-optimal flattening, while learned and non-orthogonal maps pursue remaining data-dependent gains under deployment constraints.

Cross-family comparison: The taxonomy therefore links transform freedom to both quantization objective and runtime: richer maps can approach flattening optima, but online execution may be required.Fixed randomized transforms are calibration-free and often foldable, whereas optimal whitening or learned general maps can remain online.
Diagonal scalings: Diagonal scaling migrates per-channel difficulty between activations and weights but cannot reduce crest factor within a channel.SmoothQuant’s scale can be combined with a shift to symmetrize asymmetric activations, yet the FFN down-projection exposes the limits of scaling alone.
Permutations and reallocation: Permutations and channel reallocation reorganize coordinates or split outliers while preserving function and enabling more regular low-bit paths.Atom makes mixed-precision blocks contiguous, while QLLM disassembles an offending channel into T replicated sub-channels and reassembles similar channels.
Fixed orthogonal transforms: Random orthogonal transforms spread outliers across coordinates, reducing crest factor from O(d) to O(√log d) without calibration.QuaRot, QuIP, and related methods exploit incoherence; foldable residual-stream rotations can have near-zero runtime cost.
Learned orthogonal transforms: Learned rotations recover only the residual gap between random-flat and data-optimal flattening, with reported gains of a fraction of a perplexity point on Llama-2-7B W4A4.HARP instead optimizes a Hessian-weighted, off-block-penalized proxy and reports 8.95→7.85 perplexity at context length 2048, while using 2.11 versus 2.00 bits per parameter.
Non-orthogonal affine transforms: General invertible transforms add whitening, reshaping second moments in ways pure rotations cannot, but expressiveness creates a deployability trade-off.WUSH’s inverse-transpose pairing preserves the layer output exactly and attains the floating-point AbsMax optimum; for integer grids it is near-optimal under stated distributional and block conditions.

7 Composition: Rounding and Codebooks

Rounding and allocation can substitute for transform benefits at high rate, while error feedback and lattice views clarify when stacking is redundant or complementary.

Composition taxonomy: Transforms and rounders may compose, substitute, or co-optimize depending on whether they provide distinct or overlapping benefits.The section distinguishes these relationships by whether stacking helps, is redundant, or requires jointly optimized choices.
Error-feedback rounding: GPTQ uses a shared quantization order and Hessian factorization to reduce OBS-style rounding cost across rows.Its fixed column order allows one inverse Hessian and Cholesky factor to serve every row.
Error-feedback rounding: QuIP proves its restricted linear-feedback rounder optimal for an integer-grid quadratic proxy, not for unrestricted deployment loss.The theorem covers worst- and average-case proxy loss within the class of feedback rules depending on H but not W.
Allocation versus rotation: At high rate, random rotation and WaterSIC’s per-coordinate allocation attack the same basis dependence and are near-substitutes.WaterSIC is provably basis-free, while rotation empirically closes most of the arithmetic-to-geometric-mean gap.
Allocation versus rotation: The worst-case equivalence between rotated GPTQ and WaterSIC remains open, and their overlap shrinks at 2–4 bits because clipping and weight structure matter.Layer dependence becomes important outside the high-rate regime.
Low-rate regime: QuIP’s low-rate gains require a package of rescaling, orthogonal conjugation, permutation, and clipping choices rather than rotation alone.On OPT-125m, permutation alone is reported to be worth 74 perplexity at 2 bits, while all substeps were necessary for the full gain.
Evaluation interaction: Transform gains appear larger with rounders lacking error feedback because GPTQ repairs part of the same error, making protocol choice consequential.This overlap is empirical rather than implied by the high-rate theory.
Lattice interpretation: GPTQ can be viewed as Babai nearest-plane decoding, making a rotation a lattice-basis choice while clipping and channel scales limit the analogy.Lattice reduction offers approximation machinery, but exact closest-vector minimization is computationally hard.

8 Number Format and Transform–Format Co-Design

Number format changes the transform objective: ideal FP4 weakens flattening’s benefit, MXFP4 restores a block-local incentive, and NVFP4’s scale design weakens it further.

8 Number Format and Transform–Format Co-Design: The integer-grid flattening thesis does not transfer unchanged to floating-point formats because the format and scale hierarchy alter the objective.The survey separates integer-grid theory from stochastic AbsMax and floating-point analyses.
8.1 Low-precision floating-point and microscaling formats: MXFP4’s floored power-of-two scale can waste range when the block maximum lies just above a power of two.For an E2M1 block, M/s lies in [4, 8), although the element grid’s top value is 6; one case wastes a factor of 1.5.
8.1 Low-precision floating-point and microscaling formats: Hierarchical formats such as HiFloat4 move dynamic range into scale metadata while retaining a uniformly spaced element grid.Its eight magnitudes are {0, 0.25, . . . , 1.75}, with 32 metadata bits per 64-element block.
8.2 The integer–floating-point flip: Floating-point distortion depends on joint weight–activation energy concentration, not concentration in either operand alone.The factor is high when both operands place energy on the same coordinates and remains at its isotropic value when one operand is uniform.
8.2 The integer–floating-point flip: On tested Llama-3-8B Wv matrices, rotation raised the floating-point concentration factor toward 1 and was actively harmful, reversing integer-grid behavior.The result is reported for the full weight–activation product on the tested matrices.
8.2 The integer–floating-point flip: FP4 gains less from flattening because its element grid already allocates finer spacing to the bulk and coarser spacing to tails.This reduced gain is distinct from MXFP4’s later sign change caused by block-scale behavior.
8.2 The integer–floating-point flip: MXFP4 reintroduces an ℓ∞-like block-maximum penalty, so block-confined Hadamard rotations help while QuaRot’s global Hadamard performs poorly.The global Hadamard is reported 6.4 points below plain per-block RTN and 7.7 below GPTQ.
8.3 Co-designing the transform with the format: NVFP4’s mantissa-carrying scale weakens the block penalty, making rotation omission conceivable, while format-aware scale fitting improves MXFP4.MR-GPTQ brings MXFP4 within 1–2% of NVFP4; a modified exponent grid recovers 6.5 points under RTN and 1.3 above MR-GPTQ’s rotation at 4.25 bits per parameter.

9 Beyond the Weight Matrix

Beyond weights, transform choice follows each tensor’s outlier geometry, axis structure, and runtime constraints rather than applying one universal flattening rule.

Scope: The toolkit extends to KV caches, attention matmuls, state-space models, mixture-of-experts, diffusion transformers, and training, with tensor-specific levers.Table 6 categorizes rotation, scaling, additive, reparameterization, routing, low-rank, and calibration-only interventions.
The key–value cache: Keys and values in the KV cache require different treatment: keys favor channel-wise quantization, whereas values favor token-wise quantization.KIVI and KVQuant exploit persistent key channels and the absence of equivalent value outliers.
The key–value cache: RotateKV applies a calibrated, permuted Walsh–Hadamard transform to both cache halves, while OSCAR makes the rotation data-aware.RoPE forces these rotations into the pre-rotary basis.
The key–value cache: PolarQuant reparameterizes cache vectors into polar coordinates and quantizes angles after random-rotation preconditioning.The preconditioning makes higher-level angle distributions analytically predictable and concentrated near π/4.
Attention matmuls: Attention probability matrices exploit bounded, row-normalized structure, motivating smoothing and rescaling designs distinct from weight quantization.SageAttention targets score and probability paths, while its FP4 successor rescales probability rows before block quantization.
State-space models: Quamba rotates selective-scan outputs online inside a fused kernel, with O(d log d) additions per token and an offline inverse absorbed into the output projection.State-space outliers differ from attention outliers, so the rotation targets the recurrence output.
Mixture-of-experts: For mixture-of-experts, the transform question is unchanged, but data-aware statistics must be gathered separately for each expert.The relevant transform remains optimal in the same sense once expert-specific statistics are supplied.
Diffusion transformers: Diffusion transformers make transform selection time-dependent because activation distributions drift across denoising timesteps.Methods range from timestep-diverse calibration to data-gated rotations and low-rank decompositions.

10 Systems and the Variable-Length Lane

Systems performance depends on where scaling and decoding occur: fused transforms can be cheap, but fine-grained W4A4 rescaling and variable-length coding face hardware boundaries.

Runtime cost: Fusing transforms with quantization reduces overhead substantially: five fused affine maps slow end-to-end inference by 0.07× versus 0.26× for separate QuaRot Hadamards.Fusion, rather than transform existence alone, accounts for the lower measured overhead.
Production deployment: Hadamard transforms and related stages now appear in production toolkits and tensor-core kernels as composable calibration modifiers.TensorRT Model Optimizer and vLLM expose methods including SmoothQuant, AWQ, GPTQ, QuaRot, SpinQuant, and QuIP.
Dequantization bottleneck: Per-group W4A4 scales varying along K require mid-reduction rescaling, whereas coarse scales can be applied once in the epilogue.The per-group path pays for independent weight and activation rescales inside the reduction.
Dequantization bottleneck: On an A100-class GPU, a CUDA-core operation costs about as much as 50 INT4 tensor-core operations, and partial-sum dequantization can consume 20–90% of runtime.Fusing dequantization and accumulation into the matmul is therefore central to W4A4 performance.
Four-bit lanes: W4A16 hides one-time weight expansion behind memory traffic, while coarse-scale W4A4 systems preserve an epilogue-cheap path and fine-group systems incur main-loop work.Granularity and operand count matter more than nominal bit-width alone.
Hardware datapaths: Hopper’s INT4 path can fall back to CUDA-core IMAD operations, while newer FP8 and FP4 microscaling tensor cores apply block scales natively.The hardware transition changes which low-bit formats can avoid software dequantization.
Hardware datapaths: NVFP4 training reports MMLU 76.6 versus 77.4 for an FP8 baseline at the same token budget, while matched MXFP4-versus-FP8 evidence at that scale remains unavailable.Open questions include profitable block-scale granularity and transform–hardware co-design.
Variable-length lane: Variable-length coding can approach Shannon limits for storage, but decoded tiles must become dense equal-width inputs before or during GEMM.Current implementations decode in software, inside kernels, or through pipelined hardware; the MMA instruction itself remains fixed-rate.

11 The Empirical Landscape

The empirical record shows that sufficient transforms substantially improve W4A4 activation quantization, while benefits depend on bit budget, format, granularity, rounding, and deployment overhead. Results across separately sourced protocols support consistent patterns rather than a unified leaderboard.

Activation quantization: 83.12 perplexity for SmoothQuant contrasts with 5.78–6.19 for several transform families, near the 5.47 FP16 floor.Table 10 places Atom, QuaRot, SpinQuant, FlatQuant, and PrefixQuant between 5.78 and 6.19, while DuQuant is 6.20.
Activation quantization: The transform ladder improves as degrees of freedom increase, with the best low-bit entry within 2.4 accuracy points and 0.22–0.91 perplexity of FP16.The ladder proceeds from diagonal scaling through fixed and learned rotations to learned affine maps.
Rounding interaction: GPTQ’s advantage shrinks as transforms strengthen: QuaRot changes from 8.56 with RTN to 6.10 with GPTQ, whereas FlatQuant changes from 5.79 to 5.78.The reported RTN-to-GPTQ gap decreases monotonically along the transform ladder.
Weight-only quantization: At 4-bit weight-only quantization, methods remain near FP16 at 5.56–6.15 versus 5.47, but tighter budgets make transforms more consequential.At 3 bits AWQ reaches 24.0, while QuIP# holds at 6.66; at 2 bits OmniQuant reaches 37.4.
Format flip: On MXFP4, global QuaRot falls to 62.9% average accuracy versus 69.3% for per-block RTN, while block-confined Hadamard improves accuracy to 70.5% with RTN and 73.7% with GPTQ.The comparison separates harmful global rotation from useful scale-block-confined rotation.
Format flip: On NVFP4, QuaRot recovers to 74.1% versus 74.7% RTN, while WUSH reports the best averages: 74.35% on MXFP4 and 76.1% on NVFP4.The NVFP4 block scale carries a mantissa, weakening the penalty associated with global rotation.
KV-cache quantization: KV-cache quantization is near FP16 at 4 bits but separates sharply at 2 bits, where KVQuant reaches 6.01 while naive per-token integer scaling reaches 11,779 perplexity.KVQuant combines pre-RoPE per-channel key quantization, a nonlinear codebook, attention-sink handling, and 1% full-precision retention.
Efficiency: QuaRot’s INT4 kernels report prefill speed-ups up to 2.16× on Llama-2-7B and 3.33× on the 70B model, while fused-transform online cost is 0.07× of runtime.Three separate online Hadamards cost 0.26×, nearly four times the fused-transform overhead.

12 Evaluation Pitfalls and Practical Guidance

The survey argues that transform comparisons require matched deployment conditions and honest accounting. Its practical guide therefore treats recommendations as starting hypotheses, indexed by quantizer granularity, rounder, format, axis, and runtime setting.

Evaluation pitfalls: Optimize the deployed AbsMax error rather than pre-quantization RMS or free-scale ℓ2 surrogates, because the group step is pinned by the maximum.The survey identifies surrogate mismatch as the most consequential evaluation mistake.
Evaluation pitfalls: Fix group size and scale rule across arms, since per-token, per-group, and per-tensor scales induce different crest factors.The achievable crest-factor ceiling depends directly on group size.
Evaluation pitfalls: Hold the rounder fixed, preferably using the stronger one, because transform quality otherwise becomes confounded with rounder weakness.The RTN-to-GPTQ gap shrinks sharply once a strong transform is applied.
Evaluation pitfalls: Do not transfer INT4 transform conclusions to FP4 or microscaling, and report effective bits including scales, codebooks, retained outliers, and transform parameters.The integer–floating-point flip makes format matching essential.
Evaluation pitfalls: Evaluate KV-cache methods at long context and during generation, because cache behavior may degrade separately from short-context perplexity.The survey treats short perplexity as an insufficient proxy for cache deployment behavior.
Practical guidance: Table 16 offers first choices by deployment scenario, but its entries synthesize heterogeneous models, tasks, group sizes, calibration sets, and kernels rather than a normalized benchmark.The recommendations should be tested first under one matched protocol.
Scenario-specific guidance: For correlated Mamba channels, KLT-preconditioned Hadamard decorrelates before flattening and reaches exactly tr Σ/d per channel.The recommendation contrasts with feature-axis rotation for row-sparse gradients, where row-axis mixing or leverage-score selection is preferred.

13 Open Problems and Conclusion

The conclusion frames the Great Inversion as a two-axis organizing principle while identifying unresolved joint optimization, hardware, format, and evidence problems. The survey’s strongest claims remain bounded by surrogate objectives and transcribed evaluations.

Open problems: The first open problem is jointly optimizing transform and rounding, since current optimality results hold one stage fixed.The desired object is a jointly optimal (T, Q) pair for deployed error.
Open problems: Most optimality theorems target surrogates rather than realized heavy-tailed shared-scale kernel error, leaving deployed-objective guarantees unresolved.The relevant surrogates include ℓ2 proxies, Gaussian models, and crest-factor bounds.
Hardware co-design: Allocation-flexible inference hardware could restore concentration as the relevant optimum, but whether that constraint will disappear remains unsettled.Current block-scaled tensor cores provide only restricted adaptive scaling within shared blocks.
Format co-design: Transform and format should be co-designed with block size, because power-of-two scales favor block-level rotation while mantissa-carrying scales weaken that pressure.Today’s block-scaled hardware fixes block sizes at 16 or 32, while prior work explores sub-block scaling.
Limitations: The survey reports no original benchmarks, and its empirical picture inherits source models, protocols, and within-table margins favoring proposing methods.Its own mechanism figures and Table 5 are illustrative rather than broad evaluations.
Conclusion: The Great Inversion organizes the field by allocation-and-scale constraints and number format, with operand alignment as an additional factor that rotations cannot change.The conclusion places transforms, rounders, codebooks, centering, and clipping alongside these axes rather than treating transform families alone as the main taxonomy.

A Notation

The notation section fixes the symbols used throughout the survey and relies on context to distinguish overloaded operators and matrices.

Notation: Table 17 collects the notation used throughout the survey, as fixed in Section 2.The table serves as the reference for subsequent symbols.
Notation: Q denotes an orthogonal matrix in transform sections, the quantizer when its argument is suppressed, and other context-specific objects elsewhere.Context distinguishes Q in transform, attention, and quantizer expressions.
Notation: P denotes the row-stochastic attention probability matrix, while elsewhere it denotes FlatQuant’s learned Kronecker factor or a microscaling element value.Subscripts and context separate these meanings.

B Glossary of concepts and formats

The paper distinguishes allocation-flexible coding from fixed-rate shared-scale quantization, which reward concentration and flattening respectively. It also defines function-preserving transform composition and contrasts INT4, FP4, and microscaling formats.

Rate and coding: Allocation-flexible quantization permits different per-coordinate rates, whereas shared-scale quantization assigns equal bits under one group scale.The two regimes differ independently from the element grid used by the quantizer.
The inversion: Concentration packs energy onto few coordinates and is optimal for variable-rate coding, while flattening spreads energy evenly for fixed-rate uniform-grid quantization.Hadamard transforms approach the flattening direction under the shared-scale objective.
Transforms and composition: A function-preserving transform is invertible and exactly undone on the other operand, leaving the layer output unchanged.Offline absorption can fold the inverse into consuming weights, while RMSNorm-specific orthogonal folding avoids materializing the transformed activation.
Precision lanes and formats: INT4 uses a uniform 4-bit grid, whereas FP4/E2M1 uses the non-uniform grid {0, 0.5, 1, 1.5, 2, 3, 4, 6}; MX uses shared block scaling.The format therefore forms an independent axis from the allocation regime.

C Complete index of cited works

The index accounts for the survey’s cited works by role and links classical transform-coding, quantization, neural compression, transform, rounding, format, tensor, and systems research. It also records selected theoretical guarantees, caveats, and implementation findings.

Complete index: Table 18 indexes 200 cited works, classifying each once by role while retaining per-domain appearances.The roles include classical foundation, transforms, rounding or codebooks, number formats, beyond-weights tensors, systems, and cross-cutting background.
Theory caveats: The KLT can be strictly suboptimal for non-Gaussian sources, while learned nonlinear transform codes can surpass linear KLT performance.The KLT distortion guarantee requires a same-shape-marginals hypothesis, with Gaussian sources as the familiar case.
Classical foundation: Classical entries cover KLT transform coding, rate-distortion theory, scalar and lattice quantization, entropy coding, and neural-network compression standards.The index connects these foundations to the modern LLM quantization literature.
Transform methods: Modern method entries span affine, rescaling, rotation, butterfly, frame, and other function-preserving or adjacent transform structures.The survey also records data awareness, search or construction strategy, runtime cost, granularity, and rounding composability where reported.
Rounding and codebooks: Rounding and codebook entries include GPTQ-related error-feedback bounds, water-filled grid spacings, trellis quantization, and nested-lattice results.One highlighted result reports a basis-free, rotation-immune scalar-INT optimum that provably beats equal-rate rounding.
Systems and formats: Systems entries document Hopper’s lack of tensor-core INT4 and native FP4 or FP6 microscaling support on newer hardware.The index also includes dequantization and kernel work such as Marlin, Machete, and LUT-GEMM.