PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization

Xianghong Fang, Wenlong Mou, Yuan Yuan, Dehan Kong, Tim G. J. Rudner

arXiv:2609.02107v1cs.LGcs.CV

TL;DR

Discrete visual tokenization lacks a unified framework for understanding quantization tradeoffs, while existing comparisons can confound quantizer effects with surrounding system differences. The paper uses rate-distortion theory with distortion as quantization error and rate as the nominal fixed-length coding budget, finding that distortion minimization is primary and that modern VQ achieves the lowest distortion under matched conditions.

  • Problem

    Existing tokenizer comparisons vary surrounding architectures, training setups, data, optimization, and computational budgets, while the relationship between codebook utilization, distortion, and quantization effectiveness remains less understood.

  • Method

    The paper defines distortion as quantization error and rate as R = T log2 K, then compares VQ, PQ, and SQ under matched latent distributions and nominal coding rates.

  • Results

    Under controlled comparisons, distortion is more strongly associated with reconstruction fidelity than codebook utilization, and modern VQ methods achieve lower distortion and better reconstruction than PQ and SQ.

  • Takeaways & Limitations

    Rate-distortion analysis provides a principled framework for evaluating intrinsic quantization effectiveness under fixed-rate constraints.

  • Takeaways & Limitations

    The distortion-minimization implication of full codebook utilization assumes mild conditions and excludes distributions exactly representable by at most K codewords.

Abstract

from arXiv · show

Discrete visual tokenization, predominantly driven by vector, scalar, and product quantization, lacks a unified conceptual framework for understanding quantization tradeoffs. In this paper, we propose a unified rate--distortion perspective on modern discrete visual tokenization. By viewing quantization as lossy compression, we characterize the nominal fixed-length coding rate through token count and codebook size, and quantization error as the distortion. Within this framework, we resolve three central questions. First, we theoretically and empirically show that minimizing distortion, rather than maximizing codebook utilization, is the primary intrinsic objective for reconstruction fidelity, with a direct connection to the STE-induced gradient discrepancy. Second, we establish two critical fairness conditions for intrinsic quantization comparison: controlling latent feature statistics and enforcing identical coding rates. Third, under these conditions, we recover the VQ--PQ--SQ distortion hierarchy in modern visual tokenization and show empirically that modern VQ methods achieve the lowest distortion. This work provides a foundational rate--distortion reframing of modern discrete visual tokenization, resolves ambiguities in quantizer evaluation, and provides a controlled framework for isolating intrinsic quantization effectiveness under fixed-rate constraints.

1 Introduction

The paper reframes discrete visual quantization through rate-distortion theory, identifying distortion as more fundamental than codebook utilization and specifying conditions for fair intrinsic comparisons. Under matched latent distributions and coding rates, it recovers the VQ–PQ–SQ hierarchy and finds modern VQ methods perform best.

  • A unified rate-distortion framework characterizes quantization error as distortion and the nominal fixed-length coding budget through token count and codebook size.This reframes vector, product, and scalar quantization as lossy compression.
  • Distortion minimization is more fundamental than codebook utilization for reconstruction fidelity and, under mild conditions, implies full codebook utilization.The paper also connects distortion to STE-induced gradient discrepancy and reports stronger correlations with reconstruction fidelity than utilization.
  • Fair intrinsic comparisons require matched latent feature distributions and identical nominal fixed-length rates, R = T log2 K.Latent rescaling changes minimum squared distortion proportionally with latent variance, while rate depends jointly on token count and code-space cardinality.
  • VQ, PQ, and SQ form a nested hierarchy, SQ ⊆ PQ ⊆ VQ, giving VQ greater modeling flexibility and no higher optimal distortion.VQ can additionally exploit low-dimensional source structure for better distortion scaling than fixed coordinate-factorized PQ and SQ.
  • Controlled experiments show modern VQ methods achieve the lowest distortion and best reconstruction after decoder adaptation.These results address earlier reports of inferior VQ performance associated with codebook collapse.

2 Background

Discrete visual tokenizers encode images through an encoder, quantizer, and decoder, using VQ, PQ, or SQ to impose different structures on latent representations. VQ quantizes whole vectors jointly, whereas PQ and SQ progressively factorize the representation.

  • A typical discrete visual tokenizer consists of an encoder Eθ, a quantization module Qϕ, and a decoder Dφ that reconstructs the image from quantized latents.The resulting discrete symbols can also serve as visual tokens for generative modeling.
  • The quantized representation is decoded as x̂ = Dφ(zq), while the discrete symbols are retained for downstream generative modeling.Different quantization algorithms differ in how they map continuous encoder features ze to quantized features zq.
  • VQ jointly quantizes each d-dimensional feature vector using one of K d-dimensional code vectors.Its principal optimization challenge is codebook collapse, in which only a subset of vectors is frequently selected.
  • Product Quantization: PQ factorizes a d-dimensional feature vector into M lower-dimensional subvectors and quantizes each independently.Its composite code space has the product of subcodebook sizes possible codewords, although only subcodebook entries are explicitly stored.
  • Scalar Quantization: SQ independently discretizes scalar components and is the maximally factorized case of PQ with M = d.FSQ, LFQ, and BSQ instantiate this scalar factorization with finite levels, binary values, or binary values with hyperspherical normalization.

3 Toward a Controlled Comparison of Quantization Algorithms

Existing tokenizer comparisons often vary architecture, training, data, optimization, and computational budget, preventing clean attribution of reconstruction differences to the quantizer. These confounders motivate controlled questions about optimization objectives, fair comparison conditions, and achievable distortion.

  • Independently trained tokenizer comparisons do not isolate quantizer contributions because surrounding training systems can differ substantially.Relevant factors include encoder–decoder capacity, discriminator design, data, optimization schedule, and computational budget.
  • Table 1 summarizes implementation differences that can make reconstruction-performance gaps reflect model capacity, objectives, data, or optimization resources rather than quantization alone.Such comparisons remain informative for complete tokenizer systems but are insufficient for intrinsic VQ, PQ, and SQ evaluation.
  • The resulting research questions ask what quantization should optimize, which conditions enable fair intrinsic comparison, and how VQ, PQ, and SQ compare in achievable distortion.The paper answers these questions using a unified rate-distortion perspective.

4 A Rate-distortion Perspective

The paper reframes discrete quantization as fixed-rate lossy compression, using nominal coding rate and squared quantization error to analyze objectives, fairness, and VQ–PQ–SQ effectiveness.

  • Rate and distortion: Nominal coding rate is R = T log2 K, where T is token count and K is code-space cardinality.Each token carries log2 K nominal bits, distinct from entropy-coded rate.
  • Rate and distortion: Quantization distortion is expected squared Euclidean error and directly measures information lost during quantization.Empirical evaluations approximate the expectation by averaging squared errors over observed latent features.
  • Optimization objective: Lower distortion locally reduces STE gradient discrepancy, making gradients at quantized representations closer to those at continuous representations.The squared gradient discrepancy vanishes as distortion approaches zero under the stated local smoothness condition.
  • Optimization objective: Distortion minimization is more fundamental than utilization maximization: under mild conditions, every global minimizer fully utilizes the codebook, but full utilization does not ensure optimal distortion.The latter claim is supported by quantizers with U(g) = 1 but E(g) above the minimum.
  • Fair comparison: Fair intrinsic comparisons require matched latent distributions and identical nominal coding rates.Global rescaling changes optimal squared distortion proportionally to latent variance, while differing T and K alter representational capacity.
  • VQ, PQ, and SQ: Under equal coding rates, VQ generalizes PQ and PQ generalizes SQ, yielding no higher optimal distortion for VQ and strictly better scaling on low-dimensional structures.For worst-case distributions, all three scale as Θ(d/K2/d), whereas low intrinsic dimension can favor VQ.

5 Empirical Evaluation

The experiments compare VQ, PQ, and SQ under controlled latent statistics and coding budgets. Across ImageNet-1K rate–distortion evaluations, VQ consistently achieves the lowest distortion and strongest reconstruction results.

  • Experimental setup: The primary ImageNet-1K experiments use a shared pretrained encoder and representation pipeline, keeping latent source distributions matched across quantizers.The encoder and decoder are frozen during quantizer substitution, and decoder adaptation follows afterward.
  • Experimental setup: The comparison evaluates VQ, PQ, and SQ families under matched token count and composite code-space cardinality, reporting distortion, utilization, and reconstruction metrics.Reported reconstruction metrics include PSNR, SSIM, LPIPS, r-FID, and r-IS.
  • Controlled latent-space results: 0.201 is the best VQ distortion on ImageNet-1K, compared with 0.209 for PQ and 0.231 for SQ.The comparison is conducted under matched latent feature statistics, token count, and code-space cardinality.
  • Controlled latent-space results: 0.86 is the best VQ r-FID, compared with 0.93 for PQ and 1.07 for SQ after decoder adaptation.Several PQ and SQ methods reach 100% codebook utilization despite higher distortion.
  • Rate–distortion curves: Across evaluated coding rates, VQ has the lowest distortion at every operating point, followed by PQ and SQ.Increasing coding rate monotonically reduces distortion for all three families.
  • Additional analyses: Increasing either code-space cardinality K or token count T reduces distortion, while equal-rate allocations produce similar but not identical performance.The study also validates the qualitative findings across datasets and in pixel space.

6 Conclusion

The paper reframes vector, product, and scalar quantization through rate-distortion analysis, identifying distortion minimization, matched comparisons, and a VQ–PQ–SQ hierarchy as central conclusions.

  • Main conclusions: Distortion minimization is more fundamental than codebook utilization and, under mild conditions, implies full utilization while controlling the local STE-induced gradient discrepancy.The experiments also associate lower distortion with better reconstruction performance.
  • Main conclusions: Fair intrinsic comparisons require matched latent distributions and nominal coding rates determined by token count and code-space cardinality.These controls isolate quantizer effectiveness from feature-statistics and representation-budget differences.
  • Main conclusions: Under matched representation budgets, modern VQ methods achieve lower distortion and better reconstruction performance than PQ and SQ.This empirical result supports the theoretical ordering among the quantizer families.

A Proof of Proposition 2

This section proves the distortion–utilization relationship and develops bounds explaining the structural separation among VQ, PQ, and SQ.

  • Distortion and utilization: The proof defines quantization using a measurable code assignment and evaluates squared-error distortion through optimal conditional-mean reconstructions.After substituting conditional means, distortion equals expected conditional variance.
  • Distortion and utilization: A distortion minimizer that leaves a codeword unused can split a positive-variance cell and strictly reduce distortion, contradicting optimality.The contradiction establishes full codebook utilization under the stated non-degeneracy and nonzero-distortion conditions.
  • Distortion and utilization: Every global distortion minimizer uses all K codewords, whereas a full-utilization quantizer need not be distortion-optimal.The converse fails through an explicit K = 2 uniform-distribution construction.
  • Quantizer bounds: For bounded d-dimensional sources, SQ and VQ admit K^-2/d distortion scaling, with VQ lower bounds demonstrated on the uniform cube.The construction and lower-bound arguments use coordinatewise scalar quantization and geometric second-moment bounds, respectively.
  • Quantizer bounds: VQ can exploit lower-dimensional source structure, while fixed coordinate-block PQ incurs a separation whose scaling depends explicitly on the number of blocks M.The analysis provides an explicit non-asymptotic VQ–PQ separation for an axis-misaligned one-dimensional source.
  • Implications: The framework connects distortion to STE optimization and compares VQ, PQ, and SQ under matched source representations, token counts, and composite code-space cardinalities.Locally, the STE gradient discrepancy vanishes quadratically with quantization distance under standard smoothness.

F Effect of Latent Scale on Quantization Distortion

The section shows that optimal squared quantization distortion changes predictably with latent scale, motivating matched latent distributions for fair quantizer comparisons.

  • Comparison implications: The scale analysis motivates controlling latent feature distributions when comparing intrinsic quantizer effectiveness.Without this control, feature-scale differences can confound distortion comparisons.
  • Scale dependence: Global scaling changes optimal squared quantization distortion homogeneously with degree two, without requiring Gaussian sources or high-rate approximations.This is the content of Proposition 5.
  • Scale dependence: When latent-distribution shape is fixed and only scale varies, optimal distortion grows linearly with source variance σ^2.Translation can be absorbed by an equal translation of the codebook.
  • Scope boundaries: The framework uses nominal fixed-length rate rather than entropy-coded bitstream length, so it does not directly characterize entropy-constrained or variable-length coding.Adaptive bit allocation and symbol-probability effects remain outside this analysis.
  • Scope boundaries: The theory uses squared Euclidean distortion and concerns optimal family-level distortion, not guaranteed superiority of every practical VQ method.Perceptual, semantic, task-dependent, finite-data, and finite-optimization regimes are not directly covered by this claim.
  • Scope boundaries: The conclusions address intrinsic quantization and reconstruction rather than downstream generative performance, which also depends on representation and generative-model properties.Improved reconstruction fidelity does not necessarily imply improved generation quality.

H Experimental Details

The experiments use controlled latent- and pixel-space settings to compare VQ, PQ, and SQ implementations under matched architectural and coding conditions.

  • Latent-Space Setup: Latent-space experiments use a pretrained VAR tokenizer and fixed encoder-decoder architecture to compare quantization algorithms within the same latent representation.The latent encoder produces 16 × 16 × 32 features, and the quantization pipeline yields 512 tokens.
  • Training Details: Training uses AdamW and controlled learning-rate schedules, loss weights, and adaptation procedures across the latent-space experiments.The latent objective uses perceptual, adversarial, and distribution-matching terms with specified weights.
  • Quantizer Implementations: Each latent 16-dimensional token is processed by VQ, PQ, or SQ modules, with PQ further partitioning tokens into two 8-dimensional sub-vectors.The compared VQ variants include Vanilla, EMA, Online, Wasserstein, and MMD VQ; the corresponding VP2 variants are used for PQ.
  • Pixel-Space Setup: Pixel-space experiments apply PixelUnshuffle to convert 256 × 256 × 3 images into 64 × 64 × 48 representations before convolutional processing.This setting performs quantization without a pretrained encoder-decoder.
  • Rate Control: The latent-space comparison fixes T = 512 tokens and K = 65,536 per token, while pixel-space experiments use T = 4096 with the same K.Both settings therefore control the nominal coding configuration across methods.

I Rate–Distortion Analysis of Code-Space Cardinality and Token Count

The rate–distortion analysis separates the effects of code-space cardinality and token count while also comparing alternative allocations at matched nominal rates.

  • Varying Code-Space Cardinality: Increasing K from 1,024 to 65,536 at fixed T = 512 reduces distortion E from 0.318 to 0.201 and improves r-FID from 1.90 to 0.86.Both changes indicate improved reconstruction as the nominal representation budget increases.
  • Varying Token Count: Increasing T from 256 to 2,048 at fixed K = 16,384 reduces distortion from 0.369 to 0.035 and improves adapted r-FID from 3.06 to 0.42.Token count can therefore substantially reduce information loss under this fixed-cardinality sweep.
  • Matched-Rate Allocations: Under R = T log2 K, doubling T and squaring K preserve the same nominal fixed-length coding rate.This identity concerns representation budget rather than identical distortion.
  • Matched-Rate Allocations: Matched-rate configurations achieve similar, though not identical, distortion and reconstruction performance because different rate allocations can induce different representation structures.The compared pairs include (T, K) = (512, 16,384) versus (1,024, 128), and (512, 65,536) versus (1,024, 256).

J Generalization Across Datasets

Controlled evaluations on FFHQ and CelebA-HQ reproduce the main quantization comparisons and the stronger relationship between distortion and reconstruction fidelity.

  • FFHQ: On FFHQ, the best VQ, PQ, and SQ distortions are 0.131, 0.136, and 0.155, with best r-FID values of 0.85, 1.05, and 1.54.All methods use T = 512 and K = 65,536 in the controlled latent-space protocol.
  • CelebA-HQ: On CelebA-HQ, the corresponding best distortions are 0.111, 0.115, and 0.133, with best r-FID values of 1.73, 1.96, and 2.24.The ordering remains VQ, then PQ, then SQ for both metrics.
  • Distortion–Reconstruction Relationship: On FFHQ, distortion correlates strongly with r-FID (ρ = 0.979, p = 5.49 × 10^-9), while utilization has a much weaker correlation (ρ = −0.492, p = 0.088).CelebA-HQ shows the same pattern: ρ = 0.944 for distortion and ρ = −0.559 for utilization.

K Controlled Quantization Comparison in Pixel Space

Pixel-space experiments remove dependence on a pretrained latent representation while preserving the reported VQ advantage and distortion–reconstruction relationship.

  • Controlled Pixel-Space Protocol: The pixel-space setting quantizes shared image-derived representations, reducing dependence on a particular pretrained feature distribution.All methods use the same PixelUnshuffle operation and convolutional projectors.
  • Quantization Performance: MMD VQ achieves the lowest distortion, E = 0.0021, compared with 0.0023 for the best PQ variant and 0.0032 for the best SQ variant.The comparison is conducted on CelebA-HQ under the controlled pixel-space setting.
  • Reconstruction Performance: MMD VQ achieves the best r-FID of 3.64, compared with 3.77 for the best PQ method and 4.54 for the best SQ method.The reconstruction ordering matches the distortion ordering in this experiment.
  • Distortion–Reconstruction Relationship: Pixel-space distortion correlates strongly with r-FID (ρ = 0.947, p = 2.91 × 10^-6), whereas utilization has a weaker, statistically insignificant correlation (ρ = −0.413, p = 0.182).These results support the same conclusion as the latent-space analysis.
Loading 2609.02107v1…