Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions

• NACs convert continuous speech waveforms into discrete token sequences through learned codebooks, enabling sequence-modeling methods developed for natural language.Source: §I · p. 1
• The study asks whether NAC tokens exhibit language-like regularities and whether those regularities reveal codec quality and robustness.Source: §I · p. 1
• Prior work found Zipf-like behavior in NAC tokens, but examined only clean speech, one corpus, and a small set of RVQ codecs.Its links to resynthesis quality were also based on limited evidence.Source: §I · p. 1
• The paper evaluates 13 NACs spanning multi-codebook RVQ, single-codebook VQ, and non-VQ designs across three corpora and clean, white-noise, and DEMAND-noise conditions.Source: §I · p. 1
• Corpus identity contributes little, while acoustic condition and quantizer architecture shape token statistics in metric-dependent and family-specific ways.The reported degradation signatures concentrate differently across architecture families and noise conditions.Source: §I · pp. 1–2

• Statistical laws originally studied in language have also been reported for visual codec tokens and linked to downstream task behavior.Source: B. Statistical Laws of Language · p. 2
• Prior speech-token studies examined interpretability, redundancy, and cross-domain distributions, including SSL-based semantic tokens and NAC tokens.Source: C. Statistical Analysis of Speech Tokens · pp. 2–3
• Earlier NAC analysis found Zipf and Heaps behavior but was restricted to one clean corpus, with modest condition-level correlations and unresolved reliability and generalization questions.Source: C. Statistical Analysis of Speech Tokens · p. 3
• Noise robustness matters because real-world audio includes environmental degradation, while existing evaluation frameworks focus predominantly on clean inputs.Source: D. Noise Robustness of Neural Audio… · p. 3
• The present design evaluates 13 codecs across three corpora and clean, white-noise, and DEMAND-noise conditions using standardized extraction and family-conditional processing.The pipeline includes deduplication, n-gram mapping, chunk-based estimation, and variance decomposition.Source: §III · p. 3, A. Token Extraction Pipeline · p. 3

1) Zipf’s Law

• Zipf’s law models the frequency of the r-th ranked element as a power law f(r) ∝ r^-s.Source: B. Statistical Metrics · p. 3, 1) Zipf’s Law · p. 3
• The paper reports the equivalent frequency-distribution exponent α = 1 + 1/s, with natural-language word-level reference α ≈2.Source: 1) Zipf’s Law · p. 3
• Zipf exponent α is estimated by maximum likelihood over type frequencies, using xmin and the number of qualifying types n.Fit quality is assessed with KS distance; estimates are refit when the default bound truncates α at 3.Source: 1) Zipf’s Law · pp. 3–4

• Fits are treated as undefined when the type–token ratio V/N exceeds 0.9 because the observed inventory is nearly unique and the frequency profile is degenerate.Source: 1) Zipf’s Law · p. 4

2) Heaps’ Law

• Heaps’ law models vocabulary size as V(N) = K · N^β, where K reflects initial vocabulary introduction and β controls continued type growth.Natural-language word-level references typically have β ≈0.4–0.6.Source: 2) Heaps’ Law · p. 4

3) Shannon Entropy and Redundancy

• Shannon entropy measures average information per token from token probabilities, while redundancy measures the gap from maximum entropy over the observed vocabulary.Token perplexity 2^H re-expresses entropy as an effective number of frequently used token types.Source: 3) Shannon Entropy and Redundancy · p. 4

• JSD compares empirical n-gram frequency distributions between clean reference P and degraded distribution Q.It is nonparametric rather than based on fitted α or β parameters.Source: 4) JSD · p. 4
• JSD is symmetric, bounded in [0, 1], and defined even for non-overlapping supports.The mixture distribution is used with Kullback–Leibler divergences in the definition.Source: 4) JSD · p. 4
• Resynthesized speech is evaluated after decoding complete token sequences and resampling the output to 16 kHz.Metrics are computed on the resynthesized waveform against the clean reference where applicable.Source: C. Quality Metrics · p. 4
• Quality evaluation includes dCER, UER, UTMOS, MCD, and F0 error.UER is defined only under noisy conditions, while UTMOS estimates perceived naturalness and MCD measures mel-cepstral distortion.Source: C. Quality Metrics · p. 4

• Zipf and Heaps parameters are estimated on non-overlapping chunks from 5K to 500K tokens against matched 500K-token references.Deviations from the largest reference quantify finite-sample stability.Source: 1) Chunk Analysis · p. 4, 3) Variance Decomposition · p. 5
• Codebook occupancy measures the number of unique observed tokens normalized by an estimated codebook size.For codecs without a nominal vocabulary size, the denominator uses the largest observed token index plus one.Source: 2) Codebook Occupancy · p. 4
• The design crosses 13 NACs with three corpora and three acoustic conditions, yielding 117 codec–corpus–condition token streams.The protocol includes clean, white-noise, and DEMAND-noise conditions.Source: 3) Variance Decomposition · p. 5
• The pipeline applies consecutive deduplication and family-conditional n-gram orders before chunk-based estimation.The accompanying variance analysis uses codec quantizer category, corpus, and noise condition as factors.Source: 2) Codebook Occupancy · p. 4, 3) Variance Decomposition · p. 5
• Reported η2 values are descriptive variance partitions over the cell grid, with a codec-random-intercept mixed model as a robustness check.Cells contributed by each codec are not treated as independent samples.Source: 3) Variance Decomposition · p. 5

• All 13 codecs are evaluated in default released configurations across 117 codec–corpus–condition cells.The setup avoids manual bandwidth adjustment.Source: A. NACs · p. 5
• The codec set spans multi-codebook RVQ, single-codebook VQ, and non-VQ alternatives.The categories are defined by quantizer topology.Source: A. NACs · p. 5, Table I · p. 5
• The three corpora cover single-speaker monolingual, multi-speaker, and multilingual speech variation.They are approximately matched at ∼10 hours, and all codecs use the full corpus set.Source: B. Speech Corpora · p. 5, Table II · p. 5
• Each corpus is tested clean, with Gaussian white noise at 0 dB SNR, and with DEMAND real-world noise at 0 dB SNR.DEMAND conditions pool five environments, random channels, and random start offsets using identical noisy waveforms across codecs.Source: C. Noise Conditions · p. 5

A. Finite-Sample Stability

• 114 of 117 cells have matched 500K-token references; three Mimi–TrIJEK cells are excluded because their streams are shorter.Heaps intercept K uses absolute log-ratio deviations because its scale varies across codecs.Source: A. Finite-Sample Stability · p. 6
• At 200K tokens, KS median deviation falls from 0.55 to 0.07 reference spreads, while β, α, and K remain at 0.17, 0.17, and 0.19.KS is the only parameter reaching the 0.1 guide; α is least stable at 5K tokens with deviation 1.06.Source: A. Finite-Sample Stability · p. 6

B. Selection and Sensitivity of n-gram Order

• Most multi-codebook RVQ codecs favor n = 3 or 4, whereas single-codebook and large-vocabulary codecs favor lower orders.Higher-order inventories can approach V ≈ N, making fits undefined.Source: B. Selection and Sensitivity of n-gram… · p. 6, Figure 3 · p. 6
• The adopted n-gram order matches the corpus-mean KS optimum for eight codecs, differs by one for four, and by two for Mimi.Cross-family differences under codec-specific orders are treated as descriptive because valid fits become sparse at common higher orders.Source: B. Selection and Sensitivity of n-gram… · p. 6, Table III · p. 7

1) RVQ Level

• For RVQ codecs, the first residual-quantizer level has lower KS distance than later levels in the five codecs with level-wise results.The first level is therefore retained for the full codec set.Source: 1) RVQ-Level Selection · p. 6

2) Deduplication

• Consecutive deduplication increases estimated α and β with Cohen’s d ≈0.5 but does not significantly change KS or K.This convention is applied before fitting the distributional models.Source: 2) Deduplication · p. 6, C. Pipeline Validation · p. 6

• RVQ-family averages remain similar across LJSpeech (α = 2.32, β = 0.84), VoxCeleb (α = 2.44, β = 0.91), and TrIJEK (α = 2.42, β = 0.91).Source: A. Distributional Landscape · p. 7
• Multi-codebook RVQ codecs span α ≈2.0–2.7, while SQCodec and FocalCodec sit near α ≈2.8–3.0 and S3Tokenizer at 2.41.Source: A. Distributional Landscape · p. 7
• On clean speech, entropy is positively associated with vocabulary scale, while every codec uses its observed inventory close to uniformly and the single-codebook VQ group and FocalCodec are lowest.Source: B. Entropy, Redundancy, and Perplexity · p. 7
• White noise removes about 1–4.4 bits of entropy for most codecs and raises grid-mean redundancy from 0.08 to 0.27, whereas DEMAND noise changes entropy comparatively little.Source: B. Entropy, Redundancy, and Perplexity · p. 7
• The median absolute clean-to-0 dB shift in α is 0.35 under white noise and 0.23 under DEMAND noise, with affected cells shifting by up to 1.91.Source: C. Clean-to-Noisy Distributional Shifts · p. 8
• White noise depresses α most for single-codebook VQ codecs, with ∆α = −1.02 versus −0.36 for RVQ and 0.00 for non-VQ cells.Source: C. Clean-to-Noisy Distributional Shifts · p. 8
• Cross-noise robustness rankings are metric-dependent: UER rankings agree strongly (ρ = 0.81), occupancy shifts moderately (ρ = 0.65), and α shifts not at all (ρ = −0.04).Source: C. Clean-to-Noisy Distributional Shifts · p. 8

• Acoustic condition is the largest main effect for α, β, and redundancy R, while meta-category leads for KS, Heaps intercept K, and unigram entropy H.Source: D. Variance Decomposition · p. 8
• The meta×condition interaction explains appreciable variance for α, KS, and R, indicating family-specific responses to acoustic condition, while residual variance remains large.Source: D. Variance Decomposition · p. 8
• After including codec random intercepts, the joint noise effect and meta×condition interaction remain significant for every metric (Wald p ≤0.001).Source: D. Variance Decomposition · pp. 8–9
• Corpus identity explains little variation across the three-corpus grid, with corpus terms reaching significance only for β and remaining below the noise terms.Source: D. Variance Decomposition · pp. 8–9
• Clean-speech dCER is at most 0.06 for every codec except WavTokenizer, which reaches 0.21 despite MCD and F0 errors within the remaining codecs’ range.Source: A. Clean-Speech Quality Overview · p. 9
• XCodec2 and BigCodec exceed reference UTMOS on TrIJEK and VoxCeleb by up to 0.5, while increasing reference-based spectral distance.Source: A. Clean-Speech Quality Overview · pp. 9–10

• JSD compares equal-sized clean and noisy token samples at the common unigram order n = 1, avoiding saturation and invalidity at most higher orders.Source: B. JSD and Quality under Noise · p. 10
• Under DEMAND noise, larger JSD is associated with lower UTMOS and larger median F0 error, while the MCD association is weak after level correction.Source: B. JSD and Quality under Noise · p. 10
• The DEMAND-noise UTMOS association persists after removing two non-VQ codecs, with r = −0.68 and p = 0.029.Source: B. JSD and Quality under Noise · p. 10
• Under white noise, no association with audio-based quality metrics survives both correlation statistics; dCER associations remain weak throughout.Source: B. JSD and Quality under Noise · p. 10
• The weak DEMAND-noise JSD–UER association partly reflects uniformly high token degradation, with UER cell means spanning 0.66–1.00.Source: B. JSD and Quality under Noise · p. 10
• The sequence analysis asks whether noise produces different degradation patterns across quantizer architectures using repetition rate and transition entropy.Source: §VIII · p. 10, A. Operational Definitions · p. 10

• RVQ codecs cluster in the collapse quadrant under white noise and the explosion quadrant under DEMAND noise.Source: B. Shifts by Quantizer Architecture · p. 11
• Single-codebook VQ codecs lose 33–75 percentage points of occupancy per cell under white noise while remaining near zero repetition-rate shift, so they satisfy neither signature.Source: B. Shifts by Quantizer Architecture · p. 11
• Non-VQ codecs show no collapse signature; S3Tokenizer crosses the explosion deadband in four of six cells, while FocalCodec and SQCodec remain neutral.Source: B. Shifts by Quantizer Architecture · p. 11
• Of 15 collapse instances, 14 occur under RVQ–white, while explosion concentrates under RVQ–DEMAND and four additional explosion cells belong to S3Tokenizer.Source: C. Distribution of Degradation… · p. 11
• Cross-codec comparisons should be conducted at fixed meta-category, n-gram order, and, where possible, vocabulary scale because these factors confound between-family contrasts.Source: A. Architecture and Condition as the… · pp. 11–12
• The interpretation of α near the natural-language reference value should be family-conditional rather than treated as a universal target.Source: A. Architecture and Condition as the… · p. 12

• JSD shows the clearest association with acoustic quality under real-world DEMAND noise.It tracks UTMOS and median F0 error in the expected directions.Source: B. JSD as a Complementary… · p. 12, §X · p. 12
• JSD detects token-frequency redistribution that can preserve overall power-law shape while changing individual token frequencies.Unlike α or KS, it responds to probability-mass redistribution across the full token frequency distribution.Source: B. JSD as a Complementary… · p. 12
• JSD is decoder-free and can evaluate nondecodable tokenizers such as S3Tokenizer.It uses token-level counts rather than waveform reconstruction.Source: B. JSD as a Complementary… · p. 12
• Cross-codec JSD requires a matched low n-gram order because the measure saturates when compared samples share little n-gram support.This is one of the paper’s scope conditions for interpreting JSD across codecs.Source: D. Limitations · p. 12
• The broader protocol combines family-conditional n-gram orders, fit-validity safeguards, matched-sample chunk assessment, and variance decomposition.These conventions make language-statistical analysis family-conditional rather than a universal target.Source: §X · p. 12, D. Limitations · p. 12