Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures

1. INTRODUCTION

• One protocol compares three core models, eight replication models, and two models quantized blind after the procedure was fixed.The comparison covers heterogeneous pretrained TTS pipelines and quantized-component scopes.Source: §1 · p. 1
• Component ablations reveal model-specific weight sensitivity and interactions that can amplify whole-model degradation; mixed precision and per-layer GPTQ can protect the sensitive component.Source: §1 · p. 1
• Activation sensitivity depends on both scale granularity and which components are quantized; per-channel 8-bit activations add at most 0.09 UTMOS loss to selected core-model 4-bit configurations.Source: §1 · p. 1
• Real int8 and int4 measurements show that peak memory, latency, and energy depend on runtime and hardware.This motivates testing a staged procedure on held-out models.Source: §1 · p. 1

2. RELATED WORK

• LightSpeech explores architecture search, while acoustic-model int8 PTQ paired with architectural compression is studied in.Source: §2 · p. 1
• BitTTS uses ternary quantization and weight indexing for compact TTS, and identifies its model’s vocoder as sensitive.Source: §2 · p. 1
• PTQ4ADM and timestep-aware quantization extend diffusion-model PTQ to audio, while HAWQ and Diet-KIT select precision by sensitivity in vision and speech translation.Source: §2 · p. 1

3. EXPERIMENTAL SETUP

• The core systems are Supertonic V3, OmniVoice, and Kokoro; replication models span flow-matching, feedforward, and autoregressive codec-token architectures, with Chatterbox and VoxCPM held out for blind evaluation.Evaluation uses 200 fixed FLORES-200 devtest sentences per evaluated language, with per-utterance seeds and each condition paired to its native-precision baseline on the same platform and sampling-step count.Source: §3 · pp. 1–2
• Weight sweeps use symmetric round-to-nearest quantization at 8, 6, and 4 bits, with scales shared per tensor, output channel, or group of 128 weights.The study quantizes whole models and individual components, tests mixed precision, and calibrates selected components with per-layer GPTQ or activation-aware weight scaling.Source: §3 · p. 2
• Dynamic 8-bit activations are tested with selected weight configurations, while real 4-bit execution uses ONNX Runtime on Supertonic and torchao on selected language models.Source: §3 · p. 2
• English naturalness is assessed with UTMOS, intelligibility with WER or Korean CER, and secondary naturalness proxies with NISQA-TTS and DNSMOS P.835.Paired 95% bootstrap intervals resample the 200 sentences; the paper defines quality preservation using paired UTMOS and WER thresholds for the two English proxies.Source: §3 · p. 2
• CPU runs measure real-time factor, peak resident memory, and whole-chip energy; GPU energy is measured without host power, with loading and compilation excluded.CPU energy includes model loading, whereas CPU real-time factor counts synthesis only.Source: §3 · p. 2

4.1. Scale granularity and bit width

• 4-bit per-channel weights reduce UTMOS by 2.8 on Supertonic and 0.07 on Kokoro, while OmniVoice degrades substantially; at 4 bits, OmniVoice UTMOS rises from 1.363 with per-tensor scaling to 3.761 with group:128 scaling.Per-tensor scaling also yields -3.15 ∆UTMOS on Kokoro and -1.642 on W8 Supertonic; the cited WER change for Kokoro is 0.030 to 0.261.Source: §4.1 · p. 2

4.2. Component sensitivity and mixed precision

• At W4, Supertonic’s vocoder alone yields -2.87 ∆UTMOS, while keeping it at W8 with the rest at W4 restores UTMOS to 4.252 against 4.473 at fp.For OmniVoice, the codec decoder’s -0.68 loss exceeds the token head’s -0.17 and the LM’s -0.28.Source: §4.2 · pp. 2–3
• Figure 2 compares W4 ∆UTMOS against fp at matched NFE and against a single default-step fp baseline, alongside matched relative WER penalty.Source: Figure 2 · p. 3
• Two OmniVoice component pairs lose more UTMOS together than the sum of their individual losses: -0.99 for token head with codec and -0.93 for LM with token head.Group:128 improves the decoder-only result to -0.12, whereas Supertonic’s vocoder remains degraded.Source: §4.2 · p. 3
• Across replication models, group:128 reduces degradation for Zonos, StyleTTS 2, and F5-TTS without establishing equivalence to fp, but has no effect on Kyutai.Kyutai’s depth transformer remains at -2.99 ∆UTMOS under group:32 and per-channel scaling; 6-bit and 8-bit weights yield -1.38 and -0.02.Source: §4.2 · p. 3

4.3. Sampling steps and matched baselines

• Matching each quantized run to an fp baseline at the same NFE removes the confound created by using one default-step baseline.Comparing every step count against a single default-step fp baseline makes OmniVoice quantization degradation appear to decrease as the number of steps rises.Source: §4.3 · p. 3
• Under W4, additional steps reduce the WER penalty without closing the UTMOS gap.The UTMOS gap widens with the number of steps on Supertonic.Source: §4.3 · p. 3

4.4. Calibration controls

• On OmniVoice’s LM, W4 group:128 GPTQ yields -0.029 ∆UTMOS versus -0.093 for RTN, while activation-aware scaling yields -0.100.For the token head and codec together, α = 0.25 improves per-channel degradation to -0.64 from -0.99, but remains worse than uncalibrated group:128 at -0.39.Source: §4.4 · p. 3
• No tested activation-aware scaling strength recovers the Supertonic vocoder, whereas per-layer GPTQ recovers the tested F5-TTS and Kyutai components.The reported comparisons frame calibration as the recovery step for sensitive components, not merely a refinement of already-survivable configurations.Source: §4.4 · p. 3
• At group:128, per-layer GPTQ reduces F5-TTS vocoder and Kyutai depth-transformer degradation to -0.07 and -0.08 ∆UTMOS, versus -0.48 and -2.98 for RTN.WER stays at fp level in these comparisons; the table reports paired ∆UTMOS against each model’s own fp baseline.Source: §4.4 · p. 3, Table 1 · p. 3

4.5. Activation quantization

• Per-tensor A8 on Supertonic’s vocoder yields -2.98 ∆UTMOS at WER 1.30, compared with -0.20 for per-channel and -0.08 for per-token scales.On OmniVoice, per-tensor A8 shifts the most sensitive component from the codec decoder under weight-only PTQ to the token head.Source: §4.5 · p. 3
• Across selected core-model settings, adding per-channel A8 changes W4 UTMOS by -0.10 to -0.10, -0.04 to -0.13, and -0.11 to -0.18 for Supertonic, Kokoro, and OmniVoice.WER stays within 0.01 of fp; on replication models, adding per-channel A8 to W4 costs at most 0.12 UTMOS for Orpheus.Source: §4.5 · p. 3
• Per-tensor A8 degrades four replication models by -3.10 to -1.36 UTMOS and the remaining four by 0.08 to 0.29.The severe cases are F5-TTS (-3.10), Kyutai (-2.99), CSM (-2.64), and Zonos (-1.36).Source: §4.5 · p. 3

4.6. Deployment measurements

• Supertonic’s Mac mini ONNX dynamic-int8 path raises WER from 0.028 to 1.11 and Korean CER from 0.052 to 1.27 at 2.09× fp32 latency.The path uses per-tensor activation scales; the supplied measurements contrast it with a quality-preserving x86 MatMul-only control.Source: §4.6 · p. 3
• Supertonic’s vocoder-excluded real 4-bit path loses -0.07 ∆UTMOS and runs at 0.60× fp32 latency with 41% lower peak RSS, while its int8 path is slower than fp32.With the vocoder included, the real 4-bit path degrades by -2.38 ∆UTMOS; the vocoder-excluded path has WER 0.033 against 0.028 at fp and unchanged Korean CER.Source: §4.6 · pp. 3–4, Table 2 · p. 4
• On the RTX PRO 6000, OmniVoice int4 at NFE 8 requires 2.1× compiled-fp16 latency and 2.2× energy, while Orpheus int4 runs at 0.50× bf16 latency and 0.45× energy.The Mac mini Table 2 reports system metrics for the quiet device over five repeats; the int4 row has no measured energy value.Source: §4.6 · p. 4, Table 2 · p. 4

4.7. Staged procedure and blind application

• Blind ablations found the flow-matching decoder as the sensitive component in both Chatterbox and VoxCPM, although both pre-run predictions were incorrect.Neither model class nor parameter share predicted the sensitive component.Source: §4.7 · p. 4
• On Chatterbox, the flow decoder alone degrades by -2.17 ∆UTMOS, while the T3 backbone alone yields -0.01 at W4.Group:128 scaling reduces flow-decoder loss to -0.55; keeping S3Gen at W8 and T3 at W4 per-channel yields -0.02 at fp-level WER.Source: §4.7 · p. 4
• On VoxCPM, the local DiT alone degrades by -2.73 ∆UTMOS and the VAE decoder by -1.25; group:128 reduces their losses to -1.26 and -0.25.Per-layer GPTQ improves the DiT to -0.41 but does not reach the quality-preserving band; keeping both components at W8 with the remainder at W4 group:128 yields -0.21 at fp-level WER.Source: §4.7 · p. 4
• The quality-preserving criterion requires the paired 95% ∆UTMOS interval to lie within [−0.05, 0.05] and the upper paired ∆WER interval endpoint to be at most 0.01, unrounded, against each configuration’s fp baseline.Source: Table 3 · p. 4

• Nine configurations meet both conditions, and three are 4-bit: Kokoro W4 group:128, the Chatterbox configuration, and the real int4 path.Source: §4.8 · p. 4
• The other Table 3 rows miss the UTMOS condition by 0.01 to 0.06 despite fp-level WER.Source: §4.8 · p. 4
• The Orpheus int4 path misses the WER condition by 0.002.Source: §4.8 · p. 4

• Both blind models share codec-token or latent-patch designs, and the blind tests assess component discovery rather than the procedure's ordering.Source: §5 · p. 4
• UTMOS is trained on English; Korean evidence in the text is CER, while reported UTMOS improvements concern predicted naturalness only.Source: §5 · p. 4
• Bootstrap intervals cover sentence sampling but not selection of the best strength or granularity on test sentences.Source: §5 · p. 4
• Across-seed standard deviations of paired ∆UTMOS stay below 0.06 across 22 conditions and ten models.Source: §5 · p. 4

• Across three core, eight replication, and two held-out models, the sensitive component is model-specific and not reliably predicted from model class alone.Matched-fp staged ablation identified it in two blind tests; higher precision or per-layer GPTQ can restore it to within 0.1 UTMOS of fp.Source: §6 · p. 4
• The waveform decoder or codec is the first component to test, while per-tensor scaling can severely degrade quality even at 8 bits.Source: §6 · p. 4
• Informal listening agreed with UTMOS on severe degradation (condition MOS at most 2.0) but not consistently on moderate differences.Listeners rated VoxCPM mixed precision above its DiT W4 group:128 GPTQ variant despite lower UTMOS, so the test validates neither moderate differences nor configuration rankings.Source: §6 · p. 4