Qwen3-TTS Technical Report

1 Introduction

• 3-second voice cloning, curated preset voices, and natural-language voice creation and fine-grained control support reusable voices and speaker-consistent generation across more than 10 languages.The 1.7B model is reported to deliver state-of-the-art, human-like perceptual quality without overfitting to ASR-related metrics.Source: §1 · p. 2
• 97 ms (0.6B) and 101 ms (1.7B) first-packet latencies accompany support for streaming text input and streaming audio output.The dual-track autoregressive architecture incorporates Multi-Token Prediction to model multi-codebook sequences, enabling speech decoding from the first codec frame.Source: §1 · p. 2
• 25 Hz single-codebook tokenization combines semantic and acoustic cues through the Qwen2-Audio encoder, balancing expressivity with tractable LLM modeling.Semantic tokenizers lack expressive power, whereas purely acoustic tokenizers complicate LLM modeling and accumulate long-horizon errors. Block-wise flow matching enables streaming (Xu et al., 2025), but the single-codebook design limits ultra-low-latency applications and general speech synthesis.Source: §1 · p. 2
• 12.5 Hz multi-codebook tokenization enables waveform reconstruction with a lightweight causal ConvNet, eliminating speaker-vector extraction and complex diffusion models.The scheme is inspired by Zhang et al. (2023b): its first codebook layer encodes semantic content, while subsequent layers capture acoustic details.Source: §1 · p. 2
• Qwen3-TTS achieves the lowest Seed-TTS WER and superior speaker similarity across all 10 evaluated languages compared with commercial baselines including MiniMax and ElevenLabs.Cross-lingual improvements include substantial error reductions for Chinese-to-Korean, while target-speaker manipulation outperforms GPT-4o-mini-tts. Long-form generation produces over 10 minutes of natural, fluent speech.Source: §1 · p. 2
• Over 5 million hours of speech data underpin Qwen3-TTS training, and the complete model and tokenizer families are released for community research and development.Source: §1 · p. 2

2.1 Qwen-TTS-Tokenizer-25Hz

• 25 Hz single-codebook tokenization builds on Qwen2-Audio through ASR pretraining followed by mel-spectrogram reconstruction training to incorporate acoustic information.Stage 1 adds a resampling layer and an intermediate vector quantization layer during continued ASR pretraining. Stage 2 fine-tunes the entire model with a convolution-based mel-spectrogram decoder.Source: §2.1 · p. 3
• Streaming reconstruction uses a Flow Matching-trained DiT (Lipman et al.) to generate mel-spectrogram chunks and a modified BigVGAN (Lee et al.) to synthesize waveforms.Fixed-length code blocks use an attention mask (Guo et al., 2025) with a three-block lookback and one-block lookahead to preserve context. BigVGAN also operates in chunks with a fixed receptive field.Source: §2.1 · p. 3

2.2 Qwen-TTS-Tokenizer-12Hz

• 12.5 Hz multi-codebook tokenization separates semantic content from acoustic detail using the semantic–acoustic disentangled quantization strategy of Mimi (Défossez et al., 2024).WavLM (Chen et al., 2022) supervises the first semantic codebook layer, while a 15-layer residual vector quantizer captures remaining acoustic details. Joint training uses a raw-waveform GAN and multi-scale mel-spectrogram reconstruction loss to improve naturalness, fidelity, and time–frequency consistency.Source: §2.2 · p. 3
• 12.5 Hz token emission requires no lookahead in the fully causal encoder, while the causal decoder reconstructs audio incrementally for low-latency streaming.The encoder emits both semantic and acoustic tokens sequentially, supporting real-time online services.Source: §2.2 · pp. 3–4

3.1 Architectures

• Qwen3-TTS concatenates textual and acoustic tokens along the channel axis, predicting corresponding acoustic tokens immediately upon receiving each textual token.The Qwen3 backbone uses the standard Qwen tokenizer for text and Qwen-TTS-Tokenizer for speech. A jointly trained speaker encoder controls identity, while Code2Wav converts predicted acoustic tokens into waveforms.Source: §3.1 · p. 4
• Qwen3-TTS-25Hz predicts single-level speech tokens, whereas Qwen3-TTS-12Hz predicts RVQ codebooks hierarchically for single-frame generation.The 25Hz backbone combines text features with preceding speech tokens, predicts through a linear head, and reconstructs waveforms with chunk-wise DiT. The 12Hz backbone predicts the zeroth codebook from aggregated codebook features, then MTP generates residual codebooks to improve vocal consistency and expressivity while minimizing latency.Source: §3.1 · p. 4

3.2 Training

• Over 5 million hours of multilingual speech establish general text-to-speech capabilities before high-quality continual pre-training reduces noise-induced hallucinations and improves generated speech.All training data uses ChatML to standardize inputs and support controllable generation. The general stage learns a monotonic multilingual text-to-speech mapping, followed by a dedicated quality-stratification pipeline and continual pre-training on high-quality data.Source: §3.2 · pp. 4–5
• 32,768 tokens, increased from 8,192, and upsampled long speech improve handling of extended, complex inputs and generation of contextually appropriate speech.These changes constitute the final, long-context pre-training stage.Source: §3.2 · p. 5
• Post-training combines human-preference alignment, rule-reward optimization, and lightweight speaker fine-tuning to improve speech quality, task stability, and voice-specific control.DPO (Rafailov et al., 2023) uses multilingual preference pairs constructed from human feedback, followed by GSPO with rule-based rewards. Speaker fine-tuning adapts the base model to specific voices while improving naturalness, expressiveness, and controllability.Source: §3.2 · p. 5

3.3 Features

• Voice cloning uses speaker embeddings for real-time synthesis or text–speech in-context learning for better prosody, while instructions enable voice design and style control.User-provided control instructions are prepended to input sequences, including for streaming voice cloning. Voice design builds on Qwen3 text comprehension, with a probabilistically activated thinking pattern during training improving instruction following for complex descriptions.Source: §3.3 · p. 5

3.4 Efficiency

• 16 tokens must be generated before Tokenizer-25Hz can synthesize its first 8-token mel chunk because chunk-wise DiT requires look-ahead.At 25 Hz, that chunk represents 320 ms; BigVGAN adds 130 ms of right-context look-ahead, leaving about 190 ms in the first packet, while subsequent 8-token groups produce 320 ms packets. Reported end-to-end First-Packet Latency sums LM time-to-first packet tokens and tokenizer per-packet decoding time; steady-state costs are measured per speech packet.Source: §3.4 · pp. 5–6
• 4 tokens form each 320 ms speech packet in Tokenizer-12Hz, whose left-context decoder avoids future-context waiting and supports lower first-packet latency.At 12.5 Hz, each token represents 80 ms and could be decoded individually; four-token packets avoid excessive scheduling overhead while the lightweight, batch-friendly decoder maintains low RTF under higher concurrency. Efficiency measurements use an internal vLLM engine with the vLLM V0 backend on a single computational resource, applying torch.compile and CUDA Graph acceleration to tokenizer decoding.Source: §3.4 · pp. 5–6

• Qwen3-TTS undergoes a comprehensive evaluation organized into two main categories.Source: §4 · p. 6
• Speech tokenizer evaluation constitutes one main category of the experiments.Source: §4 · p. 6
• Speech generation evaluation constitutes the other main category of the experiments.Source: §4 · p. 6

4.1.1 Qwen-TTS-Tokenizer-25Hz

• S2 acoustic fine-tuning slightly degrades ASR performance by incorporating acoustic details that reduce semantic discriminability but benefit speech generation.This deliberate trade-off favors acoustic richness for high-quality TTS and waveform reconstruction.Source: §4.1.1 · p. 6
• Qwen-TTS-Tokenizer-25Hz at S1 matches or outperforms the S3 Tokenizer series (Du et al., 2024a;b) on English and Chinese ASR benchmarks.Trained with ASR supervision, S1 achieves the lowest or near-lowest WER across multiple CommonVoice and Fleurs datasets.Source: Table 3 · p. 6, §4.1.1 · p. 6

4.1.2 Qwen-TTS-Tokenizer-12Hz

• Qwen-TTS-Tokenizer-12Hz achieves state-of-the-art speech reconstruction across all key metrics while maintaining high encoding efficiency.Evaluation used 2,620 LibriSpeech test-clean utterances, measuring STOI, PESQ, UTMOS, and WavLM-based speaker similarity (SIM). Reported configuration parameters include quantizer count, codebook size, and frames per second.Source: §4.1.2 · pp. 6–7, Table 4 · p. 7

4.2.1 Evaluation of Zero-Shot Speech Generation

• 1.24 WER on Seed-TTS test-en places post-trained Qwen3-TTS-12Hz-1.7B ahead of CosyVoice 3 and Seed-TTS.The 12Hz variants consistently outperform 25Hz variants in WER, and scaling from 0.6B to 1.7B consistently improves results. The authors attribute multilingual robustness to diverse pretraining data and suggest coarser temporal resolution helps model long-term dependencies.Source: §4.2.1 · p. 7, Table 5 · p. 8

4.2.2 Evaluation of Multilingual Speech Generation

• 6 of 10 languages show lowest WER for Qwen3-TTS, while all 10 show highest speaker similarity against MiniMax-Speech and ElevenLabs Multilingual v2.WER wins cover Chinese, English, Italian, French, Korean, and Russian; German, Portuguese, Spanish, and Japanese remain competitive with the baselines.Source: §4.2.2 · p. 8, Table 6 · p. 9

4.2.3 Evaluation of Cross-Lingual Speech Generation

• Approximately 66% lower zh-to-ko CER (4.82 versus 14.4) distinguishes Qwen3-TTS from CosyVoice3 in cross-lingual generation.Qwen3-TTS establishes state-of-the-art results for English and Korean targets and outperforms baselines on zh-to-en and en-to-zh. It maintains low error rates across evaluated directions, whereas CosyVoice2 is unstable on several pairs.Source: Table 7 · p. 9, §4.2.3 · p. 9

4.2.4 Evaluation of Controllable Speech Generation

• +28% Chinese APS improvement over GPT-4o-mini-tts highlights target-speaker editing gains, while Qwen3-TTS-12Hz-1.7B-VD achieves open-source-leading voice design on InstructTTSEval.ChatML frames voice control as language modeling, supporting novel voices from descriptions and reference-speaker attribute editing. Voice design surpasses Hume and VoiceSculptor in Description-Speech Consistency (DSD) and Response Precision (RP); editing beats GPT-4o-mini-tts across all metrics, although Gemini remains an upper bound.Source: Table 8 · p. 10, §4.2.4 · p. 10

4.2.5 Evaluation of Target-Speaker Speech Generation

• 7 of 10 languages have lower WER for speaker-fine-tuned Qwen3-TTS (Aiden Voice) than GPT-4o-Audio-Preview (Ballad Voice), despite monolingual fine-tuning.The adapted voice transfers timbre and prosody to all 10 languages without degraded stability. Qwen3-TTS versus GPT-4o-Audio-Preview WER is 3.88 versus 5.00 in Japanese and 1.74 versus 2.76 in Korean; the baseline retains a slight edge in Italian, Portuguese, and French.Source: §4.2.5 · p. 10, Table 9 · p. 11

4.2.6 Evaluation of Long Speech Generation

• 1.533 long-zh WER and 1.571 long-en WER give Qwen3-TTS-25Hz-1.7B the best long-form results against Higgs-Audio-v2, VibeVoice, and VoxCPM.The internal evaluation targets speech exceeding 10 minutes using 100 Chinese/English texts of 200–2000 words, Qwen3-ASR transcription, and speaker-fine-tuned Aiden Voice. Unlike Higgs-Audio-v2’s boundary artifacts and VibeVoice’s Chinese WER > 22, Qwen3-TTS maintains continuous prosody; 25Hz outperforms 12Hz, suggesting semantic tokens improve extended-sequence stability.Source: §4.2 · p. 7, Table 10 · p. 11, §4.2.6 · p. 11

5 Conclusion

• Qwen3-TTS combines a dual-track design with semantic-rich Qwen-TTS-Tokenizer-25Hz and low-latency Qwen-TTS-Tokenizer-12Hz to synthesize high-fidelity speech efficiently in streaming settings.Source: §5 · p. 11
• Qwen3-TTS achieves state-of-the-art zero-shot voice cloning and cross-lingual synthesis, outperforming existing baselines in challenging Chinese-to-Korean generation.A probabilistically activated thinking pattern also enables state-of-the-art voice design.Source: §5 · p. 11
• Over 10 minutes of fluent speech can be generated using a dedicated training strategy that resolves autoregressive stability issues without typical chunk-based artifacts.Source: §5 · p. 11
• A single autoregressive framework unifies zero-shot cloning, cross-lingual transfer, and fine-grained instruction control.Future work targets versatile audio generation, multilingual coverage beyond the current 10 languages, and more granular stylistic controls.Source: §5 · p. 11
• Open-sourcing the models and tokenizers aims to accelerate community research and support more natural, expressive, and accessible human-computer interfaces.Source: §5 · pp. 11–12