MOSS-TTSD: Text to Spoken Dialogue Generation

1 Introduction

• Spoken dialogue requires natural turn-taking, robust speaker switching, and long-range coherence, while short-utterance TTS leaves gaps for real-world conversations [1] [2] [3] [4] [5].The audio must also preserve speaker-specific timbre and prosody across turns and avoid stitching artifacts in long-form output.Source: §1 · p. 1
• MOSS-TTSD uses Qwen3-8B-base [6] and MOSS-Audio-Tokenizer [7] in a fully discrete generation paradigm with a multi-head delay pattern [8].It predicts RVQ codebook tokens and models the first 16 RVQ layers for robust long-context generation at a low bitrate.Source: §1 · pp. 1–2
• MOSS-TTSD conditions on dialogue scripts with speaker tags and optional per-speaker references, supporting up to 60 minutes of single-pass generation and up to five speakers.The paper also describes zero-shot voice cloning, multiple languages, and scenarios including podcasts, commentary, audiobooks, dubbing, and crosstalk.Source: §1 · p. 2
• TTSD-eval uses forced alignment to measure speaker attribution, speaker similarity, and intelligibility in long-form, multi-speaker scenarios without diarization tools.The paper reports that objective evaluations with TTSD-eval and human assessments show MOSS-TTSD outperforming strong open-source and proprietary baselines.Source: §1 · p. 2

MOSS-TTSD History

• Across releases, the system progressively improved long-form stability, speaker consistency, and feature coverage.Source: §2 · p. 2

MOSS-TTSD v0

• MOSS-TTSD v0 began with Chinese and English support and up to 960-second single-pass generation, using ASR and diarization outputs to construct explicit speaker annotations.The 30-second per-request ASR constraint led to combining coarse speaker segments with token-level speaker labels.Source: §2 · p. 2

MOSS-TTSD v0.5

• MOSS-TTSD v0.5 fine-tuned on a high-quality data subset to improve timbre switching and zero-shot voice cloning robustness, and added a 32 kHz XY-Tokenizer variant.Source: §2 · p. 2

MOSS-TTSD v0.7

• MOSS-TTSD v0.7 adopted end-to-end ASR transcripts with speaker tags and conditioned decoding on both generated and reference tokens, improving voice cloning and audio quality.The maximum single-pass generation length increased from 960 to 1700 seconds.Source: §2 · p. 2

MOSS-TTSD v1.0

• MOSS-TTSD v1.0 supports up to 60-minute single-session context, up to five speakers, and additional mainstream languages beyond Chinese and English.The release benefits from scaled data, a stronger LLM backbone, and continued improvements to the audio tokenizer and data pipeline.Source: §2 · p. 2

3.1 Data Pipeline

• Raw audio is normalized and diarized, then merged into single-, two-, or 3–5-speaker clips capped at 3600 seconds and annotated with quality, language, and sample-rate metadata.Noise-heavy domains may receive additional denoising before clips are transcribed, filtered for quality and language consistency, and retained at DNSMOS ≥2.8.Source: §3.1 · p. 3, Figure 1 · p. 3

3.2 Data Synthesis and Augmentation

• Voice-cloning data maps non-overlapping single-speaker segments from two- and multi-speaker recordings to the corresponding speaker reference slots.For synthetic multi-speaker data, diarization-attributed single-speaker segments are interleaved; selected clips require DNSMOS ≥3.4 and a shared sample rate.Source: §3.2 · pp. 3–4
• Because long-context speaker identity is difficult to annotate in 3–5-speaker scenarios, the dataset retains only a small fraction of high-quality real multi-speaker clips.Source: §3.2 · pp. 3–4
• Rule-based text augmentation increases punctuation diversity in selected training transcripts to address sparse symbols in speech data and ASR punctuation biases.Source: §3.2 · p. 4

4.1 Architecture

• MOSS-TTSD uses Qwen3-8B-base [6] and MOSS-Audio-Tokenizer [7] with autoregressive multi-head delay generation of discrete speech tokens.It models the first 16 RVQ layers at 2 kbps and a 12.5 Hz frame rate, supporting training contexts up to 3600 seconds and five speakers.Source: §4.1 · p. 4

4.2 Curriculum Learning

• A three-stage curriculum moves from single-speaker synthesis toward long-context, multi-speaker dialogue generation.Stage 1 extends sequence length and teaches speaker-tag and reference-audio control; Stage 2 emphasizes high-fidelity data; Stage 3 adds real and synthetic multi-speaker data.Source: §4.2 · p. 5

4.3 Inference

• The default voice_clone_and_- continuation inference setting combines explicitly constructed reference-conditioned cloning with continuation-based cloning.The paper reports that combining the two paradigms substantially improves voice-cloning performance.Source: §4.3 · p. 5

5.1 TTSD-eval

• TTSD-eval uses forced alignment to segment generated audio and assigns speaker identities from explicit script tags, avoiding reliance on diarization tools.It uses word-level script-to-audio alignment, then compares speaker-embedding predictions with script labels for ACC and compares each fragment with its ground-truth speaker prompt for SIM.Source: §5.1 · p. 6
• The Chinese and English test sets each contain 50 dialogues, with audio ranging from approximately 30 to 720 seconds across short and long-form scenarios.Dialogue texts are generated using Gemini 2.5 Pro; the sets combine manually collected and Seed-TTS-eval reference pairs.Source: §5.1 · pp. 6–7
• For open-source comparisons, Higgs Audio V2 and FireRedTTS2 generate chunk by chunk with a buffer of six recent chunks; proprietary models use private voice-library speakers for fair voice-cloning comparisons.Eleven V3 is split into 10-turn chunks because of its 5,000-character request limit, while the other proprietary models generate in a single pass.Source: §5.1 · p. 7
• MOSS-TTSD outperforms leading open-source and proprietary models in speaker consistency and intelligibility, as summarized in Table 1.The table reports ACC, SIM, and WER; lower WER is better.Source: Table 1 · p. 6, §5.1 · p. 7

5.2 Subjective Evaluation

• Subjective evaluations find that MOSS-TTSD surpasses leading open-source and proprietary models in human-perceived quality, with strong agreement between perceived speaker consistency and TTSD-eval scores.Open-source comparisons rate ACC, SIM, rhythm, and overall quality; proprietary comparisons ask annotators to choose the preferred sample.Source: Figure 4 · p. 7, §5.2 · p. 8, Figure 5 · p. 8

6 Conclusion

• MOSS-TTSD is a spoken dialogue synthesis model for expressive, long-form multi-party conversations.It uses a fully discrete paradigm with Qwen3-8B-base and MOSS-Audio-Tokenizer, a multi-head delay pattern, and the first 16 RVQ layers for robust long-context generation.Source: §6 · pp. 8–9
• With explicit speaker-tagged scripts and optional per-speaker references, MOSS-TTSD supports natural turn-taking, zero-shot multi-speaker cloning, up to five speakers, and 60 minutes of single-pass synthesis.The conclusion also reports multiple mainstream languages and long-form scenarios, including podcasts, commentary, dubbing, and entertainment.Source: §6 · p. 9
• Objective and subjective evaluations show that MOSS-TTSD outperforms strong open-source and proprietary baselines for spoken dialogue generation.TTSD-eval measures ACC, SIM, and intelligibility via WER without diarization tools.Source: §6 · p. 9

A Chat Templates of Training Data

• Both voice-cloning and common-TTS training templates use explicit speaker tags to keep conditioning consistent across speakers and turns.The appendix presents the two template types in Table 2.Source: A Chat Templates of Training Data · p. 13, Table 2 · p. 13
• The voice-cloning template includes per-speaker reference audio to guide timbre.The common-TTS template omits the reference-audio field.Source: A Chat Templates of Training Data · p. 13
• The common-TTS template retains speaker tags but does not include per-speaker reference audio.This distinguishes it from the voice-cloning template.Source: A Chat Templates of Training Data · p. 13

B Text Augmentation Rules

• Rule-based punctuation augmentation targets transcript symbols that are scarce in real-world speech data and biases in ASR punctuation output.The stated aim is improved robustness to diverse text inputs.Source: B Text Augmentation Rules · p. 13
• Punctuation augmentation is applied to a selected subset of training transcripts rather than to every sample.The specific substitution rules are listed in Table 3.Source: B Text Augmentation Rules · p. 13, Table 3 · p. 13
• Each augmentation rule replaces an original token, with a specified probability, by a randomly selected token from its replacement set.Table 3 summarizes the rule-based substitutions.Source: B Text Augmentation Rules · p. 13, Table 3 · p. 13

C Voice Clone and Continuation

• The three configurations are reference-conditioned voice_clone, pure autoregressive continuation, and voice_clone_and_continuation, which combines both paradigms.The combined configuration is the default setting.Source: C Voice Clone and Continuation · p. 13
• The combined voice_clone_and_continuation setting achieves the best speaker similarity and comparable results on the other reported metrics.This finding is reported across TTSD-eval and Seed-TTS-eval results.Source: C Voice Clone and Continuation · p. 13, Table 4 · p. 14, Table 5 · p. 14
• Table 4 reports Seed-TTS-eval results, while Table 5 reports TTSD-eval results for the cloning configurations.The tables report SIM and, for TTSD-eval, ACC and WER as well.Source: Table 4 · p. 14, Table 5 · p. 14

• The official Doubao voice library does not provide speaker prompts for Doubao_Podcast.Source: D Voice References used for Evaluation · p. 14
• The evaluation uses two corresponding speaker prompts generated through the official Doubao API as references.Source: D Voice References used for Evaluation · p. 14
• Table 6 summarizes the proprietary voice references used in the evaluations.Source: Table 6 · p. 14