Source-linked AI summary
DuplexDrama: A Synthesized Dialogue Dataset with Scenarios, Full-Duplex Behaviors, Expressive Speech, and Sound Events
Qingxiang Guo, Wenke Fan, Shuofeng Zhao, Dawei Yang, Zhiyang Zhou, Yingxin Shang, Hongwei Cai, Zhou Wang, Weixu Wang, Lin Yang, Shuran Zhou, Yang Song
TL;DR
Synthesized dialogue datasets lack the joint coverage of personas and scenarios, full-duplex behaviors, expressive speech, and sound events needed for this research area. DuplexDrama constructs such a dataset through a four-stage pipeline and quality validation. Among the compared corpora, it is the only one reported to cover all four dimensions jointly.
Problem
Existing synthesized dialogue corpora largely lack full-duplex behaviors, while natural corpora lack rich scenario, persona, or behavior annotations; no public dataset jointly covers all four dimensions.
Source: §1 · p. 1Method
Results
DuplexDrama is the only compared corpus jointly covering persona and scenario annotations, full-duplex behaviors, expressive speech, and script-aware sound events.
Source: §3.3 · p. 4Takeaways & Limitations
The planned open release comprises 6,400 bilingual dialogues totaling 800 hours, with audio assets and evaluation prompts to support reproducible full-duplex dialogue research.
Source: §1 · p. 2Background audio assets are processed strictly for non-commercial, academic research purposes.
Source: §2.4 · p. 2
Abstract
from arXiv · showhide
We present DuplexDrama, the first synthesized spoken dialogue dataset that simultaneously covers four dimensions: (i) complete persona and scenario settings; (ii) three full-duplex behaviors (interruption, backchannel, incomplete); (iii) expressive speech with persona-aligned emotion labels; and (iv) script-aware sound events. DuplexDrama is built via a 4-stage pipeline; quality validation on both scripts and synthesized audio confirms its quality. We have produced more than 2,000 hours audio data with a 64-voice timbre pool spanning 13 personas and 5 age buckets; 3.8% of all turns carry at least one full-duplex behavior. This data has been validated through internal full-duplex model training. We will release a curated subset of 6,400 bilingual dialogues (800 h, Chinese ~500 h + English ~300 h) to advance full-duplex spoken dialogue model research. Data samples are available at our demo page and LLM-judge evaluation prompts will be released with the dataset.
1. INTRODUCTION
DuplexDrama addresses the scarcity of synthesized full-duplex dialogue data by combining persona and scenario settings, duplex behaviors, expressive speech, and script-aware sound events. The paper introduces a four-stage construction pipeline, a quality and script-rationality evaluation protocol, and a planned open release.
1. INTRODUCTION
- Existing synthesized dialogue corpora are mostly clean read speech or simple turn-taking, while natural conversational corpora lack rich persona, scenario, or duplex-behavior annotations.Full-duplex benchmarks target evaluation and remain small; some persona-conditioned models do not release public training corpora.Source: §1 · p. 1
- Swapping the two roles in a dialogue creates two training samples, doubling the usable data.The sample figure shows persona and scenario metadata, a tagged script, and four aligned audio tracks with interruptions represented as acoustic overlap.Source: Figure 1 · p. 1
- DuplexDrama is the only corpus described as jointly covering persona and scenario annotations, full-duplex behaviors, expressive speech, and script-aware sound events.Existing synthesized corpora lack full-duplex scenarios, while natural conversational corpora lack several of these annotations.Source: §1 · p. 1
- The four-stage pipeline generates persona and scenario settings, tagged scripts, expressive speech, and script-aware sound events, then filters low-quality dialogues.Speech synthesis uses a 64-voice timbre pool spanning 13 personas and 5 age buckets.Source: §1 · pp. 1–2
- The evaluation protocol uses two LLM judges to cross-check script-rationality assessments, with prompts planned for release alongside the dataset.The stated aim is reproducible script-rationality assessment.Source: §1 · p. 2
- The planned open release contains 6,400 bilingual dialogues totaling 800 hours, plus an audio asset bank and LLM-judge evaluation prompts.The release is intended to support reproducible full-duplex spoken dialogue model research.Source: §1 · p. 2
2. PIPELINE
The pipeline builds dialogues from consistent persona–scenario settings, annotated scripts, synthesized speech, and background channels, then filters dialogues through quality validation. Its stages encode the three duplex behaviors in audio and align environmental sounds and events with the speech.
2. PIPELINE
- The pipeline progresses from persona and scenario generation through tagged scripts and dual-track speech to background mixing and quality filtering.The overview identifies four stages and a validator that discards failed dialogues before release.Source: §2 · p. 2, Figure 2 · p. 3
2.1. Persona and Scenario Generation
- Persona and scenario generation draws seeds from complementary pools and expands them into consistent setups coupling two persona tuples with a scenario specification.The persona pool covers 24 macro categories and 134 sub-types; the topic pool covers 46 macro categories and 404 sub-types.Source: §2.1 · p. 2
2.2. Dialogue Script Generation
- Dialogue scripts carry three tag families: full-duplex behaviors, emotion labels, and environmental-noise or sound-event tags.The behavior tags cover interruption, backchannel, and incomplete speech; emotion tags use seven classes aligned with IndexTTS2.Source: §2.2 · p. 2
2.3. Full-duplex Dialogue Synthesis
- IndexTTS2 synthesizes speech conditioned jointly on speaker prompt audio and emotion, using a 64-speaker pool spanning 13 personas and 5 age buckets.Forced alignment supplies word timestamps for assembling utterances into parallel speaker tracks; interruptions and backchannels are realized as overlapping audio, while incomplete tags produce in-sentence pauses.Source: §2.3 · p. 2
2.4. Background Channel Realization
- The background channel pairs each speech channel with environmental noise and sound events, placing events at forced-aligned trigger positions with configurable SNR.The noise bank contains 600 long clips across 7 scene categories; the separate event bank contains 2,203 short clips across 52 categories.Source: §2.4 · p. 2, Figure 3 · p. 3
- Audio assets are drawn from publicly accessible sources and processed strictly for non-commercial, academic research purposes.This constrains the stated use of the background-channel assets.Source: §2.4 · p. 2
2.5. Quality Validation
- Quality validation cross-checks script dimensions with two LLMs where specified and evaluates synthesized audio using WER, two MOS metrics, and speaker cosine similarity.Asset rationality is judged by one multimodal model; dialogues with SpkCons below 0.9 are discarded.Source: §2.5 · p. 3
3. DATASET ANALYSIS
The analysis reports corpus statistics, validates audio and scripts, and compares DuplexDrama’s annotation coverage with prior corpora.
3.1. Dataset Statistics
- Released-corpus statistics are summarized through an overview table, full-duplex tag counts and shares, and environment- and sound-event-tag distributions.Source: §3.1 · p. 3, Table 1 · p. 3, Table 2 · p. 3
3.2.1. Objective Audio Quality Metrics
- Mixing background costs 0.05 UTMOSv2 and 0.47 NISQA-MOS; WER and SpkCons use clean TTS audio, while both MOS metrics are scored on clean and mixed audio.The authors attribute the larger NISQA-MOS drop to its penalty for additive noise.Source: §3.2.1 · pp. 3–4, Table 3 · p. 4
- Lower-than-typical UTMOSv2 scores are attributed primarily to diverse persona stylings and emotion injection.Source: §3.2.1 · p. 4
3.2.2. Script Rationality Validation via Dual LLM Judges
- Script-rationality checks cover asset rationality, background-tag rationality, and script-scenario consistency; asset rationality and consistency are cross-checked by two LLMs.Background-tag rationality is judged by Gemini alone because the audio bank is manually curated.Source: §3.2.2 · p. 4
3.3. Comparison with Other Datasets
- DuplexDrama is the only compared corpus jointly covering all four annotation dimensions, and script-aware sound-event tags are unique to it.Source: §3.3 · p. 4, Table 5 · p. 4
4. CONCLUSION
DuplexDrama combines four annotation dimensions in a synthesized spoken-dialogue dataset, with audio and script quality assessed through objective metrics and dual-LLM checks. The authors plan an approximately 800-hour release and identify realism and label flexibility as future work.
4. CONCLUSION
- DuplexDrama jointly covers persona and scenario annotations, full-duplex behaviors, expressive speech, and sound events in a TTS-synthesized spoken-dialogue dataset.Source: §4 · p. 4
- The corpus is evaluated with four objective audio metrics and dual-LLM cross-checking of script rationality.Source: §4 · p. 4
- The authors plan to release approximately 800 hours and pursue more realistic duplex-label distributions, TTS paralinguistic phenomena, and flexible sound-event label matching.Source: §4 · p. 4