Source-linked AI summary
STREAM: A Data-Centric Framework for Mining High-Value Task-Oriented Dialogues from Streaming Media
Liang Xue, Haoyu Liu, Cheng Wang, Pengyu Chen, Haozhuo Zheng, Yang Liu
TL;DR
Vertical-domain dialogue data is scarce because expert annotation is costly, service conversations are restricted, and static corpora become stale. Stream mines public streaming media to synthesize structured, knowledge-aware dialogues with personas and Conversational Blueprints. StreamDial improves intrinsic quality and downstream Dialogue State Tracking over strong baselines, with encouraging multilingual transfer under controlled training budgets.
Problem
Vertical-domain task-oriented dialogue datasets remain limited despite growing demand for complex interactions involving requirement mining, conflicts, negotiation, and actionable guidance.
Method
Stream mines interaction signals from streaming media and synthesizes dialogues using role-grounded personas, Conversational Blueprints, and RAG-supported generation.
Results
StreamDial improves intrinsic dialogue quality and downstream Dialogue State Tracking over strong baselines, with encouraging multilingual transfer on Qwen3-8B under controlled training budgets.
Takeaways & Limitations
Streaming media is a practical and scalable source for constructing strategy-rich task-oriented dialogue corpora.
Takeaways & Limitations
StreamDial currently covers only Automotive, Restaurant, and Hotel, so scalability to additional verticals remains to be validated.
Abstract
from arXiv · showhide
Large language models for vertical domains are bottlenecked by the scarcity of complex, domain-specific task-oriented dialogues. Existing data acquisition pipelines face a persistent trilemma: expert annotation is expensive, real-world service conversations are constrained by privacy and commercial restrictions, and static corpora quickly become temporally stale. We propose Stream, a data-centric framework that leverages publicly available streaming media (live streams and short videos) to synthesize high-value service dialogues at scale. Stream mines authentic interaction signals from noisy streams and synthesizes conversations by integrating role-grounded persona construction with Conversational Blueprint construction; it further adopts retrieval-augmented generation (RAG) to support knowledge-aware responses. Based on Stream, we release StreamDial, a large-scale multi-domain dataset covering Automotive, Restaurant, and Hotel. StreamDial contains 87,498 dialogue sessions and 1,497,320 turns in total, with an average of 17.11 turns per session and a comparable scale across domains. Each session is organized as a structured quadruplet $\langle P_u, P_a, B, H \rangle$ that pairs dialogue history with explicit user/agent personas and a Conversational Blueprint, capturing realistic service behaviors such as requirement mining, constraint conflicts, negotiation, and recovery. Evaluations with automatic judges and downstream tasks show that StreamDial improves intrinsic dialogue quality over strong baselines, and models trained with StreamDial improve Dialogue State Tracking across backbones; we further report a completed human-evaluation set and encouraging multilingual transfer on Qwen3-8B under a controlled training budget. The data is released in https://github.com/hitxueliang/DialogDataSetBySTREAM.
1 Introduction
Stream addresses scarce, costly, privacy-constrained, and temporally stale vertical task-oriented dialogue data by mining streaming media and synthesizing structured, high-value conversations. StreamDial spans three service domains and supports evaluation of dialogue quality and downstream utility.
- Streaming media provides timely, authentic service interactions that expose diverse intents, service strategies, decision patterns, and domain knowledge.
- Stream combines interaction-signal mining, role-grounded personas, Conversational Blueprints, and RAG-supported generation to synthesize complex task-oriented dialogues.
- StreamDial covers Automotive, Restaurant, and Hotel and organizes sessions with explicit personas, dialogue history, and a Conversational Blueprint.
- StreamDial captures requirement mining, constraint conflicts, negotiation, and recovery rather than only canonical slot-filling behavior.
- Automatic assessment, completed human evaluation, downstream Dialogue State Tracking, and multilingual transfer provide complementary evidence of dataset utility.
2 Related Work
Prior task-oriented dialogue datasets are constrained by annotation cost, shallow crowdsourced strategies, temporal rigidity, and limited use of rich media for semantic reconstruction. StreamDial targets these gaps through streaming-media mining for strategy-rich and temporally fresh dialogue data.
- Manual annotation limits dataset scale, while crowdsourced workers may lack domain expertise and produce strategically shallow interactions.
- Static benchmarks struggle to represent evolving domain information, including changing vehicle models and legal regulations.
- Persona-driven simulations can exhibit interaction asymmetry and decoupling between strategy and action because personas may rely on templates or model knowledge.
- Earlier rich-media pipelines emphasize modality alignment and cleaning, whereas Stream mines expert strategies and user-intent shifts as Atomic Interaction Signals.
3 The Stream Framework
The Stream Framework transforms heterogeneous rich media into structured dialogue data through four cascaded phases: signal ingestion, persona synthesis, blueprinting, and interactive generation. These stages constrain roles, trajectories, knowledge use, and dataset redundancy.
- Framework formulation: Stream defines each synthesized sample as a quadruplet containing user persona, agent persona, Conversational Blueprint, and dialogue history.
- Framework stages: The framework cascades Streaming Signal Ingestion, Adaptive Persona Synthesis, Conversational Blueprinting, and Interactive Dialogue Generation.
- Phase 1: Streaming Signal Ingestion: SSI processes web pages, live streams, and short videos to extract denoised, aligned interaction signals, including questions, responses, QA pairs, strategies, and account metadata.
- Phase 1: Streaming Signal Ingestion: SSI applies ASR, domain lexicons, and quality control because streaming sources have uneven quality and domain-specific transcription errors.
- Phase 2: Adaptive Persona Synthesis: APS constructs paired high-fidelity user and agent personas that provide role constraints and behavioral priors for later synthesis.
- Phase 2: Adaptive Persona Synthesis: User Persona Modeling integrates user questions with seed dialogues, while Agent Persona Modeling incorporates account metadata, interaction patterns, service boundaries, and domain knowledge.
- Phase 3: Conversational Blueprinting: Conversational Blueprinting combines strategy signals, agent personas, and seed dialogues into an executable specification covering rhythm, key nodes, coping strategies, and dialogue paths.
- Phase 3: Conversational Blueprinting: Blueprint guidance reduces random-walk behavior in long dialogues and improves strategic consistency across turns.
4 The StreamDial Dataset
StreamDial is a large-scale, multi-domain dataset synthesized from public streaming media for complex service scenarios. Its structured sessions and observed interaction patterns capture role-specific, trajectory-level behaviors beyond conventional slot filling.
- Dataset scope: StreamDial covers Automotive, Restaurant, and Hotel service scenarios synthesized from publicly available streaming media.The dataset targets requirement mining, constraint handling, and proactive guidance toward actionable outcomes.
- Dataset scale: 87,498 dialogue sessions and 1,497,320 turns comprise StreamDial, averaging 17.11 turns per session across comparably sized domains.These statistics are reported at the session and turn levels.
- Session schema: Each session uses the quadruplet ⟨P_u, P_a, B, H⟩ to align user and agent personas, a dialogue plan, and multi-turn history.The schema supports supervision and analysis at both role and trajectory levels; persona and plan fields are explicitly textual.
- Observed interactions: StreamDial more frequently includes inventory checks, production-date inquiries, reservations, holding requests, negotiation, and follow-up arrangements than conventional public TOD corpora.Users may revise requirements mid-conversation, causing transitions between candidate solutions.
- Case study: An Automotive case progresses from model-year comparison through inventory and production-date confirmation to resolving a fuel-consumption–drivetrain conflict and scheduling a test drive.The main text presents an English translation, while the original Chinese dialogue appears in Appendix A.
5 Experiments
The experiments evaluate StreamDial through intrinsic quality, human validation, downstream Dialogue State Tracking, and multilingual transfer. Across these evaluations, StreamDial shows consistent quality and utility gains, including cross-domain and cross-language improvements under controlled settings.
- Intrinsic Evaluation: Across three judges, StreamDial leads especially on Informativeness, Diversity, and Flexibility while retaining strong Coherence and Naturalness.The relative ordering between Open Data and StreamDial remains consistent across Qwen3-Max, GPT-5.2, and Gemini3-Pro.
- Intrinsic Evaluation: StreamDial improves Overall Quality across Automotive, Restaurant, and Hotel under all judges.The cross-domain comparison tests whether intrinsic gains depend on a single service scenario.
- Human Validation: 1,431 completed sessions annotated by 53 annotators confirm that StreamDial is more informative and flexible while maintaining comparable coherence and naturalness.The human evaluation uses the same six dimensions as the automatic protocol and serves as an independent validation channel.
- Dialogue State Tracking: All three mixed-data settings improve DST over public-data training across backbones, with StreamDial Hybrid producing the strongest overall results.The comparison uses a controlled 2k budget, isolating synthesized supervision from changes in training scale.
- Dialogue State Tracking: Slot-distribution analysis shows that different synthesis sources provide complementary supervision signals for DST.The study compares slot coverage balance and slot-value diversity across data sources.
- Multilingual Transfer: StreamDial-enhanced Qwen3-8B training improves JGA and Slot-value F1 in English, French, and Korean, with larger gains in English and Korean.French shows smaller but still positive gains, and the multilingual results are reported in percentage points.
6 Limitations
The paper identifies limitations in source quality, domain coverage, evaluation scope, multilingual transfer, and responsible data-use documentation.
- Source quality: Streaming-media quality varies across platforms and creators, so residual ASR, colloquial-ambiguity, and incomplete-context errors may propagate into synthesis.The framework mitigates these risks with filtering, lexicons, ASR correction, retrieval-based checking, and consistency validation.
- Domain coverage: StreamDial currently covers only Automotive, Restaurant, and Hotel, leaving scalability to additional verticals insufficiently validated.The paper specifically identifies healthcare consultation, education advising, and legal assistance as future validation domains.
- Evaluation scope: The evaluation emphasizes intrinsic quality, human validation, and downstream DST utility but does not fully capture factual robustness, user trust, or long-horizon interactive task completion.The completed human-evaluation set contains 1,431 sessions.
- Multilingual transfer: Multilingual transfer quality can vary across languages because linguistic realization and slot-expression patterns differ.The authors call for language-aware synthesis and calibration to reduce transfer variance.
- Responsible data use: Responsible data use remains important, requiring stronger documentation of governance, filtering criteria, and ethical safeguards in future releases.
7 Conclusion
The paper presents Stream for synthesizing structured task-oriented dialogue supervision from public streaming media and releases StreamDial as its multi-domain dataset. StreamDial improves intrinsic dialogue quality and downstream DST performance, with encouraging multilingual transfer under controlled training budgets.
- Framework: Stream couples streaming-signal ingestion, persona synthesis, Conversational Blueprint construction, and RAG-enhanced generation to transform rich-media interactions into dialogue-learning supervision.
- Dataset: 87,498 sessions and 1,497,320 turns comprise StreamDial, with each session represented as the structured quadruplet ⟨P_u, P_a, B, H⟩.
- Findings: StreamDial improves intrinsic dialogue quality and downstream DST performance over strong baselines under controlled training budgets, with encouraging multilingual transfer on Qwen3-8B.
- Implication: The findings suggest that streaming media is a practical and scalable source for constructing strategy-rich task-oriented dialogue corpora.
dialogue_id: car_000 domain: car
The appendix presents a Chinese Automotive case study in dataset-native JSON and points to the exact intrinsic scores and full DST result tables used for evaluation reporting.
- Case study: The case study is an Automotive example rendered from the original JSON entry and corresponds to the case shown in Figure 2.
- Data format: The appendix preserves the dataset-native JSON structure, including dialogue IDs, linked persona and plan IDs, and turn-level records.The main paper instead shows an English translation for readability.
- Intrinsic evaluation: Table 8 lists per-judge intrinsic quality scores from 1–10 for Open Data (Baseline) and StreamDial (Hybrid) across six evaluation dimensions.
- DST evaluation: Table 9 reports full cross-domain DST performance on the RiSAWOZ public test set.
C Additional Cross-domain DST Results
The appendix provides complete cross-domain DST results on Chinese RiSAWOZ, complementing the representative backbone shown in the main text.
- Results scope: The appendix reports full cross-domain DST results on RiSAWOZ in Chinese.
- Backbone coverage: The complete results cover all reported backbones, whereas the main text presents one representative backbone for readability.
D Full Multilingual Results on X-RiSAWOZ
The appendix reports cross-lingual Dialogue State Tracking results for X-RiSAWOZ under two StreamDial multilingual synthesis strategies. Source to Target Language generally transfers best, particularly for English and Korean, while French transfer is weaker.
- Experimental setup: The appendix evaluates StreamDial multilingual synthesis across reported X-RiSAWOZ backbones and training configurations.The experiments compare two cross-lingual training strategies.
- Training strategies: Source to Target Language translates seed dialogues before target-language generation, whereas Source Language to Target Language generates source-language dialogues before translating them.These strategies differ in when translation occurs relative to synthetic dialogue generation.
- Results: Source to Target Language generally provides the best cross-lingual transfer, especially for English and Korean.The comparison concerns models trained with StreamDial (Hybrid).
- Results: French shows slightly less optimal transfer, possibly because French structural complexity affects slot extraction accuracy.The passage presents this as a possible explanation rather than a confirmed cause.
E Slot Distribution Analysis by Data Source
The slot-distribution analysis compares coverage and value diversity across five Automotive-domain training sources. Seed dialogues balance slot coverage while substantially increasing slot-value diversity, whereas stream data alone produces the highest coverage variance.
- Data sources: The analysis compares slot statistics across original training, the train_llm_slot baseline, seed dialogue, stream data, and hybrid data.The comparison focuses on the Automotive domain.
- Metrics: Coverage measures the percentage of dialogues containing each slot, while Values counts distinct observed values; the table also summarizes coverage variance and average Values.These metrics characterize distributional balance and semantic diversity across slot types.
- Key observations: 7.91 →7.38 coverage variance with seed dialogue indicates a more balanced slot distribution than the train_llm_slot baseline.The comparison is between train_llm_slot and our w/ seed dialogue.
- Key observations: 6.8 → 18.4 average slot-value diversity with seed dialogue indicates richer semantic supervision than the train_llm_slot baseline.The passage reports a substantial increase in average Values.
- Key observations: 9.67 coverage variance makes stream data the least balanced source across slot types, partly because several slots are under-represented.The passage attributes this pattern to lower frequency of those slots in streaming interactions.