Source-linked AI summary
Building a Conversational Agent Overnight with Dialogue Self-Play
Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, Larry Heck
TL;DR
M2M targets the lack of affordable, high-quality task-specific dialogue data for new goal-oriented applications. It combines developer-supplied task specifications, automated bot self-play, and crowdsourced contextual rewriting to bootstrap dialogue agents. The framework is intended to improve coverage, language and flow diversity, and label correctness, while its simulator constrains flows to engineered user behavior and its task formulation excludes some dialogue types.
Problem
New goal-oriented tasks may lack human-agent dialogue data, while Wizard-of-Oz collection can be expensive, incomplete, linguistically unsuitable, and error-prone.
Method
M2M uses a developer-provided task specification, automated user-system self-play to generate outlines, and crowdsourced rewriting to produce natural-language dialogues.
Results
M2M is reported to provide higher diversity and coverage of salient dialogue flows while maintaining naturalness, and enables rapid creation of high-quality datasets across domains.
Takeaways & Limitations
M2M offers a rapidly bootstrappable and controllable process for training goal-oriented dialogue agents before improvement from real-user feedback.
Takeaways & Limitations
The user simulator restricts generated dialogue flows to behaviors engineered into the user model, limiting unanticipated complex interactions.
Abstract
from arXiv · showhide
We propose Machines Talking To Machines (M2M), a framework combining automation and crowdsourcing to rapidly bootstrap end-to-end dialogue agents for goal-oriented dialogues in arbitrary domains. M2M scales to new tasks with just a task schema and an API client from the dialogue system developer, but it is also customizable to cater to task-specific interactions. Compared to the Wizard-of-Oz approach for data collection, M2M achieves greater diversity and coverage of salient dialogue flows while maintaining the naturalness of individual utterances. In the first phase, a simulated user bot and a domain-agnostic system bot converse to exhaustively generate dialogue "outlines", i.e. sequences of template utterances and their semantic parses. In the second phase, crowd workers provide contextual rewrites of the dialogues to make the utterances more natural while preserving their meaning. The entire process can finish within a few hours. We propose a new corpus of 3,000 dialogues spanning 2 domains collected with M2M, and present comparisons with popular dialogue datasets on the quality and diversity of the surface forms and dialogue flows.
1 Introduction
M2M addresses the cost, coverage, and quality challenges of building task-specific goal-oriented dialogue datasets by combining automated dialogue self-play with crowdsourced rewriting. The framework separates task-specific developer inputs from task-independent dialogue generation while targeting diverse flows, natural language, and correct supervision.
- Motivation: Wizard-of-Oz collection is expensive and lossy because dialogues may miss expected interactions, use unsuitable language, or contain annotation errors.These problems can require manual filtering and cleaning by the dialogue developer.
- Contribution: M2M reduces dataset-building cost by automating task-independent steps while requiring developers to provide task-specific dialogue information.The framework connects developer-provided task information with framework-provided generation components.
- Method: Dialogue self-play uses an agenda-based user simulator and finite-state system agent to exhaustively generate dialogue histories from discrete conversational actions.This approach is inspired by AI game-playing self-play and coordinates multiple conversational agents for multi-turn generation.
- Method: M2M maps a task specification consisting of a schema and API client into dialogues centered on completing database-querying tasks.The framework divides generation into outline creation and conversion of outlines into natural-language dialogues.
- Method: Self-play samples user scenarios and varies user and system behavior to generate diverse dialogue outlines covering preferences, database queries, modifications, information requests, and transactions.Outlines represent dialogue flow with template utterances and annotations while abstracting surface-language variation.
3 User simulation and dialogue self-play
M2M uses a simulated user and a rule-based system bot to generate task-specific dialogue outlines, while exposing the simulator’s coverage limitation.
- User simulation: A domain-general user simulator can be seeded with task-specific intents, slot names, and values to generate dialogue flows for a task.This design aims to reuse common user-behavior patterns across broad classes of tasks.
- User simulation: A user simulator restricts generated dialogue flows to behaviors engineered into its model.The authors frame this as a trade-off between anticipating unplanned interactions and reliably covering critical expected interactions.
- Dialogue self-play: The system bot is a finite-state machine that uses task-independent rules to construct responses and advance through predetermined sub-dialogues.Its design supports valid system turns for the task while allowing developers to modify component rules.
4 Crowdsourcing
M2M shifts difficult dialogue annotation and interaction generation away from crowdsourcing by using self-play for outlines and simpler paraphrase-oriented tasks for workers.
- Crowdsourcing: Wizard-of-Oz collection requires crowd workers to converse and later annotate dialogue acts and slot spans, making the process expensive and time-consuming.Workers must understand the precise meaning of utterances to assign the annotations.
- Crowdsourcing: Wizard-of-Oz workers may produce interactions that do not reflect real users, such as stating all constraints in one turn.Such low-quality dialogues require manual removal or additional worker instruction and training.
- Crowdsourcing: M2M uses dialogue self-play to systematically generate usable outlines, then simplifies crowdsourcing to a dialogue paraphrase task.This replaces unconstrained interaction collection with contextual rewriting of template utterances.
5 Datasets
The authors release two M2M-collected datasets totaling 3,000 dialogues for movie-ticket purchase and restaurant-table reservation tasks.
- Datasets: 3,000 dialogues span two tasks: buying a movie ticket and reserving a restaurant table.The datasets were created by generating outlines with dialogue self-play and rewriting template utterances through crowdsourcing.
6 Evaluations
The evaluation examines whether M2M improves dialogue-feature coverage and assesses the quality of its generated dialogues through crowd ratings.
- 6.1 Dialogue diversity: M2M Restaurants has greater surface-form variety than DSTC2, as indicated by higher unique unigram and bigram ratios.
- 6.1 Dialogue diversity: 3x more unique transitions per turn distinguish M2M from DSTC2 in dialogue-flow diversity.
- 6.1 Dialogue diversity: M2M has fewer repeated subdialogues than DSTC2 for sequences of 3 and 5 transitions.
- 6.2 Dialogue quality: Each completed dialogue turn was rated by 3 crowd workers on multiple quality dimensions using a 1-to-5 scale.Table 3 reports mean and standard deviation scores aggregated across dataset turns.
7 Related work and discussion
M2M is presented as an extensible alternative for rapidly bootstrapping goal-oriented agents, combining controllable generation with crowdsourced paraphrasing. Its components can be adapted or replaced for more complex tasks and integrated with dialogue-system tooling.
- M2M rapidly bootstraps goal-oriented conversational agents and is presented as producing high-quality datasets across arbitrary domains.
- The framework is controllable through the task schema, scenario generator, user profile and behavior, system policy, and template generator.
- The framework could be integrated into PyDial by adding dialogue self-play functionality.
- Task-general bots can handle transactional or form-filling tasks from a task schema, while developers may add rules for more complex tasks.The canonical utterance generator can also be extended, and components could be replaced by learned generative models.
- M2M crowdsourcing asks workers to paraphrase utterances in tasks designed to avoid both open-endedness and excessive restriction.This design addresses the lower-quality dialogues that may result from open-ended crowdsourcing tasks.
A Supplemental Material
The supplemental material documents M2M’s dialogue-act inventory, a multi-domain dialogue example, and interfaces for contextual rewriting and dialogue-quality evaluation.
- Table 4 lists the dialogue acts used in the M2M setup, based on the Cambridge dialogue act set.
- Table 5 presents a dialogue outline and paraphrase spanning interdependent movie-ticket and restaurant-reservation tasks.
- Figure 3 shows the crowd interface for paraphrasing a dialogue outline into natural language.
- Figure 4 shows the crowd interface for rating user and system turns in completed dialogues.