Source-linked AI summary
Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
Yichi Zhang, Zhijian Ou, Zhou Yu
TL;DR
Task-oriented dialog systems must handle multiple appropriate responses for the same context, but prior approaches do not model this one-to-many property. MADA discovers valid state-to-action mappings and augments training with them, yielding more diverse responses and state-of-the-art MultiWOZ results.
Problem
Existing task-oriented dialog generation approaches overlook that multiple system responses and policies can be appropriate for the same context.
Method
MADA summarizes dialog history as dialog states, discovers their valid system actions, and adds those actions as extra training pairs.
Results
DAMD with MADA achieves state-of-the-art task success on MultiWOZ and generates responses that are both appropriate and diverse.
Takeaways & Limitations
A more balanced state-to-action policy supports diverse, appropriate response generation across task-oriented dialog settings with annotated belief states and system actions.
Takeaways & Limitations
Automatic evaluation makes the appropriateness of multiple actions difficult to judge, so the paper leaves this assessment to human evaluation.
Abstract
from arXiv · showhide
Conversations have an intrinsic one-to-many property, which means that multiple responses can be appropriate for the same dialog context. In task-oriented dialogs, this property leads to different valid dialog policies towards task completion. However, none of the existing task-oriented dialog generation approaches takes this property into account. We propose a Multi-Action Data Augmentation (MADA) framework to utilize the one-to-many property to generate diverse appropriate dialog responses. Specifically, we first use dialog states to summarize the dialog history, and then discover all possible mappings from every dialog state to its different valid system actions. During dialog system training, we enable the current dialog state to map to all valid system actions discovered in the previous process to create additional state-action pairs. By incorporating these additional pairs, the dialog policy learns a balanced action distribution, which further guides the dialog model to generate diverse responses. Experimental results show that the proposed framework consistently improves dialog policy diversity, and results in improved response diversity and appropriateness. Our model obtains state-of-the-art results on MultiWOZ.
Introduction
Task-oriented dialogs have multiple appropriate responses for the same context, but existing systems overlook this one-to-many property. MADA addresses the resulting policy imbalance by learning mappings from dialog states to multiple valid actions, improving response diversity and appropriateness on MultiWOZ.
- Motivation: Task-oriented dialog datasets contain one reference response per context, leaving valid system-action frequencies highly unbalanced.Models therefore tend to capture common policies while ignoring feasible but rare user behaviors, producing skewed, low-coverage policies.
- Approach: MADA maps each condensed dialog state to multiple valid system actions instead of learning only one response mapping.The state-action representation is more structured because unique dialog states and system actions are fewer than unique utterances.
- Approach: Additional valid state-action pairs are incorporated during training so the dialog policy learns a more balanced action distribution.This augmentation is designed to enrich policy diversity and guide generation toward diverse system responses.
- Results: On MultiWOZ, the proposed approach improves response generation quality across learning models and achieves state-of-the-art response-generation results.Human evaluation also finds that DAMD with augmentation produces diverse and valid responses.
Related Work
Prior work models one-to-many variation mainly in open-domain dialog, while task-oriented systems largely focus on task completion or retrieval. This work instead uses diverse, interpretable system actions to generate varied task-oriented responses.
- Task-oriented dialog: Task-oriented dialog research has also shifted from independently trained models toward end-to-end trainable architectures.Examples include joint belief-state tracking and response generation in a single sequence-to-sequence model.
- Open-domain dialog: Open-domain systems model one-to-many variation using response-level information such as specificity to increase social-response diversity.These methods address the larger response space of social conversation through additional response metadata.
- Task-oriented dialog: Existing task-oriented work that models one-to-many variation retrieves system responses rather than generating them.The proposed approach targets generation of more diverse task-oriented dialog responses.
- Task-oriented dialog: The proposed multiple system actions are interpretable and controllable, and diverse decoding methods can increase action diversity.This contrasts action-level generation with response retrieval while retaining semantic policy structure.
Multi-Action Data Augmentation Framework
MADA discovers valid mappings from summarized dialog states to system actions and augments training with additional actions for each state. It balances action distributions while using state representations and sampling choices suited to multi-domain task-oriented dialogs.
- Framework overview: MADA discovers one-to-many mappings from summarized dialog states to sets of appropriate system actions, then adds valid actions as training examples.The balanced mapping is intended to produce diverse system actions and, consequently, diverse appropriate responses.
- Dialog representation: Each dialog state summarizes current domain, belief state, database search results, and current user action.These components provide information used to determine the next system action, including domain-specific search and direct user feedback.
- Dialog representation: A system action represents the semantic content of a system utterance through dialog domain, dialog act, and slots.For example, “hotel-request(price, area)” specifies a domain, act, and requested slots.
- Balanced policy learning: For a given state, multiple valid actions may exist, but maximum-likelihood training on imbalanced pairs favors majority mappings and reduces policy diversity.MADA addresses this by incorporating other valid actions associated with the same state as additional training data.
- Balanced policy learning: MADA over-samples rather than under-samples so the model can learn from all available dialogs.It can group semantically similar actions by dialog-act type and uniformly sample across groups to improve efficiency and action-type diversity.
- Scope: The framework requires system-action annotations to obtain state-action mappings and is applicable to task-oriented models using system-action supervision.MultiWOZ is used to evaluate the framework in a challenging multi-domain setting.
Domain Aware Multi-Decoder Network
DAMD is an end-to-end multi-domain model with separate decoders for belief spans, system actions, and responses. It uses domain-aware representations and diverse action decoding to generate varied responses.
- Architecture: DAMD uses one encoder and three decoders for belief span, system action, and system response generation.The model is trained end-to-end with the three decoder losses optimized jointly.
- Domain-Adaptive Delexicalization: Domain-adaptive delexicalization represents the same slot across domains with one token, while the active domain resolves response-generation ambiguity.For example, domain-independent slot tokens such as <v.phone> are used.
- Dialog State: The dialog state combines the current domain, belief state, database results, and current user action to represent information needed for policy decisions.The belief span is updated from the previous response, current user utterance, and previous belief span using a copy-enabled sequence-to-sequence decoder.
- System Action Decoder: The system action decoder represents actions as domain-act-slot token sequences and incorporates additional valid state-action mappings from the augmentation framework.Database results are combined with utterance and belief-state representations when decoding actions.
- Response Generation: During testing, diverse beam search, top-k sampling, and nucleus sampling generate different system actions and corresponding responses.Responses are generated from the decoded action, user utterance, belief state, and database results.
Dataset
MultiWOZ is a challenging human-human, Wizard-of-Oz task-oriented dialog dataset spanning multiple domains and complex multi-domain conversations.
- Dataset Composition: MultiWOZ contains tourist-clerk conversations across hotel, restaurant, attraction, train, taxi, hospital, police, and general domains.The general domain covers acts such as greeting and goodbye.
- Multi-Domain Setting: Each dialog covers one to three domains, and multiple domains may appear within a single turn.The dataset combines a complex ontology with flexible human expressions.
- Challenge: MultiWOZ is described as extremely challenging for dialog-system development because of its multi-domain setting, complex ontology, and flexible expressions.
Experimental Settings
Experiments evaluate multi-action diversity and response generation on MultiWOZ using DAMD variants, baselines, automatic metrics, and sampling methods.
- Evaluation Metrics: Response quality is measured by inform rate, success rate, BLEU, and a combined score defined as (Inform + Success) × 0.5 + BLEU.
- Baselines and Model Variations: The study compares DAMD model variations with other baselines on MultiWOZ response generation.The comparison includes Seq2Seq models, MD-Sequicity, SFN + RL, and HDSA.
- Decoding: Top-k sampling selects the next word from the k most probable vocabulary choices, while top-p sampling uses the smallest probability mass reaching p.
- Implementation: DAMD uses a one-layer bidirectional GRU encoder, three GRU decoders, Adam optimization, and development-set combined score validation.The encoder and decoders use hidden size 100, with embedding size 50, vocabulary size 3,000, and batch size 128.
Results and Analysis
MADA consistently improves dialog-policy diversity across models, while DAMD achieves strong task-completion performance on MultiWOZ. Human evaluation further indicates higher response diversity and appropriateness, although the method still has slot, information-collection, and generation errors.
- Policy diversity: Action and slot diversity improve consistently after multi-action data augmentation across evaluated models.Top-k sampling reaches 3.43 unique dialog acts on average across five generated actions.
- Task completion: DAMD significantly outperforms models with different system-action forms on inform and success rates.With oracle belief states, augmentation provides only a limited combined-score improvement, from 6 to 7.
- Case study: Balanced state-action training helps capture less frequent recommendation policies that unbalanced training tends to miss.In the hotel example, recommendation becomes available alongside the common policy of requesting another slot.
- Error analysis: The model still makes slot-type, insufficient-information, and response-generation errors despite outperforming models trained on unbalanced state-action data.Reported causes include ignored slot-level diversity, insufficient injected task knowledge, and forgetting distant information under long action spans.
- Human evaluation: Human evaluation finds that augmentation significantly improves diversity and gives DAMD the best average appropriateness score.Appropriateness is rated from 1 to 3, while diversity is rated from 1 to 5 across five responses per model and 100 dialog turns.
Conclusion
The paper addresses one-to-many task-oriented dialog behavior by learning diversified dialog policies that generate responses with higher diversity while preserving appropriateness. MADA and DAMD support balanced state-to-action mappings and achieve state-of-the-art task success on MultiWOZ.
- MADA enables dialog models to learn a more balanced state-to-action mapping for higher-diversity task-oriented responses.The framework is intended for dialog tasks with annotated belief states and system actions.
- DAMD leverages MADA to learn a more diverse state-to-action policy and generate responses that are both appropriate and diverse.
- DAMD achieves the state-of-the-art task success rate on the challenging MultiWOZ dataset.