Source-linked AI summary
Global-Locally Self-Attentive Dialogue State Tracker
Victor Zhong, Caiming Xiong, Richard Socher
TL;DR
Dialogue state tracking must infer user goals and requests from dialogue context, especially when rare slot-value pairs have few training examples. GLAD combines global parameter sharing with local slot-specific feature learning over user utterances and previous system actions. It achieves state-of-the-art goal and request accuracy on both WoZ and DSTC2.
Problem
Dialogue state tracking must extract rare slot-value pairs that frequently affect turn-level and joint-goal tracking despite limited training examples.
Method
GLAD uses global modules to share parameters across slot estimators and local modules to learn slot-specific representations from user utterances and previous system actions.
Results
GLAD achieves state-of-the-art goal and request accuracy on both WoZ and DSTC2, including 88.1% goal accuracy on WoZ and 74.5% on DSTC2.
Takeaways & Limitations
Global-local sharing and self-attentive representations support generalization to rare slot-value pairs, while temporal dependencies help state tracking.
Takeaways & Limitations
DSTC2 evaluation uses noisy ASR transcriptions, whereas WoZ provides user utterances without ASR outputs.
Abstract
from arXiv · showhide
Dialogue state tracking, which estimates user goals and requests given the dialogue context, is an essential part of task-oriented dialogue systems. In this paper, we propose the Global-Locally Self-Attentive Dialogue State Tracker (GLAD), which learns representations of the user utterance and previous system actions with global-local modules. Our model uses global modules to share parameters between estimators for different types (called slots) of dialogue states, and uses local modules to learn slot-specific features. We show that this significantly improves tracking of rare states and achieves state-of-the-art performance on the WoZ and DSTC2 state tracking tasks. GLAD obtains 88.1% joint goal accuracy and 97.1% request accuracy on WoZ, outperforming prior work by 3.7% and 5.5%. On DSTC2, our model obtains 74.5% joint goal accuracy and 97.5% request accuracy, outperforming prior work by 1.1% and 1.0%.
1 Introduction
Task-oriented dialogue systems use dialogue state tracking to infer user goals and requests from utterances and conversation history, enabling context-sensitive responses. GLAD addresses rare slot-value tracking through global parameter sharing and local slot-specific features, achieving state-of-the-art results on WoZ and DSTC2.
- Motivation: Dialogue state tracking estimates conversation state from the current user utterance and conversation history.The estimated state supports planning the system’s next action and response.
- Motivation: Existing dialogue state trackers can accumulate errors from SLU systems or rely on hand-crafted semantic dictionaries and delexicalization.Direct state inference avoids SLU error accumulation but retains these representation limitations.
- Motivation: 38.6% of WoZ turns contain a joint goal with a rare slot-value pair seen in fewer than 20 training examples.Slot-value pairs average 214.9 training examples, but rare pairs frequently affect joint-goal tracking.
- Contribution: GLAD shares parameters across slot estimators with global modules while learning slot-specific representations through local modules.This design targets generalization to rare slot-value pairs with few training examples.
- Results: 88.1% goal accuracy and 97.1% request accuracy on WoZ exceeded prior best results by 3.7% and 5.5%.On DSTC2, GLAD achieved 74.5% goal accuracy and 97.5% request accuracy, exceeding prior best results by 1.1% and 1.0%.
2 Global-Locally Self-Attentive Dialogue State Tracker
GLAD decomposes dialogue-state prediction into slot-value-specific estimators, using global-local self-attentive encoders for utterances, system actions, and candidate slot values. It scores evidence from the user utterance and previous system actions, while sharing global parameters and retaining slot-specific representations.
- GLAD predicts each slot-value pair with a distinct binary estimator, decomposing the multi-label state prediction problem into binary decisions.The predicted turn state is conditioned on the user utterance and previous system actions.
- The encoder separately represents the user utterance, previous system actions, and candidate slot-value pair using global-locally self-attentive encoders.The model includes an encoder module and a scoring module.
- Global-Locally Self-Attentive Encoder: Global self-attention assigns normalized weights across sequence elements and forms a weighted context from their representations.For each element, GLAD computes a scalar attention score, normalizes scores with softmax, and sums the weighted representations.
- Global-Locally Self-Attentive Encoder: Local self-attention computes a slot-specific context alongside the global context.The encoder defines global and local attention contexts as part of its sequence encoding function.
- Scoring Module: The utterance scorer attends to words conditioned on the candidate slot value, while the action scorer uses previous system actions when the utterance is insufficient.The action pathway can also ignore previous actions through a sentinel action.
- Scoring Module: The final prediction score is a learned weighted combination of utterance and action scores, normalized with the sigmoid function.The combination weight is learned.
3 Experiments
GLAD is evaluated on DSTC2 and WoZ using request and joint-goal tracking accuracy, with ablations examining its attention, recurrent, and global-local sharing components. Results show improvements over prior methods and stronger performance on rare slot-value pairs.
- 3.4 Comparison to Existing Methods: GLAD improves previous state-of-the-art performance on DSTC by 1.1% for joint goal tracking accuracy and 1.0% for turn-level request tracking accuracy.DSTC evaluates noisy ASR outputs instead of user utterances.
- 3.5 Ablation study: Temporal order improves state tracking, as removing LSTMs produces weaker performance.The authors suggest temporal-dependency representations help interpret phrases for state tracking.
- 3.5 Ablation study: Self-attention improves performance over mean-pooling by allowing slot-specific feature learning and attention to relevant words.The model can capture phrases such as “within 5 miles” for the area slot that n-gram and embedding matching may miss.
- 3.5 Ablation study: Global-local sharing improves goal tracking, while using only the global module underperforms on both goal and request tracking.The local-only variant degrades goal tracking without affecting request tracking, whereas the global-only variant lacks slot-specific specializations.
- 3.5 Ablation study: GLAD consistently outperforms both sharing variants for rare slot-value pairs, while abundant training data removes significant performance differences.Figure 5 groups WoZ development-set F1 performance by the number of training occurrences.
4 Related Work
Dialogue state tracking research moved from SLU-dependent pipelines toward direct history-based tracking, while attention and parameter sharing provided relevant modeling foundations. These lines of work motivate GLAD’s approach to dialogue-state estimation.
- Dialogue State Tracking: Joint speech understanding and dialogue tracking can avoid error accumulation from a separate SLU component.Such approaches directly consume ASR N-best lists but rely on hand-crafted features and domain-specific lexicons, limiting scalability.
- Neural attention models in NLP: Self-attention and coattention have improved representation learning across language understanding, generation, and question answering tasks.The cited work includes machine translation, summarization, language modeling, sentiment analysis, natural language inference, semantic role labeling, and coreference resolution.
- Example predictions: Table 3 presents GLAD predictions on the WoZ development split, marking false positives with “+” and false negatives with “−”.The examples illustrate how predicted slot-value pairs differ from ground truth.
- Parameter sharing between related tasks: Parameter sharing between related tasks is a longstanding strategy for improving joint performance.Prior examples include multitask learning, joint multilingual translation, and models combining modality-specific extractors with shared encoders.
5 Conclusions
The paper introduces GLAD, whose global-locally self-attention encoder shares parameters between slots while learning slot-specific features. GLAD generalizes to rare slot-value pairs and achieves state-of-the-art results on both WoZ and DSTC2.
- Architecture: GLAD uses global modules for parameter sharing between slots and local modules for slot-specific feature learning.This global-locally self-attention design is the model’s core architectural contribution.
- Generalization: GLAD generalizes on rare slot-value pairs with few training data.The conclusion links this capability to the model’s global-locally encoder.
- Results: 88.1% goal accuracy and 97.1% request accuracy on WoZ, alongside 74.5% goal accuracy and 97.5% request accuracy on DSTC2.These are reported as state-of-the-art dialogue state tracking results on both tasks.