Multi-domain Dialog State Tracking using Recurrent Neural Networks

• Most modern dialog systems target a single, well-defined domain, while open-domain conversation remains difficult.Source: §1 · p. 1
• The paper proposes multi-domain RNN belief trackers that use dialog data from different domains to learn generalizable dialog features.Source: §1 · p. 1
• The hierarchical procedure first learns general dialog features across domains, then specializes models for individual domains while retaining cross-domain patterns.Source: §1 · p. 1
• The resulting trackers show robust performance across investigated domains and typically outperform models trained only on target-domain data.Source: §1 · p. 1
• Cross-domain initialization improves tracking for entirely new domains regardless of the amount of in-domain training data available.Source: §1 · p. 1

• Data-driven dialog trackers replaced traditional rule-based systems because they are more robust and provide probabilistic state distributions for POMDP managers.Source: §2 · p. 1
• RNN belief trackers map ASR output directly to belief-state updates without complex semantic decoders while achieving state-of-the-art performance.Source: §2 · p. 2
• Earlier domain-adaptation work in dialog addressed tagging, generation, or expanded versions of the same topic rather than disjoint dialog domains.Source: §2 · p. 2

• Belief tracking represents user goals as slot-value constraints, with each domain’s slots and possible values defining its ontology.Source: §3 · p. 2
• The RNN outputs a distribution over goal slot-value pairs for each utterance while maintaining a memory vector for dialog context.Source: §3 · p. 2
• Each turn combines ASR hypotheses, the last system action, the current memory vector, and the previous belief state as model input.Source: §3 · p. 2
• Lexical features capture dialog dynamics but are not ideal for sharing information across different slots and domains.Source: §3 · p. 2
• Delexicalised n-grams replace slot names and values with generic tags, allowing transfer across slots and operation on unseen values or new slots.Source: §3 · p. 2
• The model updates a separate distribution for each slot and value while also updating its recurrent memory state.Source: §3 · p. 2
• Training unrolls the model across turns using backpropagation through time and stochastic gradient descent.Source: §3 · p. 2
• The architecture omits the original component mapping lexical n-grams to specific slot values to move toward domain independence.Source: §3 · p. 2

• Delexicalisation enables transfer between domains because generic features learned for one topic can support tracking in another.Source: §4 · p. 3
• Shared initialization ties parameters across slots and replaces slot-value occurrences with one generic tag before training a shared RNN.Source: §4 · p. 3
• The multi-domain shared model is trained on a combined dataset formed by delexicalising slot values across all domains.Source: §4 · p. 3
• The shared RNN extracts lexical and delexicalised features capturing general dialog dynamics, while later specialization learns their differing relevance to particular slots.Source: §4 · p. 3

• Joint goal accuracy measures whether the tracker correctly infers users’ constraints across all slots in a domain.Source: §5 · p. 3
• The evaluation covers six domains varying in topic and geographical location, including restaurant, hotel, tourist-information, and laptop dialogs.Source: §5 · p. 3
• The Laptops dataset is substantially different from the other domains, providing a challenging test of multi-domain models.Source: §5 · p. 3
• The All Restaurants model combines data from three restaurant domains, while the R+T+H model adds hotels, pubs, and coffee shops.Source: §5 · p. 3
• The R+T+H+L model is the most general model, trained on all available dialog data.Source: §5 · p. 3

6.1 Training General Models

• Multi-domain models improve over domain-specific models for all but one constituent domain, and the broadest model achieves the highest geometric mean across six domains.The R+T+H model performs better across four domains than R+T+H+L, but the latter balances performance more effectively across all six domains.Source: §6.1 · pp. 3–4

6.2 Slot-specialising the General Models

• Slot-specialisation improves performance in the vast majority of experiments, and all three slot-specialised general models outperform the RNN result reported in DSTC 2.Source: §6.2 · p. 4

6.3 Out of Domain Initialisation

• The hierarchical procedure uses out-of-domain dialogs to initialise improved shared models for new domains.The evaluation compares slot-specialised models trained from in-domain-only and combined in-domain plus out-of-domain shared models.Source: §6.3 · p. 4
• Out-of-domain initialization improves joint goal accuracy even when the entire in-domain dataset is available, with the strongest starting-point benefit when in-domain data is small.Figure 1 evaluates Michigan Restaurants and Laptops as a function of available in-domain training dialogs.Source: §6.3 · pp. 4–5, Figure 1 · p. 5
• The Laptops experiment shows a larger and more consistent initialization advantage because out-of-domain dialogs provide delexicalised features absent from the in-domain training data.Additional in-domain dialogs then supply features not present in the out-of-domain data, producing continued improvements.Source: §6.3 · p. 5

• General belief-tracking models can address multiple dialog topics at once.Source: §7 · p. 5
• Training on diverse domains helps models capture dialog dynamics applicable across domains.Source: §7 · p. 5
• The most general model robustly outperforms most domain-specific models across the investigated domains.Source: §7 · p. 5
• Hierarchical training adapts the general model to new domains using very small in-domain datasets.Source: §7 · p. 5
• The adaptation procedure still improves tracking when substantial in-domain data are available, but it requires annotated in-domain dialogs.Source: §7 · p. 5, §7.1 · p. 5