Source-linked AI summary
Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation
Liliang Ren, Jianmo Ni, Julian McAuley
TL;DR
Predefined-ontology dialogue state trackers become harder to scale as slot counts grow, especially in multi-domain dialogues. COMER addresses this by hierarchically generating belief-state sequences with constant inference complexity, while achieving similar state-of-the-art performance on MultiWoZ and WoZ.
Problem
Predefined-ontology DST models require scoring candidate slot-value pairs, and their inference complexity grows with the number of slots, making larger multi-domain ontologies difficult to scale.
Source: §1 · pp. 1–2Method
COMER uses a hierarchical encoder-decoder that generates slot sequences followed by value sequences, with shared decoder parameters and conditional memory over belief, system, and user representations.
Source: §1 · p. 2, §3.2 · p. 4Results
COMER has constant inference time complexity and achieves similar state-of-the-art joint goal performance on both MultiWoZ and WoZ.
Source: §4.3 · pp. 6–7, §6 · p. 9Takeaways & Limitations
The hierarchical generation framework scales with predefined domains, slots, and values while retaining competitive dialogue state tracking performance.
Source: §1 · p. 2, §6 · p. 9The paper identifies the single-value assumption in existing dialogue-state datasets as a representational limitation, since some slots require multiple ordered values.
Source: §2 · p. 2
Abstract
from arXiv · showhide
Existing approaches to dialogue state tracking rely on pre-defined ontologies consisting of a set of all possible slot types and values. Though such approaches exhibit promising performance on single-domain benchmarks, they suffer from computational complexity that increases proportionally to the number of pre-defined slots that need tracking. This issue becomes more severe when it comes to multi-domain dialogues which include larger numbers of slots. In this paper, we investigate how to approach DST using a generation framework without the pre-defined ontology list. Given each turn of user utterance and system response, we directly generate a sequence of belief states by applying a hierarchical encoder-decoder structure. In this way, the computational complexity of our model will be a constant regardless of the number of pre-defined slots. Experiments on both the multi-domain and the single domain dialogue state tracking dataset show that our model not only scales easily with the increasing number of pre-defined domains and slots but also reaches the state-of-the-art performance.
1 Introduction
Dialogue state tracking traditionally scores predefined slot-value candidates, making inference complexity grow with ontology size and limiting scalability to multi-domain dialogues. COMER instead generates hierarchical belief-state sequences with shared parameters and constant inference complexity.
1 Introduction
- Multi-domain belief states use a hierarchy of domains, slots, and values, reflecting dialogues that include multiple domains simultaneously.Traditional states are represented through slot-value pairs, while MultiWoZ introduces the hierarchical domain-slot-value structure.Source: §1 · p. 1
- Predefined-ontology DST models score candidate slot-value pairs, but their inference complexity grows with the number of slots and struggles to scale across larger multi-domain ontologies.The inference-time comparison is defined using m ontology values and n slots.Source: §1 · pp. 1–2, Table 1 · p. 1
- COMER formulates dialogue state tracking as sequence generation and reports constant inference time complexity regardless of predefined ontology size.This replaces pair-wise prediction over candidate slot-value pairs with direct belief-state generation.Source: §1 · pp. 1–2
- COMER uses a hierarchically stacked encoder-decoder that first generates slot sequences and then generates value sequences for each slot, sharing parameters across decoders.BERT contextualized embeddings and BPE support sequence encoding and representations of unseen words.Source: §1 · p. 2
2 Motivation
Multi-domain belief states must preserve domain-slot structure, while single-value representations can fail when a slot requires multiple ordered values. The paper motivates structured sequence generation to represent such dialogue states accurately.
2 Motivation
- Multi-domain dialogue tracking must represent each slot-value pair together with its domain, such as train and hotel goals in one dialogue.A combined slot-value representation can encode the domain but ignores the subordination relationship between domains and slots.Source: §2 · p. 2, Figure 1 · p. 2
- The WoZ2.0 example shows that a single value such as seafood cannot support a response about Chinese restaurants, requiring the structured value seafood > chinese.The operator > indicates that information retrieved for seafood should have higher priority than information retrieved for Chinese restaurants.Source: Figure 2 · p. 2, §2 · p. 2
- Structured sequence generation is therefore needed to represent multiple values and their retrieval priority within a dialogue-state slot.The example motivates DST modules that can generate structured value sequences rather than assuming one value per slot.Source: §2 · p. 2
3 Hierarchical Sequence Generation for DST
COMER formulates dialogue state tracking as hierarchical sequence generation, producing domains, slots, and values rather than scoring predefined slot-value candidates. Its shared Conditional Memory Relation Decoders encode dialogue context and enable inference complexity that is constant with respect to predefined slots and values.
3.2 Conditional Memory Relation Decoder
- All Conditional Memory Relation Decoders share parameters, allowing the hierarchical structure to scale without separate parameters for each domain or slot.The shared decoders form a two-dimensional autoregressive model over condition generation and sequence generation.Source: Figure 3 · p. 3, §3.2 · p. 4
3 Hierarchical Sequence Generation for DST
- COMER adapts dialogue state tracking from pair-wise prediction to sequence generation with a hierarchically stacked encoder-decoder network.The model generates domain sequences, then slot sequences for each domain, and value sequences for each slot.Source: §3 · p. 3
3.1 Encoding Module
- COMER encodes user utterances, system actions, and previous belief states with BERT-based embeddings and bidirectional LSTMs before decoding.The CMR decoder combines belief, system, and user representations through hierarchical attention, residual connections, and relation reasoning.Source: §3 · p. 3, §3.1 · pp. 3–4, §3.2 · p. 4
3 Hierarchical Sequence Generation for DST
- The hierarchical procedure first generates domains, then domain-specific slots, and finally slot-specific values, updating the belief state iteratively.The first decoder generates domains; subsequent decoders condition on domain and slot representations to generate the next hierarchy level.Source: §3 · p. 3
- O(1) inference time complexity with respect to predefined slots and values follows because COMER generates domains and slots instead of receiving them as fixed inputs.The number generated at each turn depends on the dialogue contents rather than the number of predefined ontology entries.Source: §3 · p. 3
3.2 Conditional Memory Relation Decoder
- The CMR decoder uses a dynamic vocabulary whose parameterization is independent of vocabulary size, supporting generation of unseen words.Its hidden representation is projected to the embedding space and compared with token embeddings to produce output probabilities.Source: §3.2 · p. 5
4 Experiments
Experiments on WoZ2.0 and MultiWoZ evaluate COMER’s accuracy, scalability, hierarchical generation, and qualitative attention behavior. COMER maintains near-state-of-the-art joint goal accuracy while achieving approximately constant inference-time scaling with ontology size.
Experimental Setting
- COMER generates domains, slots, and values hierarchically, and evaluation uses exact matching of the generated belief dictionary for joint goal accuracy.Joint goal accuracy requires all predicted slots and values to match the labels, with empty triplets removed during post-processing.Source: §4.1 · p. 6
Results
- 2.54× actual inference-time scaling from WoZ2.0 to MultiWoZ matches the theoretical 2.15× estimate, supporting O(1) complexity with respect to ontology slots and values.Runtimes were 65.6 seconds and 835.2 seconds across validation sets, averaged over five runs and adjusted for the fivefold dataset-size difference.Source: §4.3 · p. 6, §4.1 · p. 6
- 48.79% joint goal accuracy on MultiWoZ marginally outperforms the previous state of the art, while WoZ2.0 performance drops only 0.3%.Table 3 compares joint goal accuracy on both test sets against prior systems.Source: §4.3 · p. 7, Table 3 · p. 7
Ablation Study
- Removing hierarchical attention and residual connections lowers WoZ2.0 accuracy by 1.95%, while removing MLP reduces relational reasoning capacity and accuracy.The MLP effect is attributed partly to fewer parameters and partly to the loss of deeper relational reasoning over concatenated representations.Source: §4.4 · p. 7, Table 4 · p. 7
- Given correct slot prediction, value accuracy reaches 87.42%, while slot accuracy given the correct domain is 58.43%, identifying slot prediction as the main hierarchical weakness.Domain prediction reaches 95.52%; the authors relate the slot weakness to limited previous-belief context and multi-turn mapping.Source: Table 5 · p. 7, §4.4 · p. 8
Qualitative Analysis
- Attention visualizations show CMRD selecting system evidence for departure, user keywords for day and destination, and prior belief slots when available.For a newly generated destination slot absent from the previous belief state, the belief attention has no corresponding prior slot to attend to.Source: §4.5 · p. 8
5 Related Work
Prior dialogue state trackers commonly depend on ontology-defined slots or candidate values, which limits scalability or flexibility as domains and values grow. Related sequence-generation and contextual-embedding work motivates COMER’s generation-based design but retains distinct trade-offs around complexity and vocabulary.
Ontology-Based Trackers
- Semi-scalable trackers avoid enumerating all values but still iterate over every ontology slot and require exact entity extraction from dialogue history.This approach fails when a label is an abstraction without an exact surface-form match in the dialogue.Source: §5 · p. 8
- StateNet supports dynamically changing slots and values with parameters independent of candidate-set size, but its inference complexity remains proportional to the number of slots.It performs inference once for each slot requiring tracking.Source: §5 · p. 8
Sequence Generation Trackers
- TRADE achieves strong MultiWoZ performance using copy-based value generation, but its O(n) inference complexity comes from processing domain-slot combinations.Its improvement is mainly associated with copying names for the name slot.Source: §5 · p. 8
- DSTRead frames tracking as reading comprehension over predefined combined slots, retaining O(n) complexity and limiting generation to words appearing in dialogue history.It also requires a manual combination strategy with another joint state tracker.Source: §5 · p. 9
Foundations
- Contextualized word embeddings produce representations conditioned on complete input context, while sequence-generation models capture label correlations through autoregressive decoding.These ideas provide representational and dependency-learning foundations relevant to dialogue state generation.Source: §5 · p. 9
6 Conclusion
COMER is presented as a dialogue state tracker with constant inference-time complexity relative to ontology size and near-state-of-the-art joint goal accuracy on single- and multi-domain datasets. The conclusion identifies future extensions involving stronger sequence modeling, open-vocabulary generation, and more efficient dialogue-history access.
Conclusion
- COMER provides constant inference-time complexity with respect to predefined domains, slots, and values while achieving similar joint goal accuracy to state-of-the-art systems on MultiWoZ and WoZ.The conclusion presents scalability and competitive accuracy as the model’s central outcomes.Source: §6 · p. 9
- Future work includes transformer structures, open-vocabulary and copy mechanisms for unseen-word generation, and improved dialogue-history access for inter-turn reasoning.These directions follow from the flexibility of the hierarchical encoder-decoder and CMR decoder.Source: §6 · p. 9