Source-linked AI summary
Dialogue Act Classification with Context-Aware Self-Attention
Vipul Raheja, Joel Tetreault
TL;DR
Dialogue-act classification often treats utterances in isolation, limiting its use of conversational dependencies. This paper combines hierarchical neural modeling with context-aware self-attention and reports improved benchmark performance, including a 1.6% gain over prior methods on SwDA.
Problem
Many dialogue-act classification approaches treat each utterance in isolation, leaving them unable to leverage conversation-level contextual dependencies that can help predict the current act.
Source: §1 · p. 1Method
Results
The model outperforms state-of-the-art methods on SwDA by 1.6% and comes within 0.6% on MRDA.
Source: §5.1 · p. 4Takeaways & Limitations
Self-attention enables the model to learn richer, more effective utterance representations for dialogue-act classification.
Source: §6 · p. 5On MRDA, noisier label dependencies and longer conversations make learning long-range dependencies challenging and limit the model's gains relative to SwDA.
Source: §5.1 · p. 4
Abstract
from arXiv · showhide
Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks. We build on this prior work by leveraging the effectiveness of a context-aware self-attention mechanism coupled with a hierarchical recurrent neural network. We conduct extensive evaluations on standard Dialogue Act classification datasets and show significant improvement over state-of-the-art results on the Switchboard Dialogue Act (SwDA) Corpus. We also investigate the impact of different utterance-level representation learning methods and show that our method is effective at capturing utterance-level semantic text representations while maintaining high accuracy.
1 Introduction
Dialogue acts are utterance functions that serve as basic units of communication and support tasks such as question answering and conversational agents. Because utterances depend on conversational context, the paper proposes a hierarchical model that uses context at multiple levels and explores ways to learn utterance representations.
1 Introduction
- Conversation context matters: preceding utterance text or dialogue-act labels can help predict the current act, as answers or statements often follow questions.The introduction identifies isolated-utterance classification as a limitation of many recent approaches.Source: §1 · p. 1
- The proposed hierarchical model combines sequence prediction with a CRF, conversation-level context via self-attention, and character embeddings.The paper reports state-of-the-art performance on the Switchboard Dialogue Act Corpus.Source: §1 · p. 1
- The paper also explores different methods for learning utterance representations, which serve as building blocks for its hierarchical dialogue-act classifier.Source: §1 · p. 2
2 Related Work
Prior dialogue-act classification research includes both utterance-level text classifiers and sequence-labeling models, with recent work increasingly using hierarchical architectures. This work combines techniques from prior models with self-attentive representation learning.
2 Related Work
- Earlier studies approached dialogue-act classification as either structured prediction or text classification, while many recent studies applied deep learning.Source: §2 · p. 2
- Sequence-labeling research increasingly used hierarchical architectures to model utterance and dialogue-act dependencies, often pairing RNNs with CRF decoders.Chen et al. (2018) achieved the highest performance to date on the two datasets discussed.Source: §2 · p. 2
- The paper extends hierarchical models by combining CRF decoding, contextual attention, and character-level embeddings with self-attentive representation learning.The authors report that this combination achieves state-of-the-art performance.Source: §2 · p. 2
3 Model
The model maps conversations to dialogue-act label sequences using utterance-level encoding, context-aware self-attention, and conversation-level sequence modeling. Its attention uses conversation context to represent each utterance, and a CRF jointly decodes labels.
3 Model
- Each conversation is a variable-length sequence of utterances mapped to a corresponding sequence of dialogue-act labels.Each utterance is itself a sequence of words.Source: §3 · p. 2
- The architecture combines an utterance-level RNN, context-aware self-attention over word representations, and a conversation-level RNN followed by a CRF.The utterance-level RNN encodes word- and character-level information; the attention aggregates word representations into utterance representations.Source: §3 · p. 2
3.2 Context-aware Self-attention
- The self-attention combines word-level GRU states with the conversation context so far to produce a two-dimensional representation for each utterance.The representation is projected to a one-dimensional utterance embedding for the conversation-level GRU.Source: §3.2 · p. 3
- The conversation-level context from one utterance is used to learn attention scores and a two-dimensional representation for the next utterance.Source: §3.2 · p. 3
3.3 Conversation-level RNN
- A bidirectional conversation-level GRU encodes utterances, and a linear-chain CRF jointly decodes the optimal label sequence using correlations between contextual labels.Source: §3.3 · p. 3
4 Data
The evaluation uses standard SwDA and MRDA datasets with established train, validation, and test splits. The datasets are highly imbalanced, with dominant dialogue-act classes accounting for over half of labels in each set.
4 Data
- Evaluation covers SwDA with 43 classes and the 5-class version of MRDA, using the train, validation, and test splits defined in Lee and Dernoncourt (2016).Source: §4 · pp. 3–4
- Over 50% of labels in each dataset belong to its most frequent dialogue-act classes: Statement-non-opinion and Acknowledge/Backchannel in SwDA, and Statement in MRDA.Source: §4 · p. 4
5 Results
The model outperforms prior state of the art on SwDA and approaches it on MRDA. The results also show gains from self-attention and conversation-level context, while utterance-representation choices affect performance.
5.1 Dialogue Act Classification
- The model outperforms state-of-the-art methods by 1.6% on SwDA and comes within 0.6% on MRDA.It also beats the TF-IDF GloVe baseline by 16.4% on SwDA and 12.2% on MRDA.Source: §5.1 · p. 4, Table 3 · p. 4
- Compared with SwDA, MRDA’s sixfold normalized label entropy reflects noisier label dependencies and longer network path lengths, challenging long-range learning and limiting the model’s gains on MRDA.Previous-label entropy correlated negatively with accuracy on MRDA (-0.68), while SwDA showed a low positive correlation (+0.22), which could be compensated by significantly shorter conversations.Source: §5.1 · p. 4
5.2 Utterance Representation Learning
- The tested utterance representations include TF-IDF-weighted GloVe, corpus-pretrained Skip-Thought or Paragraph Vectors, and jointly learned representations.Table 4 compares their performance within the hierarchical model on both datasets.Source: §5.2 · pp. 4–5, Table 4 · p. 5
- Self-attention improves performance beyond word-level attention, and adding conversation context boosts it significantly.The representation-learning comparisons replace the utterance-level RNN plus context-aware self-attention with alternative sentence-representation methods before the conversation-level recurrent layers.Source: §5.2 · pp. 4–5
- Combining conversation-level context in utterance representation learning with a conversation-level CRF yields collective performance gains, particularly on SwDA.Both context-aware attention variants show significant gains on SwDA.Source: §5.2 · p. 5
6 Conclusion
The model significantly outperforms earlier models on SwDA and is close to state of the art on MRDA. The authors conclude that self-attention yields effective utterance representations and identify alternative attention mechanisms for future investigation.
6 Conclusion
- The model significantly outperforms earlier models on SwDA and is very close to state of the art on MRDA.The conclusion attributes richer, more effective utterance representations to self-attention.Source: §6 · p. 5
- Self-attention enables richer, more effective utterance representations for dialogue act classification.The conclusion reports experiments with different utterance-representation learning methods and says lower-level representations can affect higher-level classification performance.Source: §6 · p. 5
- Future work will explore other attention mechanisms, including multihead, directional, block, and hierarchical attention.The cited alternatives are described as addressing limitations of vanilla attention and self-attention through context, temporal order, or granularity.Source: §6 · p. 5
A.1 Training & Hyperparameters
Hyperparameters were tuned one at a time on validation splits, then the chosen settings were used to train separate SwDA and MRDA models.
A.1 Training & Hyperparameters
- The authors tuned one hyperparameter at a time on validation splits before training the final models.Table 5 lists the tested ranges and chosen values.Source: §A.1 · p. 7, Table 5 · p. 7
- The final hyperparameter settings were used to train separate models on the SwDA and MRDA training splits.Source: §A.1 · p. 7
- Dropout was applied to utterance embeddings, and validation-based early stopping used a patience of 15 epochs.Source: §A.1 · p. 7