Source-linked AI summary
DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, Alexander Gelbukh
TL;DR
ERC requires models that capture conversational context despite limitations in sequential context propagation. DialogueGCN uses a graph to model inter-speaker and self-dependencies, and it outperforms baselines and existing state of the art across benchmark datasets. The paper also identifies multimodal information as a direction for addressing remaining errors.
Problem
RNN-based ERC models do not always propagate long-term contextual information effectively, while benchmark datasets lack labels for intent, topic, and personality.
Method
DialogueGCN represents conversations as directed graphs and uses graph convolution to model inter-speaker and self-dependency among utterances.
Results
DialogueGCN outperforms state-of-the-art and baseline models on all evaluated datasets, with statistically significant differences under a paired t-test (p <0.05).
Takeaways & Limitations
Modeling speaker-level context is important for conversational emotion recognition and helps address long-distance dependency issues in sequential encoders.
Takeaways & Limitations
The experiments use multimodal datasets but do not utilize audio and visual modalities, causing some non-neutral utterances to be misclassified.
Abstract
from arXiv · showhide
Emotion recognition in conversation (ERC) has received much attention, lately, from researchers due to its potential widespread applications in diverse areas, such as health-care, education, and human resources. In this paper, we present Dialogue Graph Convolutional Network (DialogueGCN), a graph neural network based approach to ERC. We leverage self and inter-speaker dependency of the interlocutors to model conversational context for emotion recognition. Through the graph network, DialogueGCN addresses context propagation issues present in the current RNN-based methods. We empirically show that this method alleviates such issues, while outperforming the current state of the art on a number of benchmark emotion classification datasets.
1 Introduction
ERC matters for applications such as affective dialogue systems, where emotion depends on conversational context. DialogueGCN models speaker dependencies and relative utterance positions with a directed graph to address limitations of sequential context propagation.
- ERC supports affective dialogue systems that use users’ emotions and sentiment to generate emotionally coherent, empathetic responses.
- RNN-based ERC models process utterances sequentially, but long-term contextual information is not always propagated effectively.
- Attention-based DialogueRNN variants pool conversation information without modeling speaker identity or relative utterance positions.
- DialogueGCN represents utterances as graph nodes and encodes speaker dependencies together with relative positions between context and target utterances.The graph is directed, and its edges represent dependency relations between utterances.
- The resulting representations capture richer contextual information from distant utterances than DialogueRNN, as evaluated experimentally.
2 Related Work
ERC research has grown with open conversational datasets and includes textual, acoustic, and visual settings. Graph neural networks provide related techniques for relational modeling and other prediction tasks.
- ERC has attracted increased attention as open-sourced conversational datasets have become more available.
- Prior ERC research includes models for multimodal data containing textual, acoustic, and visual information.
- Graph neural networks have been applied to semisupervised learning, entity classification, link prediction, and large-scale knowledge-base modeling.
3 Methodology
DialogueGCN combines sequential and speaker-level context modeling for emotion recognition in conversation. It represents utterances as a directed, relation-labeled graph and applies local graph convolutions to encode speaker-dependent context.
- Context Modeling: ERC modeling distinguishes sequential context from speaker-level context, represented through neighboring utterances around each target utterance.The paper also emphasizes inter-speaker dependency and self-dependency as central aspects of conversational emotional dynamics.
- Framework: DialogueGCN comprises a Sequential Context Encoder, Speaker-Level Context Encoder, and Emotion Classifier.The framework is illustrated in Figure 3.
- Sequential Context Encoder: A bidirectional GRU transforms each utterance into a sequential context-aware representation before graph construction.The initial encoding is speaker agnostic because utterances are encoded irrespective of their speakers.
- Graph Construction: Past and future context windows restrict each utterance's graph neighborhood, avoiding the O(N^2) edges of a fully connected graph.Each vertex connects to p past utterances, f future utterances, and itself; the graph may contain edges in both directions with different relations.
- Feature Transformation: Relation-specific graph convolutions aggregate local neighborhood information in two steps to transform speaker-independent features into speaker-dependent representations.The resulting speaker-level features are intended to encode inter-speaker and self-dependency, and are combined with sequential features for classification.
4 Experimental Setting
DialogueGCN is evaluated on three multimodal benchmark datasets using only textual utterance information, with comparisons against context-aware, memory-based, and speaker-modeling baselines. The datasets differ in dialogue structure and emotion annotation, while IEMOCAP and AVEC use a validation split taken from training dialogues.
- Datasets Used: DialogueGCN is evaluated on IEMOCAP, AVEC, and MELD using textual information only, excluding multimodal emotion recognition.All three datasets contain textual, visual, and acoustic information, but multimodal recognition is left for future work.
- Datasets Used: IEMOCAP contains dyadic dialogues with six categorical emotion labels across ten speakers.Only the first eight speakers from sessions one through four belong to the training set.
- Datasets Used: AVEC contains human-agent interactions annotated with four real-valued affective attributes: valence, arousal, expectancy, and power.The attributes are defined over [−1,1] for the first three dimensions and [0,∞) for power.
- Datasets Used: MELD contains multiparty Friends dialogues with more than 1,400 dialogues and 13,000 utterances labeled across seven emotion classes.Each utterance also includes textual, acoustic, and visual information.
- Baselines and State of the Art: The evaluation compares DialogueGCN with context-independent CNN, memory-network Memnet, contextual c-LSTM variants, speaker-specific CMN and ICON, and state-of-the-art DialogueRNN baselines.CMN and ICON are limited to two-speaker conversations, whereas DialogueRNN can be applied to multiparty datasets.
5 Results and Discussions
DialogueGCN outperforms DialogueRNN and other baselines across IEMOCAP, AVEC, and MELD, while its speaker-level context modeling addresses limitations of sequential recurrent encoders. Ablations and qualitative analyses indicate that speaker-dependent relations and contextual neighborhoods are important for emotion recognition.
- Comparison with State of the Art and Baseline: DialogueGCN outperforms the state-of-the-art and all baseline models on every evaluated dataset, with statistical significance under a paired t-test (p <0.05).Results are averaged over 5 runs.
- Comparison with State of the Art and Baseline: 64.18% average F1-score and 65.25% accuracy on IEMOCAP are approximately 2% better than DialogueRNN and at least 5% better than other baselines.These are reported as new state-of-the-art results.
- Comparison with State of the Art and Baseline: DialogueGCN outperforms the state-of-the-art on all four AVEC emotion dimensions: valence, arousal, expectancy, and power.The supplied passage reports the comparison across all four dimensions without providing their individual metric values.
- Model Analysis: DialogueGCN’s neighborhood-based convolution is intended to mitigate the long-term information-propagation issues affecting recurrent speaker-level encoders in long conversations.IEMOCAP and AVEC contain many conversations exceeding 70 utterances, with average lengths of 50 and 72 utterances respectively.
- Comparison with State of the Art and Baseline: 58.10% F1 score on MELD exceeds DialogueRNN by more than 1%, although multiparty MELD is harder to model than the dyadic IEMOCAP and AVEC datasets.Short utterances, many participants, and sparse speaker turns make inter- and self-dependency modeling difficult in MELD.
- Ablation Study: Removing both sequential and speaker-level context encoders reduces IEMOCAP F1 to 36.7%, while the speaker-level encoder is slightly more important overall.The ablation is presented as evidence for the importance of contextual modeling.
6 Conclusion
DialogueGCN models inter- and self-party dependency to improve context understanding for utterance-level emotion detection. Across three benchmark ERC datasets, it outperforms strong baselines and existing state of the art by a significant margin.
- DialogueGCN models inter- and self-party dependency to improve context understanding for utterance-level emotion detection in conversations.
- On three benchmark ERC datasets, DialogueGCN outperforms strong baselines and existing state of the art by a significant margin.
- Future work will incorporate multimodal information, detect speaker-level emotion shifts, and ground conversational emotion reasoning conceptually.
- The authors plan to use DialogueGCN in dialogue systems to generate affective responses.