Source-linked AI summary
DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations
Dou Hu, Lingwei Wei, Xiaoyong Huai
TL;DR
ERC methods have struggled to extract and integrate the rich emotional clues in conversational context. DialogueCRN addresses this gap with cognitive, multi-turn reasoning that combines intuitive retrieval and conscious reasoning. Across three benchmark datasets, it consistently outperforms comparison methods, with reported gains on IEMOCAP and SEMAINE.
Problem
Existing ERC approaches inadequately extract and integrate emotional clues from conversational context, limiting their contextual understanding.
Method
DialogueCRN uses perceptive context modeling and multi-turn cognitive reasoning that iteratively retrieves and logically integrates emotional clues.
Results
DialogueCRN consistently outperforms comparison methods on three public benchmark datasets, including 3.2% relative Acc. improvement on IEMOCAP and 11.1% MAE on SEMAINE Arousal.
Takeaways & Limitations
The experiments support cognitive reasoning over emotional clues as an effective approach for emotion recognition in conversations.
Abstract
from arXiv · showhide
Emotion Recognition in Conversations (ERC) has gained increasing attention for developing empathetic machines. Recently, many approaches have been devoted to perceiving conversational context by deep learning models. However, these approaches are insufficient in understanding the context due to lacking the ability to extract and integrate emotional clues. In this work, we propose novel Contextual Reasoning Networks (DialogueCRN) to fully understand the conversational context from a cognitive perspective. Inspired by the Cognitive Theory of Emotion, we design multi-turn reasoning modules to extract and integrate emotional clues. The reasoning module iteratively performs an intuitive retrieving process and a conscious reasoning process, which imitates human unique cognitive thinking. Extensive experiments on three public benchmark datasets demonstrate the effectiveness and superiority of the proposed model.
1 Introduction
DialogueCRN addresses limitations in contextual emotion recognition by reasoning over emotional clues through iterative cognitive processes. Experiments on three public benchmark datasets demonstrate the model’s effectiveness and superiority.
- Research gap: Existing ERC methods capture situation- or speaker-level context but inadequately extract and integrate its rich emotional clues.Static-memory retrieval limits clue extraction, while attention-based integration can ignore semantic order and logical relationships.
- Approach: DialogueCRN models conversational context from a cognitive perspective using a perceptive phase and a cognitive phase.The perceptive phase captures situation-level and speaker-level context; the cognitive phase reasons over retrieved context.
- Approach: Multi-turn reasoning modules iteratively perform intuitive retrieval and conscious reasoning to extract and integrate emotional clues.Intuitive retrieval matches clues in static global memories, while conscious reasoning uses dynamic working memory to learn their logical order.
- Evaluation: DialogueCRN consistently outperforms comparison methods on IEMOCAP, SEMAINE, and MELD in extensive benchmark experiments.The results are reported as demonstrating the model’s effectiveness and superiority.
- Contribution: The work presents cognitive reasoning over emotional clues as a novel direction for fully understanding conversational context in ERC.The authors identify this as the first attempt to explore cognitive factors for emotion recognition in conversations.
2 Methodology
DialogueCRN models conversational context through perception and cognition phases before predicting each utterance’s emotion. It combines situation- and speaker-level context with multi-turn reasoning that retrieves and integrates emotional clues.
- Problem Statement: The task predicts a predefined emotion label for each utterance in a multi-speaker conversation.A conversation is represented as utterances associated with their corresponding speakers.
- Overall Architecture: DialogueCRN comprises perception, cognition, and emotion-classification components.The perception phase produces contextual representations and global memories, while the classifier predicts the utterance emotion.
- Perception Phase: Bidirectional LSTMs capture sequential situation-level dependencies and self-dependencies among utterances from the same speaker.The resulting context representations encode conversational and speaker-specific information.
- Perception Phase: Linear transformations convert situation- and speaker-level context representations into global memories storing different contextual information.These memories provide the contextual information used by the cognition phase.
- Cognition Phase: Multi-turn cognition alternates attention-based retrieval from global memory with LSTM-based conscious reasoning over dynamic working memory.Queries are initialized from the current utterance context, while working memory retains and updates prior information to integrate clues in logical order.
- Cognition Phase: Separate cognition phases process situation-level and speaker-level clues for T s and T v turns before their outputs are concatenated for classification.The two cognition outputs form the final representation, and cross-entropy loss trains the classifier.
3 Experimental Setups
The experiments evaluate DialogueCRN on three benchmark datasets for textual conversation emotion recognition, using established comparison methods and dataset-specific metrics. Training uses validation-based hyperparameter tuning and reported implementations of selected baselines.
- 3.1 Datasets: IEMOCAP, SEMAINE, and MELD are used as benchmark datasets, with the study focusing on textual conversations despite their multimodal origins.Multimodal emotion recognition is left for future work.
- 3.2 Comparison Methods: The comparison includes context-independent, memory-based, recurrent, attention-based, and speaker-aware conversational models such as TextCNN, Memnet, bc-LSTM+Att, CMN, ICON, and DialogueRNN.These baselines represent different approaches to modeling utterance and conversational context.
- 3.3 Evaluation Metrics: IEMOCAP and MELD are evaluated with accuracy, Weighted-F1, and Macro-F1, while SEMAINE uses Mean Absolute Error for each affective attribute.Lower MAE indicates better detection performance.
- 3.4 Implementation Details: Hyperparameters are tuned on validation sets, with Adam optimization, dropout, early stopping, and dataset-specific bidirectional LSTM configurations.The perceptive phase uses two-layer bidirectional LSTMs for IEMOCAP and SEMAINE and a single-layer version for MELD; the cognitive phase uses a single-layer LSTM.
- 3.4 Implementation Details: DialogueGCN and DialogueRNN results are implemented from public code under the same environment.This provides a shared implementation setting for those comparisons.
4 Results and Analysis
DialogueCRN consistently outperforms comparison methods across IEMOCAP, SEMAINE, and MELD, with statistically significant gains. Ablations and turn analyses show that both perceptive and cognitive phases, their contextual inputs, and multi-turn reasoning contribute to performance.
- Experimental Results: DialogueCRN consistently outperforms comparison methods on all datasets, with statistically significant differences under paired t-tests (p<0.05).
- Experimental Results: 3.2%, 4.0%, and 4.7% relative gains over previous best baselines occur on IEMOCAP for Acc., Weighted-F1, and Macro-F1, respectively.
- Experimental Results: 11.1% MAE improvement on SEMAINE’s Arousal attribute demonstrates a large margin over prior baselines.
- Experimental Results: 2.9% Weighted-F1 improvement on MELD gives DialogueCRN the best recognition results despite the dataset’s shorter conversations and limited contextual information.
- Ablation Study: Removing cognition and perception modules successively greatly reduces performance, indicating that both phases are important for ERC.
- Ablation Study: Removing the cognitive phase decreases IEMOCAP Acc., Weighted-F1, and Macro-F1 by 4.3%, 4.3%, and 6.5%, while increasing SEMAINE Valence, Arousal, and Expectancy MAE by 2.3%, 12.5%, and 2.9%.
- Ablation Study: Removing either situation-level or speaker-level context lowers performance, with speaker context more useful in perception and situation context more useful in cognition.
- Turn Analysis: The best turn settings are {2, 2} on IEMOCAP and {1, 3} on SEMAINE, yielding 66.20% Weighted-F1 and 0.1522 MAE for Arousal, respectively.SEMAINE requires more speaker-level cognitive turns, suggesting speaker-level contextual clues may matter more for arousal emotion.
5 Related Work
Related ERC work captures conversational context through sequence-based or graph-based deep learning methods. DialogueCRN differs by introducing cognitive factors to extract and integrate emotional clues in a cognitive manner.
- Background: ERC differs from conventional emotion recognition because both situation-level and speaker-level context significantly affect utterance emotion identification.
- Sequence-based Methods: Sequence-based methods use LSTMs, memory networks, or transformers to capture contextual features from utterance sequences.
- Graph-based Methods: Graph-based methods design conversational graph structures and apply graph neural networks to capture multiple dependencies.
- Cognitive Reasoning: DialogueCRN makes the first attempt to explore cognitive factors for ERC by extracting and then integrating rich emotional clues cognitively.
6 Conclusion
The paper concludes that DialogueCRN uses cognitive reasoning to understand contextual emotional clues and improves ERC performance across three benchmark datasets. Its case study further illustrates how cognitive factors support emotional-clue understanding.
- Conclusion: DialogueCRN models situation-level and speaker-level context through perceptive and cognitive phases with iterative intuitive retrieval and conscious reasoning.
- Conclusion: Experiments on three benchmark datasets demonstrate the effectiveness and superiority of DialogueCRN.
- Conclusion: The case study shows that considering cognitive factors can better understand emotional clues and boost ERC performance.