Source-linked AI summary
SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model
Zaijing Li, Fengxiao Tang, Tieyu Sun, Yusen Zhu, Ming Zhao
TL;DR
Conversation emotion recognition methods have emphasized speaker relationships while inadequately modeling utterance emotional tendency and correlations between sentences. The paper proposes SEOV and the SEOVER model, which extract emotion-oriented vectors and jointly use them with dialogue sentiment analysis. Experiments on MELD and IEMOCAP report better performance than baseline models, while error analysis identifies reduced accuracy for adjacent emotions as a limitation.
Problem
Existing conversation emotion recognition methods focus on speaker relationships but do not fully extract emotional tendency correlations between utterances, especially during sudden emotion turns.
Method
SEOVER extracts sentence-level emotion orientation vectors with transformer-emo and jointly uses them with a dialogue sentiment analysis model for speaker emotion classification.
Results
SEOVER outperforms the compared baseline models on the MELD and IEMOCAP datasets.
Takeaways & Limitations
SEOV provides a representation that encodes emotional intensity through vector size and emotion tendency through vector direction.
Takeaways & Limitations
Emotion-vector fusion improves flipped-emotion classification but slightly reduces classification accuracy for adjacent emotions such as sad and angry.
Abstract
from arXiv · showhide
For the task of conversation emotion recognition, recent works focus on speaker relationship modeling but ignore the role of utterance's emotional tendency.In this paper, we propose a new expression paradigm of sentence-level emotion orientation vector to model the potential correlation of emotions between sentence vectors. Based on it, we design an emotion recognition model, which extracts the sentence-level emotion orientation vectors from the language model and jointly learns from the dialogue sentiment analysis model and extracted sentence-level emotion orientation vectors to identify the speaker's emotional orientation during the conversation. We conduct experiments on two benchmark datasets and compare them with the five baseline models.The experimental results show that our model has better performance on all data sets.
1 Introduction
Conversation emotion recognition seeks to identify speakers’ emotions, but existing methods emphasize speaker relationships and may inadequately represent utterance emotional tendency. The paper proposes SEOV and SEOVER to model emotion intensity and cross-sentence emotional direction for recognition.
- Research gap: Existing conversation emotion recognition work focuses mainly on speaker identification and relationship modeling, while BERT-based methods do not fully capture emotional correlations during sudden emotion turns.Earlier CNN encodings also inadequately express grammatical and semantic features, motivating richer utterance representations.
- Research gap: CNN and BERT produce opposite feature-map trends for an example MELD utterance, suggesting that conventional encodings may lose features needed to represent emotional tendency.The comparison uses normalized 600-dimensional sentence vectors and a hot-zone visualization.
- Proposed representation: SEOV represents emotional intensity through vector size and emotional tendency between sentence vectors through vector direction.Figure 2 depicts correlations among emotion vectors as the direction of SEOV.
- Contribution: SEOVER combines sentence-level emotion orientation vectors with dialogue sentiment analysis to classify each speaker’s emotion.The model extracts SEOVs with transformer-emo and jointly uses them with contextual dialogue modeling.
2 Methodology
The methodology introduces SEOV to represent sentence emotion through vector size and direction, then combines it with dialogue context modeling for speaker emotion classification.
- Model Architecture: SEOVER extracts sentence vectors with transformer-emo, maps them into emotion vectors, and jointly processes SEOVs with a dialogue sentiment analysis model.The architecture contains sentence-level encoding, emotion-level encoding, and context modeling stages.
- Sentence-level Encoder: Transformer-emo splits each conversation into sentences and maps them to sentence vectors Q.Each sentence vector has dimensionality d, set to 786 in the experiment.
- Emotion-level Encoder: SEOV represents emotional intensity through vector size and emotion tendency through vector direction.The method treats sentence semantic information and classification attributes as components of this representation.
- Emotion-level Encoder: The emotion vector q* is obtained by mapping q into a k*-dimensional space, where its elements represent sentence-level emotion probabilities.When k* equals the number of emotion categories, q* can be interpreted as the classification result.
- Context Modeling: The model concatenates q and q* into e, then reassembles SEOVs and feeds them to DSAM for contextual speaker-state modeling.DialogueRNN, DialogueGCN, and bc-LSTM are used as benchmark dialogue emotion analysis models.
3 Experimental Setting
The experiments use the IEMOCAP and MELD conversation emotion datasets and compare the proposed approach with established conversation emotion recognition baselines.
- Datasets: IEMOCAP contains multimodal conversations from ten actors labeled with six emotions: happy, sad, neutral, angry, excited, and frustrated.
- Datasets: MELD contains 1,432 conversations from Friends and 13,708 sentences with video, text, voice, and other data.
- State-of-the-art Baselines: The baseline set includes DialogueRNN, DialogueGCN, DialogXL, bc-LSTM, TRMSM, and BERT-based classification.These methods model speaker relationships, graph structure, long-term context, recurrent context, transformer-based relations, or pretrained language representations.
4 Results and Analysis
SEOVER outperforms the compared baseline models on IEMOCAP and MELD, while ablation and error analyses examine the role and limits of emotion-vector fusion.
- 4.1 Comparison with baseline models: SEOVER outperforms the compared baseline models on the IEMOCAP and MELD datasets.Table 1 reports F1-score results for SEOVER variants using DialogueRNN, DialogueGCN, and bc-LSTM as fine-tuning models.
- 4.1 Comparison with baseline models: The model’s stronger performance is attributed to transformer-emo’s richer syntactic and semantic representations and SEOV’s mapping of emotion orientations between sentence vectors.
- 4.2 Ablation Study: Removing emotion vectors reduces weighted accuracy and F1 on MELD compared with DialogueRNN, highlighting the contribution of emotion-tendency encoding.The ablation results are reported in Table 2.
- 4.3 Error Analysis: On MELD, emotion-vector fusion improves classification of flipped emotions but slightly reduces classification accuracy for adjacent emotions such as sad and angry.The confusion matrix is reported for the test set with DialogueRNN as the benchmark model.
5 Conclusion
The paper introduces SEOV to address information loss in conventional CER discourse representations and builds SEOVER around it. Experiments on MELD and IEMOCAP show that SEOVER outperforms state-of-the-art methods.
- 5 Conclusion: SEOV represents sentence-vector emotion intensity through size and emotion tendency through direction, addressing discourse-representation information loss in CER.
- 5 Conclusion: SEOVER uses Transformer-emo to encode sentence and emotion vectors, fuses them as SEOV, and feeds the result into a dialogue emotion analysis model.
- 5 Conclusion: Comparative experiments on MELD and IEMOCAP show that SEOVER outperforms the state-of-the-art methods.