Source-linked AI summary
Recognizing Emotion Cause in Conversations
Soujanya Poria, Navonil Majumder, Devamanyu Hazarika, Deepanway Ghosal, Rishabh Bhardwaj, Samson Yu Bai Jian, Pengfei Hong, Romila Ghosh, Abhinaba Roy, Niyati Chhaya, Alexander Gelbukh, Rada Mihalcea
TL;DR
The paper addresses the under-explored problem of identifying what causes emotions in conversations, which is difficult because conversational causes can be implicit and distributed across interlocutors. It introduces the RECCON task and dataset with two subtasks and Transformer-based baselines, which outperform several prior neural approaches, while the low subtask scores leave substantial room for improvement.
Problem
Identifying emotion causes in conversations is under-explored and difficult because causes may be implicit, distant, and dependent on interlocutors’ interactions and commonsense reasoning.
Method
The paper introduces RECCON, a dialogue-level dataset, and two subtasks—causal span extraction and causal emotion entailment—with Transformer-based baselines.
Results
The baselines surpass several newly introduced neural emotion-cause approaches, while context generally improves baseline results and Fold 1 is confirmed as the superior training dataset.
Takeaways & Limitations
Recognizing emotion causes in conversations is presented as a challenging task relevant to interpretable and explainable emotion-aware artificial intelligence.
Takeaways & Limitations
Low scores in the two subtasks indicate substantial room for model improvement.
Abstract
from arXiv · showhide
We address the problem of recognizing emotion cause in conversations, define two novel sub-tasks of this problem, and provide a corresponding dialogue-level dataset, along with strong Transformer-based baselines. The dataset is available at https://github.com/declare-lab/RECCON. Introduction: Recognizing the cause behind emotions in text is a fundamental yet under-explored area of research in NLP. Advances in this area hold the potential to improve interpretability and performance in affect-based models. Identifying emotion causes at the utterance level in conversations is particularly challenging due to the intermingling dynamics among the interlocutors. Method: We introduce the task of Recognizing Emotion Cause in CONversations with an accompanying dataset named RECCON, containing over 1,000 dialogues and 10,000 utterance cause-effect pairs. Furthermore, we define different cause types based on the source of the causes, and establish strong Transformer-based baselines to address two different sub-tasks on this dataset: causal span extraction and causal emotion entailment. Result: Our Transformer-based baselines, which leverage contextual pre-trained embeddings, such as RoBERTa, outperform the state-of-the-art emotion cause extraction approaches Conclusion: We introduce a new task highly relevant for (explainable) emotion-aware artificial intelligence: recognizing emotion cause in conversations, provide a new highly challenging publicly available dialogue-level dataset for this task, and give strong baseline results on this dataset.
1 Introduction
The paper introduces recognizing emotion cause in conversations as an under-explored NLP task focused on identifying stimuli behind emotions in conversational utterances. It presents cause types, the RECCON dataset, and two challenging subtasks with strong baselines.
- Motivation: Emotion-cause reasoning extends emotion recognition by identifying the events, situations, opinions, or experiences responsible for an emotion in context.Causes may also involve interpersonal emotional influence from another speaker’s reaction.
- Contributions: The paper introduces RECCON, a new task for extracting stimuli behind emotions in conversational utterances, including causes in the same or contextual utterances.The task is presented as one of the first attempts to recognize emotion causes specifically in conversations.
- Motivation: Conversational causes can be implicit, distant from the target utterance, or intertwined with interlocutors’ reactions, requiring complex reasoning.The examples include shared events and another speaker’s emotional response as alternative stimuli.
- Contributions: The authors define relevant emotion-cause types and provide a new annotated dataset containing acted and real-world dyadic conversations.They state that no other dataset for this conversational task was known to them.
- Contributions: The paper introduces two challenging subtasks and strong baselines that surpass several recently introduced neural emotion-cause approaches.The cited examples include ECPE-MLL, RankCP, and ECPE-2D.
2 Related Work
Earlier emotion-cause research largely used rule-based or corpus-based approaches, while related work expanded toward jointly identifying emotions and causes. RECCON extends this line to more difficult dyadic conversations with implicit, distant, and commonsense-dependent causes.
- Research context: The related literature spans emotion analysis, emotion-cause extraction, emotion-cause pair extraction, and conditional emotion-cause identification.These strands provide the task lineage that RECCON extends to conversations.
- Prior emotion-cause extraction: Early emotion-cause extraction used rule-based approaches and datasets built from news articles to reduce implicit-cause and informal-language complexity.The news corpus was designed to simplify annotation by avoiding latent emotions and implicit causes associated with informal text.
- Related task formulations: Emotion-cause pair extraction jointly identifies emotions and their corresponding causes, while conditional pair identification restricts causal validity to particular contexts.RECCON incorporates contextual validity by annotating multiple spans in conversational history that sufficiently indicate a cause.
- RECCON’s distinction: RECCON extends prior work with a more difficult dyadic-conversation dataset and a task involving implicit emotion, informal connections, distant causes, co-reference, and commonsense reasoning.These challenges make conversational causes harder to annotate and solve than causes in earlier settings.
3 Definition of the Task
The paper distinguishes emotion evidence from emotion cause and operationalizes the task as identifying causal spans in a conversation’s history for a target utterance. It represents correct target–span combinations as UCS pairs, with other combinations treated as negatives during training.
- Core distinction: Emotion evidence indicates that a speaker is emotional, whereas emotion cause expresses why the speaker feels that emotion.Evidence operates between text and reader, while cause operates between described circumstances and the described emotional state.
- Core distinction: A text span can simultaneously serve as emotion evidence and cause, although the paper states that this is generally not the case.The distinction therefore separates recognizing an emotion from identifying its reason.
- Scope of the definition: The paper explains the notion of emotion cause through examples and a labeled dataset rather than relying solely on a short formal definition.It presents defining the notion itself as a central goal of the paper.
- Operationalization: A causal span is a maximal substring from the target utterance’s conversational history that forms part of its emotion cause.The target utterance has a known emotion label, and its history includes all utterances from the conversation’s beginning through the target.
- Operationalization: Recognizing emotion cause means identifying all correct UCS pairs in a text, where each pair links an utterance with a causal span.The paper treats correct pairs as positive examples and pairs with non-causal spans as negative examples.
4 Building the RECCON dataset
RECCON is built from selected IEMOCAP and DailyDialog conversations, annotated for causal spans under a multi-span framework and aggregated through utterance- and span-level agreement procedures.
- Emotional Dialogue Sources: RECCON combines subsets from IEMOCAP and DailyDialog, forming RECCON-IE and RECCON-DD with different emotion label sets.IEMOCAP contributes longer, more emotionally shifting dialogues, while DailyDialog sampling avoids its strong neutral-label skew.
- Annotation Process: Annotators identified causal spans for labeled target utterances, including inferred causes when no explicit causal span was present.They were instructed to search the dialogue, but the final dataset retained only causes in the target utterance’s history.
- Annotation Process: The annotation framework distinguishes emotion causes from reactionary emotional expressions and supports multiple controlling variables through a multi-span formulation.This design follows span-based emotion-cause extraction while extending it to conversational settings.
- Annotation Aggregation: Annotations were aggregated first by utterance-level majority voting and then by span-level overlap, with a third annotator resolving disagreements.Span unions were accepted when intersection size reached at least 50% of the shortest candidate span.
- Annotation Aggregation: 34 causal utterances were discarded because the third annotator could not resolve the span-level tie.The aggregation procedure preferred shorter spans when they preserved the causal information.
- Dataset Statistics: Inter-annotator agreement reached 0.7928 kappa and 0.8839 macro F1 at utterance level, 0.8201 macro F1 after removing negatives, and 0.8035 F1 at span level.The lower negative-excluded macro F1 reflects the authors’ concern that frequent non-causal utterances can skew span-extraction agreement.
5 Types of Emotion Causes
RECCON categorizes emotion causes by where the stimulus originates in the conversational context, including within the target utterance, across speakers, within a speaker’s prior mood, or outside explicit context.
- Cause Taxonomy: RECCON identifies five predominant cause types based on the source of stimuli responsible for the target emotion.The categories organize explicit, conversational, self-contagious, hybrid, and latent causes.
- No Context: No Context denotes a cause explicitly present within the target utterance itself.The emotion-bearing speaker directly mentions the cause.
- Inter-Personal Emotional Influence: Inter-Personal Emotional Influence occurs when another speaker’s utterance supplies the cause through a trigger event or emotional dependency.The target speaker may respond to the other speaker’s event-related content or emotion.
- Self-Contagion: Self-Contagion describes emotions driven primarily by a stable mood induced in the same speaker by previous dialogue turns.A mood established earlier can persist across later utterances.
- Hybrid: Hybrid causes jointly combine inter-personal emotional influence and self-contagion.Both type 2 and type 3 mechanisms can contribute to one utterance’s emotion.
- Unmentioned Latent Cause: Unmentioned Latent Cause applies when no explicit causal span appears in the target utterance or conversational history.Some such causes may be revealed only in future utterances, which is outside the retained historical context.
6 Experiments
The experiments construct positive and negative UCS examples and evaluate causal span extraction with context-aware and context-free MRC formulations using Transformer baselines.
- 6.1.1 Generating Negative Examples: The dataset contains positive causal UCS pairs, so three strategies construct negative examples for model training and analysis.Fold 1 samples non-causal utterances from the same dialogue history; Folds 2 and 3 sample across dialogues, with Fold 3 matching emotions.
- 6.1.1 Generating Negative Examples: Fold 1 pairs each target utterance with every historical utterance that is not causal, while Folds 2 and 3 sample non-causal utterances from other dialogues.Fold 3 additionally requires the sampled non-causal emotion to match the target emotion.
- 6.2 Subtask 1: Causal Span Extraction: Causal Span Extraction identifies the emotion-cause span for a non-neutral target utterance and is formulated as a Machine Reading Comprehension task.Two settings are evaluated: with conversational context and without conversational context.
- With Conversational Context: With conversational context, the model receives the historical dialogue as context, a target-and-evidence question, and the causal span as the answer.Negative pairs receive an empty answer; multiple causal spans generate separate instances.
- Baseline Models: Two pretrained Transformer baselines benchmark span extraction: RoBERTa Base and SpanBERT fine-tuned on SQuAD 2.0.RoBERTa predicts span start and end logits, while SpanBERT uses a model pretrained with contiguous-span masking and fine-tuned for question answering.
- 6.2.3 Evaluation Metrics: Evaluation reports Exact Match, positive-example F1, negative-example F1, and overall F1 across RECCON-DD and RECCON-IE test sets.The reported Table 4 scores are percentages selected at the best validation F1.
6.3 Subtask 2: Causal Emotion Entailment
Causal Emotion Entailment predicts which conversational-history utterances cause a target utterance’s non-neutral emotion, with and without context. Results show that context helps, while dataset and cause structure strongly affect performance.
- Task formulation: Causal Emotion Entailment identifies historical utterances responsible for a target non-neutral emotion, formulated with or without conversational context.With context, the task is triplet classification; without context, it is sentence-pair classification.
- Models: RoBERTa-based classifiers use the target emotion label and separated utterance or context inputs, while ECPE baselines model utterance relationships differently.The compared baselines include RoBERTa Base/Large, ECPE-2D, ECPE-MLL, and RankCP.
- Results: SpanBERT achieves the best causal span extraction performance on both RECCON-DD and RECCON-IE, with context consistently improving its RECCON-DD results over RoBERTa Base.The comparison covers positive, negative, and overall F1-related metrics under contextual settings.
- Results: RoBERTa Large outperforms RoBERTa Base for causal emotion entailment with context, whereas fold-specific results vary between datasets without context.In Fold 1 without context, RoBERTa Base leads by 2% on RECCON-DD, while RoBERTa Large leads by 5.5% on RECCON-IE.
- Results: Models perform better on utterances with one cause than multiple causes, with an average gap of +7% across settings and models.Contextual information substantially improves the non-contextual counterpart, and RECCON-IE is generally harder than RECCON-DD.
- Discussion: Low scores indicate that both subtasks remain difficult and leave substantial room for model improvement.The paper also reports that complex ECPE baselines do not outperform the simpler RoBERTa baselines.
7 Further Analysis and Discussion
Further analysis shows that context helps distinguish constructed negative examples, but training and testing across fold types exposes a major mismatch in negative-sample difficulty. Fold 1 is harder and more representative of within-dialogue causal reasoning.
- Fold analysis: Context improves results on Folds 2 and 3 by exposing contextual discrepancy or coherence in randomly generated negative samples.The authors caution that this improvement does not establish Folds 2 and 3 as better training datasets than Fold 1.
- Fold analysis: Models trained on Folds 2 and 3 perform abysmally on Fold 1 negative examples, reflecting the differing difficulty of the fold constructions.Folds 2 and 3 contain easier negatives detectable through contextual incoherence, whereas Fold 1 negatives remain contextually coherent.
- Fold analysis: Fold 1-trained models show no significant performance drop when evaluated on Folds 2 and 3, supporting Fold 1 as the stronger training dataset.The asymmetry indicates that Fold 1 captures a harder generalization setting.
- Discussion: Fold 1 is the more challenging and practical setting because real applications require identifying causes within a single dialogue.Its negatives are coherent with the target context, requiring more sophisticated reasoning than detecting incoherence.
8 Challenges of the Task
Emotion-cause recognition in conversations requires reasoning over multiple spans, turns, speakers, and implicit relations. The paper highlights temporal, numerical, emotional, commonsense, and co-reference challenges, while noting that its annotations do not encode full reasoning procedures.
- Multiple spans: Causal span extraction may require multiple spans whose combined coverage supports logical reasoning, including numerical reasoning.The TV-price example requires connecting a final-price phrase with the earlier $2000-versus-$2500 discussion.
- Emotional dynamics: Causes can depend on long-term emotional dynamics across turns, including interpersonal influence, shared causes, and propagated emotions.Contextual emotions help annotators interpret greetings, conflicts, agreements, empathy, and multi-turn causal chains.
- Commonsense knowledge: Commonsense knowledge is essential when conversational causes or emotions are implicit rather than explicitly stated.The help-desk example requires inferring happiness from offering help and fear from a broken computer.
- Co-reference: Dialogue co-reference is difficult because causal phrases may refer to entities or properties introduced by another speaker.The watch example requires linking “flashy red lines” with the counterpart’s purchased watch.
- Temporal reasoning: Temporal reasoning can connect a gift, its description, and a speaker’s reaction into one explanation for happiness.The birthday-present, gold-watch, and “I love it!” spans form a multi-turn causal sequence.
- Cause representation: Informal dialogue sometimes requires annotators to label perceived causes that can be rephrased as the exact underlying cause.For example, wanting a flip phone to work represents the exact cause that the phone is not working.
- Annotation scope: The dataset records causes but does not ask annotators to explain the reasoning steps used to identify them.The authors suggest future extensions with reasoning procedures, while noting that designing annotation instructions is difficult.
9 Connection to Interpretability of the Contextual Models
Emotion-cause identification can test whether contextual emotion models rely on the utterances that actually evoke a target emotion. It also distinguishes cases where causes occur in context from cases where the target utterance itself contains causal cues.
- Causal utterances can be removed to probe whether attention-based contextual models rely on them for emotion prediction.This is especially relevant when an utterance appears neutral but its emotion is inferred from context.
- When the cause occurs in the target utterance, attention scores can be inspected for causal spans without requiring contextual information.
- A model may predict an emotion correctly without identifying its cause, so prediction accuracy alone does not establish causal understanding.
- Cause identification can therefore make conversational emotion models more trustworthy, interpretable, and explainable.
10 Conclusion
The paper addresses emotion-cause recognition in conversations through the RECCON dialogue-level dataset and Transformer-based baselines. It reports a dyadic benchmark while identifying multi-party, reasoning-annotation, and multimodal extensions for future work.
- RECCON contains more than 1,126 dyadic dialogues and 10,600 utterance causal span pairs.
- The paper proposes two subtasks and Transformer-based strong baselines for recognizing emotion cause in conversations.
- Future work targets multi-party conversations, annotated reasoning steps, and multimodal extensions of the data and benchmark algorithms.
Compliance with Ethical Standards
The authors report no studies involving human participants or animals and disclose no financial or non-financial conflicts related to the manuscript.
- The article contains no studies with human participants or animals performed by the authors.
- The authors certify that they have no affiliations or involvement with organizations holding financial interests in the subject matter.
- The authors also certify that they have no relevant non-financial interests in the manuscript’s subject matter or materials.