Source-linked AI summary
Directed Acyclic Graph Network for Conversational Emotion Recognition
Weizhou Shen, Siyue Wu, Yunyi Yang, Xiaojun Quan
TL;DR
ERC requires modeling conversational context because an utterance’s emotion may depend on surrounding and same-speaker history. This paper represents conversations as DAGs and develops DAG-ERC to aggregate nearby and remote predecessor information. Experiments and ablations across four ERC benchmarks support the proposed architecture and its modules, while errors remain for similar emotions and majority-class neutral samples.
Problem
ERC emotion prediction depends on conversational context, but graph-based and recurrence-based approaches have complementary limitations in capturing distant, sequential, and nearby information.
Method
The paper builds conversation DAGs using speaker-identity and positional constraints and proposes DAG-ERC with predecessor aggregation, relation-aware transformation, and a contextual information unit.
Results
Extensive experiments and ablations on four ERC benchmarks support DAG-ERC’s performance, the proposed DAG structure, and the impact of its modules.
Takeaways & Limitations
DAG structures with speaker-identity and positional constraints model ERC context effectively, and the network is reported to be less prone to over-smoothing as layers increase.
Takeaways & Limitations
DAG-ERC struggles to distinguish similar emotions and tends to misclassify other emotions as neutral in several datasets with many neutral samples.
Abstract
from arXiv · showhide
The modeling of conversational context plays a vital role in emotion recognition from conversation (ERC). In this paper, we put forward a novel idea of encoding the utterances with a directed acyclic graph (DAG) to better model the intrinsic structure within a conversation, and design a directed acyclic neural network, namely DAG-ERC, to implement this idea. In an attempt to combine the strengths of conventional graph-based neural models and recurrence-based neural models, DAG-ERC provides a more intuitive way to model the information flow between long-distance conversation background and nearby context. Extensive experiments are conducted on four ERC benchmarks with state-of-the-art models employed as baselines for comparison. The empirical results demonstrate the superiority of this new model and confirm the motivation of the directed acyclic graph architecture for ERC.
1 Introduction
ERC predicts each utterance’s emotion, making conversational context central because emotions can depend on same-speaker utterances and surrounding context. The paper proposes representing conversations as DAGs and introduces DAG-ERC to combine nearby and remote information.
- ERC identifies the emotion of each utterance and supports applications such as social-media opinion mining and empathetic dialogue systems.
- Conversational context is crucial because a query utterance’s emotion can depend on same-speaker utterances and surrounding context, especially when the query is short.
- Graph-based methods capture surrounding utterances within a window but neglect distant and sequential information, whereas recurrence-based methods use limited nearest-utterance information.
- A conversation modeled as a DAG propagates information from predecessors to successors, allowing a query utterance to gather both neighboring and remote context.
- DAG-ERC builds DAGs using speaker-identity and positional constraints, recurrently aggregates predecessors within one layer, and adds relation-aware transformation plus a contextual information unit.
- The paper’s contributions include treating conversations as DAGs for ERC, constructing constrained DAGs, and proposing a DAG neural network with two ERC-specific improvements.
2 Related work
Prior ERC work models conversational context mainly with graph-based or recurrence-based methods. Related DAG architectures provide a foundation for DAG-ERC, while differing in layer structure and aggregation mechanisms.
- Graph-based Models: Graph-based ERC models connect utterances to surrounding context, with some methods adding positional encodings or treating attention as a fully connected graph.
- Recurrence-based Models: Recurrence-based ERC models use GRUs, memory networks, or multiple recurrent networks to encode utterances, conversations, dialog dynamics, and commonsense knowledge.
- Directed Acyclic Graph Neural Network: DAGs are used in areas including source-code parsing and logical formulas, and prior neural architectures include Tree-LSTM, DAG-RNN, D-VAE, and DAGNN.
- Directed Acyclic Graph Neural Network: DAGNN supports stacked layers and graph-attention aggregation rather than only summation or element-wise multiplication of predecessor representations.
3 Methodology
DAG-ERC represents each conversation as a directed acyclic graph whose edges encode speaker identity and positional relations, then processes information in temporal order. Its layers combine graph-style aggregation with recurrent propagation and add contextual and relation-aware mechanisms for ERC.
- 3.2 Building a DAG from a Conversation: Each utterance is represented as a graph node, with directed edges carrying relation types that distinguish same-speaker from different-speaker connections.The graph is G = (V, E, R), with relation type 1 for same-speaker utterances and 0 otherwise.
- 3.2 Building a DAG from a Conversation: The DAG construction forbids backward edges, links each utterance to a previous same-speaker cutoff utterance, and connects intervening utterances as local context.The cutoff is the ω-th latest utterance by the same speaker; earlier information is treated as remote and intervening information as local.
- 3.2 Building a DAG from a Conversation: The resulting graph separates local and remote information while avoiding edges from future utterances to previous utterances.Figure 2 illustrates the distinction with solid local-information edges and dashed remote-information edges.
- 3.3.4 Training and Prediction: The model concatenates an utterance’s hidden states across DAG-ERC layers, feeds the representation to a feed-forward network, and trains with cross-entropy loss.RoBERTa-derived utterance features are used as inputs, while the final network predicts each utterance’s emotion.
- 3.3.2 GNN, RNN and DAGNN: DAGNN combines GNN-style neighbor aggregation with RNN-style temporal propagation, allowing information from distant utterances and multiple neighbors to update states at the same layer.DAG-ERC computes hidden states recurrently from the first utterance to the last at each layer.
- 3.3.3 DAG-ERC Layers: DAG-ERC adds a contextual information unit because nodal propagation alone does not fully exploit context when an utterance’s emotion depends on surrounding utterances.It also uses attention over predecessors and relation-aware feature transformation before combining aggregated information with the previous-layer utterance state.
4 Experimental Settings
DAG-ERC is evaluated on four ERC datasets using textual inputs and dataset-specific F1 metrics. The experiments compare it with recurrence-based, graph-based, feature-extraction, and prior extracted-feature baselines, with hyperparameters selected using validation data.
- 4 Experimental Settings: Hyperparameters are selected by hold-out validation for each dataset, including learning rate, batch size, dropout rate, and the number of DAG-ERC layers.The default comparison uses ω = 1; hidden vectors have size 300 and RoBERTa features have size 1024.
- 4 Experimental Settings: The study evaluates DAG-ERC on four ERC datasets and reports their statistics in Table 1.The datasets include IEMOCAP and MELD, both described as multimodal ERC datasets, although only text is used here.
- 4 Experimental Settings: The experiments use only textual modality, with micro-averaged F1 excluding neutral for DailyDialog and weighted-average F1 for the other datasets.The metric choices follow prior ERC evaluation practice cited by the authors.
- 4 Experimental Settings: Baselines cover recurrence-based methods, graph-based methods, RoBERTa, and prior models using the authors’ extracted features, including DAGNN.The comparison set includes DialogueRNN, DialogRNN-RoBERTa, COSMIC without external knowledge, DialogurGCN, KET, DialogXL, RGAT, and related RoBERTa variants.
5 Results and Analysis
Across four ERC datasets, DAG-ERC achieves competitive performance and a new state of the art on IEMOCAP, DailyDialog, and EmoryNLP, while its benefits vary by dataset and context structure. Analyses attribute gains to the proposed DAG structure and model components, while identifying limitations in incoherent or emotionally shifting conversations.
- 5.1 Overall Performance: DAG-ERC achieves competitive performance across four datasets and reaches a new state of the art on IEMOCAP, DailyDialog, and EmoryNLP.
- 5.1 Overall Performance: Graph-based models generally outperform recurrence-based models on IEMOCAP, DailyDialog, and EmoryNLP when using the same feature extractor.
- 5.1 Overall Performance: On MELD, neither graph-based models nor DAG-ERC outperforms recurrence-based models because consecutive utterances can be incoherent in the television-derived data.
- 5.1 Overall Performance: DAG-ERC consistently outperforms improved RoBERTa-based graph models and DAGNN, supporting the DAG structure and its ERC-specific improvements.
- 5.2 Variants of DAG Structure: The proposed DAG structure performs best among tested structures, outperforming a common DAG with similar predecessor counts; increasing ω beyond 1 generally does not help.
- 5.3 Ablation Study: Removing relation-aware transformation causes sharp drops on IEMOCAP and DailyDialog but only slight drops on MELD and EmoryNLP, reflecting differences in speaker counts.
- 5.3 Ablation Study: Ablating either the nodal or contextual information unit causes similar, noncritical drops, while combining both yields further improvement.
- 5.4 Number of DAG-ERC Layers: RGAT suffers significant degradation beyond 6 layers, whereas DAGNN and DAG-ERC fluctuate within a narrow range as layers increase, indicating limited over-smoothing.
6 Conclusion
The paper concludes that DAG-ERC’s conversation-derived DAG structure and network design improve ERC modeling, while identifying persistent errors and directions for further investigation.
- DAG structures built with speaker-identity and positional constraints outperform alternative structures in DAG-ERC.
- The commonly used same-speaker graph relation is insufficient for multi-speaker conversations.
- DAG networks are less susceptible to over-smoothing than GNNs as the number of layers increases.
- Many DAG-ERC errors involve similar emotions, neutral samples, and emotional shifts, which remain open issues for future work.