PaperlayerView exact revision on arXiv

Source-linked AI summary

Knowledge Bridging for Empathetic Dialogue Generation

Qintong Li, Piji Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen

arXiv:2009.09708v3cs.CL

TL;DR

Empathetic dialogue systems must infer implicit emotions from limited histories, where history-response non-stopword overlap occurs in only 0.5% of samples. KEMP builds an external-knowledge emotional context graph and models emotional dependencies; it outperforms state-of-the-art methods in automatic and human evaluations.

  • Problem

    Empathetic dialogue requires inferring emotional cues beyond dialogue history, since history-response non-stopword overlap occurs in only 0.5% of samples.

    Source: Introduction · p. 1

  • Method

    KEMP enriches dialogue history with external knowledge in an emotional context graph, perceives context-emotion signals, and uses emotional cross-attention to model dependencies for response generation.

    Source: Overview · p. 3, Emotional context graph · p. 3

  • Results

    KEMP outperforms state-of-the-art methods in both automatic and human evaluations.

    Source: Conclusion and outlook · p. 7

  • Takeaways & Limitations

    The authors verify the effectiveness of KEMP's emotional context graph, emotional context encoder, and emotion-dependency decoder.

    Source: Conclusion and outlook · p. 7

    KEMP uses heuristic rules to construct its emotional context graph, which limits flexibility across different knowledge resources.

    Source: Conclusion and outlook · p. 7

Abstract

from arXiv · show

Lack of external knowledge makes empathetic dialogue systems difficult to perceive implicit emotions and learn emotional interactions from limited dialogue history. To address the above problems, we propose to leverage external knowledge, including commonsense knowledge and emotional lexical knowledge, to explicitly understand and express emotions in empathetic dialogue generation. We first enrich the dialogue history by jointly interacting with external knowledge and construct an emotional context graph. Then we learn emotional context representations from the knowledge-enriched emotional context graph and distill emotional signals, which are the prerequisites to predicate emotions expressed in responses. Finally, to generate the empathetic response, we propose an emotional cross-attention mechanism to learn the emotional dependencies from the emotional context graph. Extensive experiments conducted on a benchmark dataset verify the effectiveness of the proposed method. In addition, we find the performance of our method can be further improved by integrating with a pre-trained model that works orthogonally.

Introduction

The section frames label-free empathetic dialogue as difficult and argues that external knowledge can provide emotional cues, while emotional transitions also require modeling. KEMP combines a knowledge-enriched context graph, emotion perception, and dependency-aware decoding; experiments report effectiveness on EMPATHETICDIALOGUES.

Introduction

  • Prior work has addressed empathetic dialogue through multi-task learning, mixture-of-experts, emotion mimicry, and multi-resolution user feedback.Source: Introduction · p. 1
  • Only 0.5% of dialogue samples share non-stopwords between response and history, while 80.1% offer hints through external-knowledge paths.The authors infer that empathetic dialogue requires useful emotional knowledge beyond the dialogue history.Source: Introduction · p. 1
  • Listeners often mirror interlocutors' emotions, but complex transition patterns also occur, motivating modeling emotional dependencies between interlocutors.Source: Introduction · p. 2
  • KEMP integrates dialogue history with external knowledge in an emotional context graph, encodes it to perceive context emotions, and decodes responses by modeling emotion dependencies.A multi-task learning framework jointly optimizes the objectives.Source: Introduction · p. 2
  • KEMP demonstrates effectiveness on EMPATHETICDIALOGUES in both automatic and human evaluations.Source: Introduction · p. 2
  • KEMP is presented as an external-knowledge approach to perceiving and expressing implicit emotions without explicit emotion labels.The section describes label-free empathetic dialogue as a challenge for chatbots.Source: Introduction · pp. 1–2

Related work

The related work distinguishes emotional-response generation conditioned on explicit labels from empathetic dialogue, which does not require selecting an emotion type explicitly. It also situates the study among empathetic systems and emotion-labelled dialogue corpora.

Emotional dialogue generation

  • Emotional dialogue approaches condition responses on manually specified labels, while real-world listeners infer speakers' emotions (Rashkin et al. 2019).The section describes prior methods for controlling response emotion, including explicit labels and affect-promoting terms.Source: Emotional dialogue generation · p. 2

Empathetic dialogue generation

Preliminaries

KEMP uses ConceptNet commonsense knowledge and NRC VAD emotional lexical knowledge. The section describes VAD-based emotion intensity values for dialogue words and external concepts, emphasizing higher-intensity concepts during graph construction.

Preliminaries

  • ConceptNet provides commonsense knowledge as tuples of head concept, relation, tail concept, and confidence score.It comprises 5.9 million tuples, 3.1 million concepts, and 38 relations.Source: Preliminaries · p. 3
  • NRC VAD supplies three-dimensional valence, arousal, and dominance vectors for 20,000 English words; Table 1 presents their interpretations.The passage gives “nice” as an example VAD vector: [0.93, 0.442, 0.65].Source: Preliminaries · p. 3, Table 1 · p. 3
  • KEMP uses NRC VAD to compute emotion intensity for dialogue words and external concepts, setting intensity to 0 when a word is absent from the lexicon.The intensity uses valence and arousal values with min-max normalization and an Lk norm; higher-intensity ConceptNet concepts are injected into KEMP.Source: Preliminaries · p. 3

Method

KEMP enriches dialogue history with external knowledge, encodes the resulting emotional context graph, and uses its emotional signals and dependencies to guide response generation.

Overview

  • KEMP enriches dialogue history with emotional knowledge in a graph, then distills emotional signals and uses cross-attention to learn dependencies for response generation.The overview describes these as three successive phases.Source: Overview · p. 3

Emotional context graph

  • For each non-stopword, KEMP retrieves ConceptNet tuples, filters and ranks them by relation, confidence, and emotion intensity, then connects selected concepts and dialogue words with directed edges.The graph also connects the CLS token to other vertices; words and emotional concepts form its vertices.Source: Emotional context graph · p. 3

Emotional context encoder

  • The encoder represents each vertex with word, positional, and vertex-state embeddings, applies graph attention to neighbors, and uses Transformer layers to incorporate global context.Graph attention updates vertices using immediate neighbors; the Transformer layers add information from all other vertices.Source: Emotional context encoder · pp. 3–4
  • The encoder weights vertex representations by emotion intensity to form ce, predicts an emotion category distribution, and trains emotion perception against the ground-truth dialogue emotion.The resulting emotional vectors are fed to the decoder as signals for response generation.Source: Emotional context encoder · p. 4

Emotion-dependency decoder

  • At each decoding step, KEMP combines graph context with emotional signals and applies emotional cross-attention to model dependencies between the graph and target response.An emotional attention loss focuses on vertices with higher emotion intensity values.Source: Emotion-dependency decoder · pp. 4–5
  • The decoder generates words sequentially, mixing vocabulary generation with copying graph vertices and jointly optimizing emotion perception, emotional attention, and generation losses.The transformed emotional signal is prepended to prior word embeddings supplied to the response decoder.Source: Emotion-dependency decoder · pp. 4–5

Experimental Settings

Experiments use EMPATHETICDIALOGUES, compare KEMP with empathetic-dialogue baselines and ablations, and evaluate automatic and human judgments.

Dataset

  • EMPATHETICDIALOGUES contains about 25k one-to-one open-domain conversations with 32 emotion labels; the speaker’s label is hidden at test time.The splits contain 17,802 training, 2,628 validation, and 2,494 test dialogues.Source: Dataset · p. 5

Baselines for comparison

  • Comparisons include Transformer, EmoPrepend-1, MoEL, MIME, and EmpDG; ablations remove the emotional context encoder or emotion-dependency decoder, with DialoGPT also tested.Source: Baselines for comparison · p. 5

Implementation details

  • Training uses Glove initialization, a 24,647-token vocabulary, Adam, batch size 16, and early stopping; KEMP trains for about three hours.The maximum concepts are 10 per dialogue and 5 per token; inference decoding is capped at 30 steps.Source: Implementation details · p. 5

Evaluation metrics

  • Automatic evaluation uses Perplexity, Distinct-1, Distinct-2, and Emotion Accuracy; human ratings assess Empathy, Relevance, and Fluency on five-point scales.Three annotators rate 100 sampled dialogues and generations.Source: Evaluation metrics · pp. 5–6

Results and analysis

KEMP outperforms strong baselines in automatic and human evaluations. Further analyses examine knowledge quantity, attention patterns, pre-trained-model integration, and generated examples.

Automatic evaluation results

External knowledge analysis

Automatic evaluation results

  • Removing either the emotional context encoder or emotion-dependency decoder worsens performance; the latter lowers most metrics dramatically.Source: Automatic evaluation results · p. 6

Emotion-dependency analysis

  • KEMP directs attention to informative knowledge and dialogue-history words, unlike EmpDG’s focus on general words.The examples include “fight” and “grow” in external knowledge and “keep” and “healthy” in dialogue history.Source: Table 5 · p. 7, Emotion-dependency analysis · p. 7

Effectiveness of pre-trained model

Case study

  • In two examples, KEMP recognizes negative emotion and produces a context-consistent response balancing content and emotion.Source: Table 7 · p. 7, Case study · p. 7

Conclusion and outlook

KEMP enhances emotion perception and dependency modeling with emotion-related concepts, and outperforms state-of-the-art methods in automatic and human evaluations. Its emotional context graph uses heuristic rules that are not flexible across knowledge resources.

Conclusion and outlook

  • KEMP outperforms state-of-the-art methods in both automatic and human evaluations.Source: Conclusion and outlook · p. 7
  • KEMP uses emotion-related concepts to enhance empathetic dialogue systems’ emotion perception and dependency abilities.Source: Conclusion and outlook · p. 7
  • The emotional context graph, emotional context encoder, and emotion-dependency decoder are each reported as effective components of KEMP.Source: Conclusion and outlook · p. 7
  • KEMP’s heuristic graph-construction rules are not flexible across different knowledge resources.The authors identify automatic graph construction using knowledge reasoning models as future work.Source: Conclusion and outlook · p. 7
Loading 2009.09708v3…