Source-linked AI summary
MojiTalk: Generating Emotional Responses at Scale
Xianda Zhou, William Yang Wang
TL;DR
Emotional response generation lacks large-scale, fine-grained labeled data, while controlling a generated response’s target emotion remains difficult. MojiTalk uses emoji-rich Twitter conversations as natural labels and trains reinforced conditional variational models; the authors report strong emotional response generation at scale, with some emoji classes underrepresented.
Problem
Emotional response generation lacks large-scale manually labeled datasets and fine-grained emotion labels, while explicitly controlling generated emotions is difficult.
Method
The paper trains reinforced conditional variational generative models on Twitter conversations, using response emojis as natural emotion labels and generation controls.
Results
The authors report state-of-the-art emotional text generation at scale, with CVAE improving perplexity and emoji accuracy over the base model and Reinforced CVAE further improving emoji accuracy.
Takeaways & Limitations
Naturally occurring emojis can support large-scale emotional response generation and conditioning without manually labeling the full training corpus.
Takeaways & Limitations
Fear and disgust are underrepresented and emoji-label distributions are somewhat unbalanced because of the social-media data.
Abstract
from arXiv · showhide
Generating emotional language is a key step towards building empathetic natural language processing agents. However, a major challenge for this line of research is the lack of large-scale labeled training data, and previous studies are limited to only small sets of human annotated sentiment labels. Additionally, explicitly controlling the emotion and sentiment of generated text is also difficult. In this paper, we take a more radical approach: we exploit the idea of leveraging Twitter data that are naturally labeled with emojis. More specifically, we collect a large corpus of Twitter conversations that include emojis in the response, and assume the emojis convey the underlying emotions of the sentence. We then introduce a reinforced conditional variational encoder approach to train a deep generative model on these conversations, which allows us to use emojis to control the emotion of the generated text. Experimentally, we show in our quantitative and qualitative analyses that the proposed models can successfully generate high-quality abstractive conversation responses in accordance with designated emotions.
1 Introduction
MojiTalk addresses limited large-scale emotional training data and coarse emotion labels by using emoji-rich Twitter conversations as naturally labeled data. It trains conditional generative models to produce abstractive responses for designated emotions.
- Motivation: Emotional response generation is needed for empathetic conversational agents, but large-scale manually labeled emotional datasets remain scarce and costly.Prior datasets are often too small for deep models with many parameters.
- Motivation: Existing medium- and large-scale corpora commonly use coarse labels such as happiness, sadness, anger, or binary sentiment, limiting emotional nuance.
- Approach: The paper uses emoji-rich Twitter conversations, treating fine-grained response emojis as natural emotion labels without manual annotation.The approach assumes users’ emoji choices convey the response’s underlying emotion.
- Approach: The task is to generate abstractive responses with an arbitrarily specified emotion rather than retrieve responses matching a target emotion.
- Approach: The system applies sequence-to-sequence, conditional variational, and reinforced conditional variational models to emotional response generation.A hybrid objective with policy gradient is used to improve emotion expression, evaluated with a sentence-to-emoji classifier.
- Contributions: The authors report a publicly available large-scale emoji-labeled Twitter dataset and strong, state-of-the-art emotional text generation at scale.
2 Related Work
The paper builds on sentiment analysis, emoji-based Twitter modeling, variational dialog generation, and sequence-to-sequence systems. Its focus is emotional response generation conditioned on a target emoji.
- Sentiment and emoji modeling: Sentiment analysis studies algorithms for understanding emotional text, including models trained on naturally emoji-labeled Twitter data.
- Sentiment and emoji modeling: DeepMoji uses a large naturally labeled Twitter emoji dataset for sentiment classification, whereas this work generates responses conditioned on context and target emoji.
- Variational dialog generation: Variational autoencoders encode data as probability distributions and sample from them to generate examples; conditional VAEs add conditioning to generation.
- Sequence-to-sequence dialog models: A modified attention-based sequence-to-sequence model serves as the base system and supports conditioning on single emojis.
- Research gap: Prior reinforced sequence-to-sequence dialog models did not model emotions or conditional labels, while prior conditional VAE dialog work did not explore emotion modeling or reinforcement learning.
3 Dataset
The dataset consists of English Twitter conversation pairs whose responses contain one of 64 common emojis, which provide the response emotion labels. Preprocessing selects labels, normalizes text, and splits the corpus into train, validation, and test sets.
- Collection: The authors collect Twitter conversations containing 64 common emojis, while noting that emoji usage is uneven and not always emotional.
- Collection: Each conversation pair contains an original post and response, and the response must include at least one of the 64 emoji labels.The crawl covered Twitter data collected from 12th to 14th August 2017.
- Filtering: Only English tweets without multimedia are retained, and conversations with fewer than three alphabetical words are excluded.
- Labeling: When a response contains multiple emoji types, the most frequent within-response emoji becomes the label; ties are resolved using corpus frequency.Among equally frequent response emojis, the least frequent corpus-wide emoji is selected.
- Preprocessing: Preprocessing removes mentions and hashtags, separates adjacent punctuation and emojis, normalizes repeated letters and symbols, tokenizes text, and limits the vocabulary to 20K tokens.
- Splits: The corpus is randomly split into 596,959 training, 32,600 validation, and 32,600 test conversation pairs.
4 Generative Models
The paper builds emotional response generators by conditioning sequence-to-sequence models on emoji labels, extending them with latent variables and policy-gradient rewards. The resulting Reinforced CVAE further trains generation toward emotion accuracy while constraining response appropriateness.
- 4.1 Base: Attention-Based Sequence-to-Sequence Model: The base SEQ2SEQ model encodes the original tweet and concatenates its representation with an embedded emoji before decoding the response.The original-tweet encoder uses bidirectional GRU cells, while the decoder is a one-layer GRU network.
- 4.2 Conditional Variational Autoencoder (CVAE): The CVAE extends SEQ2SEQ with response encoding, recognition, and prior networks to model latent variation in responses conditioned on the tweet and emoji.During training, the recognition network supplies latent z; during testing, the prior network supplies it because the target response is unavailable.
- 4.2 Conditional Variational Autoencoder (CVAE): The CVAE retains the base model’s attention mechanism, generating responses from the attention memory together with the condition c and latent variable z.This attention connection between the original-tweet encoder and decoder distinguishes the model from earlier text CVAEs described here.
- 4.2 Conditional Variational Autoencoder (CVAE): Because recurrent VAEs may ignore latent variables, the models use KL annealing, early stopping, and bag-of-word loss to balance latent learning and reconstruction.Without effective latent use, the VAE can mathematically deteriorate into a plain SEQ2SEQ model.
- 4.3 Reinforced CVAE: The Reinforced CVAE combines policy gradients with the CVAE and uses a separately trained emoji classifier to reward generated responses expressing the target emotion.The policy receives reward R from the classifier, uses baseline r to reduce variance, and is trained with a hybrid REINFORCE and variational lower-bound objective.
- 4.3 Reinforced CVAE: The reinforced objective can drive generation toward generic responses, so training adjusts rewards by emoji rank and combines emotion accuracy with response appropriateness.The balancing coefficient λ is set to 1 in the experiments.
5 Experimental Results and Analyses
The experiments evaluate generation quality, emotion accuracy, diversity, emotion controllability, and human judgments across the proposed models. CVAE improves general generation and diversity over SEQ2SEQ, while Reinforced CVAE strengthens emotional expression but introduces some length and relevance trade-offs.
- Evaluation setup: The evaluation covers perplexity, emoji accuracy, diversity, controllability by emoji, human preference judgments, and qualitative generated examples.The models are compared with a base SEQ2SEQ system using automatic and human evaluations.
- General results: CVAE significantly reduces perplexity and increases emoji accuracy over the base model, while Reinforced CVAE further improves emoji accuracy at a slight perplexity cost.The same emoji classifier is used for evaluation, and top-5 accuracy accommodates overlapping emoji meanings.
- Generation Diversity: CVAE models achieve substantially greater response diversity than SEQ2SEQ, while Reinforced CVAE somewhat compromises diversity when generating more emotional responses.Diversity is measured with unigram, bigram, and trigram type-token ratios.
- Controllability of Emotions: Conditioning on emojis enables responses for arbitrary target emotions, and CVAE improves top-5 accuracy for every one of the first 32 emoji labels.Reinforced CVAE produces larger gains for less common emojis, consistent with emoji-specified policy training.
- Human Evaluation: In a simplified Turing test, 18% of subjects misclassified machine-generated responses as human-written and 27% could not distinguish the responses.Human evaluation used crowdsourced judges and pairwise comparisons of generated responses.
- Case Study: Qualitative analysis finds SEQ2SEQ predictable and CVAE more diverse, but CVAE responses can become too diversified and less likely to reply to the original tweet.Reinforced CVAE can also generate lengthy responses by stacking sentences, although the paper reports that fluency is not impaired.
6 Conclusion and Future Work
The paper uses naturally annotated emoji-rich Twitter conversations to generate emotional responses without laborious human annotations, while acknowledging uneven emotion coverage in the data.
- The study collected more than half a million Twitter conversations whose response emojis serve as fine-grained emotion labels.The models generate responses with arbitrarily designated emotions and are evaluated automatically and by humans.
- Experiments show the proposed model generates high-quality emotional responses without laborious human annotations.
- Fear and disgust are underrepresented, and emoji frequencies are unbalanced because of the nature of social media text.
- Future work will accumulate data, increase underrepresented-emoji ratios, and develop more sophisticated abstractive generation methods.
A.1 Bag-of-Word Loss
The bag-of-words loss supplements the CVAE objective by modeling response words independently of their order, using token probabilities derived from the latent and conditioning variables.
- The response x is decomposed into word order xo and unordered words xbow, assumed conditionally independent given z and c.This yields p(x|z, c) = p(xo|z, c)p(xbow|z, c).
- Given z and c, p(xbow|z, c) is computed as the product of the probabilities of every response token.
- Function f maps z and c into vocabulary space and selects the element corresponding to token xt as its logit.
- The modified objective L′ adds the expected log probability of the bag-of-words representation to the original objective L.The expectation is taken over qR(z|x,c).
- CVAE training minimizes the modified objective L′.
A.2 Emoji Classifier
A separate neural emoji classifier evaluates whether generated responses express their designated emotions, achieving higher coverage at top-5 than top-1 prediction accuracy.
- The emoji classifier uses skip-connected bidirectional GRU-RNN layers and maps response Tweets to emoji labels.It is trained with dropout 0.2 and Adam at a 1e-3 learning rate, with gradients clipped to 5.
- Figure 5 reports top-1 and top-5 accuracy separately for the 32 most frequent emoji labels.Less common emojis may have low accuracy because they are underrepresented.
- The classifier outputs probabilities for all 64 emoji labels.
- 32.1% of test responses rank their labeled emoji highest, while 57.8% place it among the five highest-probability labels.These measures are called top-1 and top-5 accuracy, respectively.
A.3 Training Process of the Reinforced CVAE
Reinforced CVAE training combines variational optimization with policy-gradient updates based on rewards from the emoji classifier.
- Each training batch first computes the variational objective L′ and obtains generated text.
- The process then computes policy gradient J′ from generated-word probabilities and rewards determined by the emoji classifier.
- Gradient descent updates the CVAE components using a hybrid objective L′′ that includes L′.
A.4 Experiment Setting
The experiments train Reinforced CVAE models with staged optimization, tuned latent-variable settings, and emoji-based reward guidance. During generation, five sampled responses are compared and the one with the highest designated-emoji probability is selected.
- Model settings: The base and CVAE models use 128-dimensional word embeddings, 128-unit RNN layers, 12-dimensional emoji embeddings, and latent variables of size 268.Recognition and prior networks use three-layer tanh MLPs.
- Training procedure: Training pretrains CVAE by minimizing Eq. 9, then repeatedly processes batches through the CVAE to compute word probabilities and the variational lower-bound objective.The reinforced training procedure also computes reward information with an emoji classifier using target responses.
- Reinforcement signal: The reward R is the classifier probability of emoji label e for generated response x′, and α assigns values according to the response’s ranking across labels.The ranking rule distinguishes first place, ranks two through five, and other cases.
- Training procedure: Reinforced CVAE introduces policy loss only after two epochs, initializes from a fully converged SEQ2SEQ model, and uses a hybrid objective with λ set to 1.These settings stage reinforcement after initial CVAE training rather than applying the hybrid loss from the beginning.
- Training procedure: The final models combine bag-of-words loss with KL annealing to 0.5 by the end of epoch six, allowing more freedom than strict convergence on the CVAE objective.The authors interpret the lower final KL weight as an early-stopping strategy for finding a better result before convergence.
- Generation: During generation, the model samples five CVAE responses and returns the response with the highest probability of the designated emoji label.This selection step exploits latent-variable randomness while enforcing the requested emotional label.