Source-linked AI summary
Affect-LM: A Neural Language Model for Customizable Affective Text Generation
Sayan Ghosh, Mathieu Chollet, Eugene Laksana, Louis-Philippe Morency, Stefan Scherer
TL;DR
Affect-LM targets the limited integration of affective information into neural language models for conversational text. It extends an LSTM with affect categories and a tunable strength parameter, and evaluations report expressive generation, grammaticality, and lower perplexity than a baseline. The method is evaluated on conversational corpora with differing emotional content, domains, and sizes.
Problem
Neural language modeling had not sufficiently incorporated affective information despite the importance of affect in verbal communication and spoken-language systems.
Method
Affect-LM extends an LSTM language model with context-derived affect categories and an affect-strength parameter controlling the influence of affective information on word prediction.
Results
Affect-LM generated expressive emotional text at varying strengths without affecting grammatical correctness and achieved lower perplexity than a baseline LSTM when affect was inferred from context words.
Takeaways & Limitations
Affect-LM provides customizable affective conversational generation while affect-category information can improve language-model prediction.
Takeaways & Limitations
The evaluation uses corpora that differ in domain and size, with the Fisher corpus 25 times larger than the other corpora combined.
Abstract
from arXiv · showhide
Human verbal communication includes affective messages which are conveyed through use of emotionally colored words. There has been a lot of research in this direction but the problem of integrating state-of-the-art neural language models with affective information remains an area ripe for exploration. In this paper, we propose an extension to an LSTM (Long Short-Term Memory) language model for generating conversational text, conditioned on affect categories. Our proposed model, Affect-LM enables us to customize the degree of emotional content in generated sentences through an additional design parameter. Perception studies conducted using Amazon Mechanical Turk show that Affect-LM generates naturally looking emotional sentences without sacrificing grammatical correctness. Affect-LM also learns affect-discriminative word representations, and perplexity experiments show that additional affective information in conversational text can improve language model prediction.
1 Introduction
Affect-LM addresses the limited integration of affective information with neural language modeling by generating conversational text conditioned on affect categories and tunable affect strength. The paper evaluates customizable generation, perceptual quality, and language-modeling performance.
- Motivation: Affect describes emotional, mood-related, and personality-related experiences conveyed alongside syntactic information in verbal messages.Affective language processing matters for spoken-language systems, dialogue systems, and conversational agents.
- Proposed approach: The model overview demonstrates emotionally colored conversational generation across five affect categories and three distinct affect strengths.Figure 1 includes three generated examples for the happy category.
- Research gap: Neural language models outperform n-gram baselines, but existing work had made limited use of affective information in language modeling.The paper identifies a gap in augmenting neural language models with affective information and generating emotional text from data.
- Proposed approach: Affect-LM extends an LSTM language model to generate conversational text conditioned on affect categories and a customizable affect-strength parameter.The affect category may be manually selected or inferred from preceding context words using emotional-word features.
- Research questions: The paper evaluates whether affect strength enables controllable emotional generation, whether generated sentences are perceived as expressive and grammatical, and whether affect improves prediction.The experiments include MTurk perception studies and perplexity comparisons with a baseline model.
2 Related Work
Prior work covers neural language modeling and affective text generation, but affective generation had not sufficiently adopted customizable neural methods or broad quantitative evaluation. Affect-LM is positioned as a data-driven alternative using state-of-the-art neural language modeling without syntactic prior knowledge.
- Neural language modeling: Neural language models address n-gram limitations by generalizing through word representations, with recurrent architectures predicting words from preceding context.Related work also conditions language models on other modalities, such as images for caption generation.
- Affective text generation: Earlier affective language-generation work used heuristic or knowledge-based strategies for specialized reports and simple emotional sentences.These approaches were not trained on large corpora and used additional syntactic knowledge such as parts of speech.
- Positioning: The paper contrasts Affect-LM with prior systems by using state-of-the-art neural language modeling, no syntactic prior knowledge, and data-driven generation of expressive emotional text.This addresses the stated lack of customizable neural techniques and quantitative evaluation across multiple emotionally colored corpora.
3 Model
Affect-LM extends an LSTM language model with affective information, learning affect-discriminative word representations and allowing generation to vary in emotional strength.
- 3.1 LSTM Language Model: The baseline LSTM predicts the next word from all preceding context words through one-hot inputs and learned word representations.The output representation f(.) is produced by an LSTM; U represents words, and b_i captures unigram occurrence.
- 3.2 Proposed Model: Affect-LM: Affect-LM adds an affect-dependent energy term to the word-prediction formulation.The additional term combines affect-category information with learned word embeddings and is weighted by affect strength β.
- 3.2 Proposed Model: Affect-LM: The learned V_i embeddings are intended to discriminate affective information conveyed by vocabulary words.The paper visualizes these affective representations in Figure 4.
- 3.3 Descriptors for Affect Category Information: The model conditions next-word generation on preceding words and an affect category inferred from context data during training.LIWC keyword spotting extracts five affective-process features: positive emotion, angry, sad, anxious, and negative emotion.
- 3.4 Affect-LM for Emotional Text Generation: Affect strength β controls the influence of affective information, ranging from β = 0 for neutral generation to β = ∞ for emotionally colored words without grammatical structure.The model supports positive emotion, anger, sadness, anxiety, and negative emotion categories.
4 Experimental Setup
The experiments evaluate Affect-LM on conversational corpora, testing emotional controllability, grammatical correctness, and prediction performance with affective information.
- 4 Experimental Setup: The experiments address whether Affect-LM generates emotional conversational text without sacrificing grammatical correctness and improves perplexity over a baseline LSTM.The setup includes the model architecture and corpora used for training and evaluation.
- 4.1 Corpora: The Fisher English Training Speech Corpus is the main training corpus, supplemented by three emotionally colored conversational corpora.Table 1 reports corpus statistics including total words and the fraction of emotionally colored words.
- 4.1 Corpora: The additional corpora contain more emotional content than Fisher but differ in domain and are significantly smaller.Examples of domain mismatch include the DAIC mental-health setting and CMU-MOSI opinion videos.
- 4.2 Model Configuration: The baseline and Affect-LM language models use a 10000-word vocabulary, two hidden LSTM layers with 200 neurons each, 20-step unrolling, and minibatches of 20.The baseline follows a non-regularized TensorFlow implementation, while Affect-LM adds a separate affect-category energy term.
- 4.4 Sentence Generation Perception Study: Affect-LM generation is evaluated through examples conditioned on different affect categories and a perception study using Amazon Mechanical Turk.More than 200 sentences were generated for four sentence beginnings, and each sentence was rated by two human raters on five affect dimensions using a seven-point scale.
- 4.4 Sentence Generation Perception Study: The perception analysis uses MANOVA followed by univariate ANOVAs to test how affect ratings change with affect strength β.The study conducts 5 MANOVAs and 30 follow-up ANOVAs, applying Bonferroni-corrected significance at p<0.001.
5 Results
Affect-LM generated affect-targeted conversational sentences whose perceived emotion could be manipulated with affect strength, while grammatical correctness generally remained acceptable. The model also learned affective word representations and reduced perplexity relative to the baseline across evaluated corpora.
- MTurk Perception Experiments: Affect strength significantly manipulated perceived affect for positive emotion, negative emotion, angry, sad, and anxious sentences.Positive, angry, and anxious targets showed significant affective-valence effects; negative emotion behaved as a broad negative-affect category, while sadness was specifically manipulated.
- MTurk Perception Experiments: Only the highest affect strength significantly reduced grammatical correctness for positive and negative emotion sentences.For sad and anxious sentences, correctness deteriorated with increasing affect strength, with sad showing significant deterioration at the two highest levels.
- Language Modeling Results: Affect-LM achieved lower average perplexity than the baseline across Fisher, DAIC, SEMAINE, and CMU-MOSI evaluations.The average improvement was 1.44 perplexity points (1.94%) for the Fisher-trained model and 0.79 (1.31%) for adapted models; larger reductions occurred in more emotional corpora.
- Word Representations: Affect-LM learned affective embeddings that separated positive-emotion words from clustered negative-emotion words.Figure 4 visualized word embeddings, coloring LIWC words by affect category and showing positive emotion separated from sad, angry, and anxious words.
6 Conclusions and Future Work
Affect-LM generates affective conversational text conditioned on context words, affect category, and affect strength. The model produces expressive text across emotional strengths while retaining grammatical correctness and achieves lower perplexity than a baseline LSTM when context words provide the affect category.
- Affect-LM generates affective conversational text conditioned on context words, an affective category, and an affective strength parameter.
- MTurk perception studies show that Affect-LM generates expressive text at varying emotional strengths without affecting grammatical correctness.
- Affect-LM achieves lower perplexity than a baseline LSTM when the affect category is obtained from context words.
- Future work will investigate generation conditioned on facial images and speech, including applications to dialogue generation for virtual agents.