PaperlayerView exact revision on arXiv

Source-linked AI summary

Polite Dialogue Generation Without Parallel Data

Tong Niu, Mohit Bansal

arXiv:1805.03162v1cs.CLcs.AIcs.LG

TL;DR

The paper addresses stylistic dialogue generation when parallel regular-to-stylistic data are unavailable and responses must remain fluent and contextually relevant. It introduces three weakly supervised models and finds that LFT and Polite-RL improve politeness without sacrificing dialogue quality, unlike Fusion and retrieval-based models.

  • Problem

    Stylistic dialogue generation lacks parallel regular-to-stylistic datasets while requiring fluent, contextually relevant, and paralinguistically accurate responses.

  • Method

    The paper presents three weakly supervised models: Fusion combines a dialogue decoder with a polite language model, LFT conditions training on a continuous politeness label, and Polite-RL rewards sampled responses using a politeness classifier.

  • Results

    LFT and Polite-RL produce significantly more polite responses without sacrificing dialogue quality, whereas Fusion and retrieval-based models improve politeness with poorer dialogue quality.

  • Takeaways & Limitations

    Weak supervision can generate a broad polite-to-rude response spectrum without parallel data, with LFT and Polite-RL retaining dialogue quality better than Fusion and retrieval-based models.

  • Takeaways & Limitations

    Polite-RL is motivated by the limitation that many contexts do not naturally elicit a polite response, so forcing the target politeness score may be unnatural.

Abstract

from arXiv · show

Stylistic dialogue response generation, with valuable applications in personality-based conversational agents, is a challenging task because the response needs to be fluent, contextually-relevant, as well as paralinguistically accurate. Moreover, parallel datasets for regular-to-stylistic pairs are usually unavailable. We present three weakly-supervised models that can generate diverse polite (or rude) dialogue responses without parallel data. Our late fusion model (Fusion) merges the decoder of an encoder-attention-decoder dialogue model with a language model trained on stand-alone polite utterances. Our label-fine-tuning (LFT) model prepends to each source sequence a politeness-score scaled label (predicted by our state-of-the-art politeness classifier) during training, and at test time is able to generate polite, neutral, and rude responses by simply scaling the label embedding by the corresponding score. Our reinforcement learning model (Polite-RL) encourages politeness generation by assigning rewards proportional to the politeness classifier score of the sampled response. We also present two retrieval-based polite dialogue model baselines. Human evaluation validates that while the Fusion and the retrieval-based models achieve politeness with poorer context-relevance, the LFT and Polite-RL models can produce significantly more polite responses without sacrificing dialogue quality.

1 Introduction

The paper addresses stylistic dialogue generation without parallel regular-to-stylistic data, focusing on politeness while preserving conversational relevance. It introduces three weakly supervised generative models and finds that LFT and Polite-RL balance politeness with dialogue quality better than Fusion and retrieval baselines.

  • Motivation: Politeness matters for conversational agents because users may learn interaction styles from systems that lack politeness.The paper focuses on politeness versus rudeness as a diverse paralinguistic style axis.
  • Motivation: Stylistic dialogue responses must be fluent, contextually relevant, and paralinguistically accurate, but style and conversation data are usually unpaired.The paper therefore targets indirectly supervised generation using separate style and dialogue domains.
  • Approach: Three weakly supervised models generate diverse, natural, contextually relevant polite or rude responses from separate style and dialogue domains.The models use the Stanford Politeness Corpus with Wikipedia and StackExchange requests, and MovieTriples with IMSDB movie scripts.
  • Approach: The models incorporate style through decoder fusion, label fine-tuning, or classifier-guided reinforcement learning.The Fusion model combines a dialogue decoder with a language model trained on polite utterances.
  • Findings: Human evaluations show that LFT and Polite-RL produce polite responses without sacrificing dialogue coherence and relevance, unlike Fusion and retrieval models.Fusion and retrieval models increase politeness at the cost of poorer dialogue quality.

2 Related Works

Related work covers style transfer with and without parallel data, persona-based dialogue, and politeness classification. This paper differs by adding continuous politeness control while prioritizing dialogue-context relevance rather than strict content preservation.

  • Style Transfer with Parallel Data: Parallel style-transfer systems often treat conversion between styles as a translation problem using phrase-based or sequence-to-sequence models.Examples include modern-to-Shakespearean language conversion and labeled sequence transduction.
  • Style Transfer without Parallel Data: Nonparallel style-transfer methods use style keywords, latent representations, variational autoencoders, or cross-alignment to separate content and style.These approaches include bag-of-words annotations and latent style representations combined with new content at inference.
  • Task Distinction: Unlike strict content-preserving style transfer, this task adds style while maintaining relevance to conversational context.The distinction is especially important for dialogue responses.
  • Task Distinction: The models treat politeness as a continuum controlled through fusion rate, continuous label magnitude, or reinforcement-learning weight.This differs from fixed style categories or topics such as Shakespearean language.
  • Related Dialogue and Classification Work: Persona-based dialogue and politeness classification provide related foundations for adapting responses to speaker style and identifying polite requests.Prior work uses shared Seq2seq and autoencoder decoders for persona adaptation, while classifiers range from engineered-feature SVMs to neural CNNs.

3 Politeness Classification Model

The paper develops an LSTM-CNN classifier that combines recurrent and convolutional features to predict whether an utterance is polite or rude. It reports comparable in-domain and improved cross-domain accuracy over prior state-of-the-art systems.

  • Architecture: The classifier uses a bidirectional LSTM followed by a convolutional layer to capture long-distance relationships and windowed features.The architecture extends a prior CNN-based politeness classifier.
  • Role in the System: The model is designed to provide politeness scores that support stylistic dialogue response generation.Its predicted style information is used by the paper’s weakly supervised generation models.
  • Architecture: LSTM hidden states are convolved with filters of various sizes, max-pooled, and passed to a softmax over Polite and Rude labels.The pipeline produces multiple features before classification.
  • Results: The classification model achieves comparable in-domain accuracy and improved cross-domain accuracy over state-of-the-art prior results.The paper discusses these evaluation results in a later section.

4 Polite-Style Dialogue Models

The paper builds three weakly supervised polite-style dialogue models on a Seq2seq backbone, using separate style and dialogue data rather than parallel pairs. Fusion mixes a polite language model with the decoder, LFT scales a prepended politeness label, and Polite-RL uses classifier rewards to encourage stylistic responses while retaining conversational relevance.

  • Base Seq2seq Dialogue Model: The base dialogue model is a bidirectional LSTM encoder with a multilayer LSTM decoder and additive attention over conversation history.It serves as the backbone for the three proposed models.
  • Fusion Model: Fusion linearly combines the Seq2seq decoder and a polite-only language model to produce each final output distribution.The polite language model is trained on utterances whose classifier scores exceed 0.8, while α controls the Seq2seq contribution.
  • Label-Fine-Tuning Model: LFT scales a trainable prepended politeness-label embedding by classifier scores during training and by desired scores at test time.Test-time scaling supports polite, neutral, and rude responses; the paper describes polite scores as 0.5–1.0 and rude scores as 0.0–0.5.
  • Polite Reinforcement Learning Model: Polite-RL combines maximum-likelihood training with policy-gradient training that rewards sampled responses according to their politeness-classifier scores.The reward can be inverted to encourage rudeness instead of politeness.
  • Polite Reinforcement Learning Model: Polite-RL addresses LFT’s one-time style supervision, while LFT can generate multiple styles at test time and reward-based training operates in one direction at a time.The paper notes that label conditioning could extend reward-based training to multiple styles.

5 Experimental Setup

The experiments use separate politeness and dialogue datasets, evaluate stylistic models through human judgments, and supplement those studies with automatic metrics. Training and evaluation include specific optimization, preprocessing, and reproducibility choices.

  • Datasets: The style domain uses the Stanford Politeness Corpus, while the dialogue domain uses the MovieTriples corpus of movie-script conversations.The politeness corpus contains labeled Wikipedia and Stack Exchange requests; MovieTriples contains 245K conversations in X-Y-X triplet format.
  • Human evaluation: Human evaluations compare three stylistic models, a Seq2seq base model, and retrieval-based models on politeness and dialogue quality.The study samples 300 contexts covering all conversation types and evaluates context relevance and coherence alongside politeness.
  • Human evaluation: Dialogue quality is assessed with overall human judgments rather than several separate fine-grained metrics, following prior findings about limited added information and annotator confusion.The quality study uses a five-point Likert scale, with labels ranging from “Very good” to “Very poor.”
  • Automatic evaluation: Automatic evaluation uses BLEU only as complementary trend-verification information because no ground-truth stylized responses exist for MovieTriples.Human perception studies remain the primary evaluation method for this setting.
  • Training details: The models use 300-dimensional embeddings, Adam with learning rate 0.001, dropout 0.2, minibatches of 96, and separate training schedules for the classifier and stylistic models.The classifier is trained for 3 epochs and the stylistic models for 35 epochs; fusion and reinforcement-learning weights are tuned on development data.
  • Training details: The paper notes that additional reproducibility details and analysis examples will be provided in a post-publication supplement.This supplements the training details reported in the main paper.

6 Results

The results evaluate the politeness classifier, base dialogue model, stylistic dialogue systems, and LFT variants using automatic metrics and human judgments. LFT and Polite-RL improve politeness while preserving dialogue quality better than Fusion and retrieval-based models.

  • 6 Results: The LSTM-CNN politeness classifier improves cross-domain accuracy over prior SVM and CNN models while maintaining comparable in-domain accuracy.Cross-domain accuracy is important for applying style control across dialogue domains.
  • 6 Results: The base Seq2seq dialogue model achieves quality comparable to Serban et al.'s HRED models on the reported PPL and WER metrics.The comparison uses matched vocabulary and preprocessing protocols for fair evaluation.
  • 6 Results: All three generative stylistic models improve politeness over Seq2seq, but Fusion sacrifices dialogue quality because its standalone politeness language model ignores context.Fusion's politeness gain is not statistically significant, while its dialogue-quality loss is significant.
  • 6 Results: LFT is the most polite model and retains dialogue quality statistically equal to Seq2seq, while Polite-RL significantly improves politeness and slightly improves dialogue quality.LFT and Polite-RL have statistically equal politeness levels and avoid harming context relevance.
  • 6 Results: Retrieval models are highly polite but have significantly lower dialogue quality than Seq2seq, Polite-RL, and LFT.Their politeness comes with one of the worst balances between politeness and dialogue quality.
  • 6 Results: Automatic metrics reproduce the human-study pattern: retrieval models and LFT are most polite, while LFT and Polite-RL maintain better dialogue quality than Fusion.The untrained Seq2seq baseline has an average politeness score of 0.49.

7 Analysis

The analysis examines classifier behavior, generated-response examples, and Polite-RL saliency. The examples show learned politeness strategies alongside relevance failures in retrieval and occasional persona errors.

  • 7.1 Analysis of Politeness Classifier: The politeness classifier usually assigns reasonable scores and captures strategies including gratitude, deference, greetings, positive lexicon, indirection, and modal usage.It occasionally assigns strongly polite or rude scores to mild or neutral responses.
  • 7.2 Output Examples of Stylistic Dialogue: Retrieval responses can have oracle-level fluency and grammaticality yet remain irrelevant to the conversation, lowering their human-rated dialogue quality.The examples describe retrieved responses as sharing words with the context while being totally irrelevant.
  • 7.2 Output Examples of Stylistic Dialogue: The generative models demonstrate politeness strategies such as indirection, positive lexicon, counterfactual modals, and politeness during disagreement.These strategies appear in comparisons involving Fusion and LFT outputs.
  • 7.2 Output Examples of Stylistic Dialogue: LFT and Polite-RL can produce apparently relevant compliments that address the wrong person, motivating future persona-based politeness models.The issue concerns maintaining the correct conversational persona rather than politeness alone.
  • 7.1 Analysis of Politeness Classifier: Polite-RL saliency heatmaps visualize token contributions to the classifier reward and indicate recognition of multiple politeness strategies.The visualization uses absolute derivative magnitudes with respect to each dimension.

8 Conclusion and Future Work

The paper concludes that weakly supervised, classifier-guided models can generate polite-to-rude dialogue without parallel data. LFT and Polite-RL preserve dialogue quality, while future work targets broader styles and improved generation quality.

  • 8 Conclusion: The three generative models produce a rich polite-to-rude spectrum without parallel data by relying on a style classifier.The models are designed for dialogue responses rather than strict content-preserving sentence rewriting.
  • 8 Conclusion: LFT and Polite-RL generate more polite responses without losing dialogue quality, unlike Fusion and the retrieval-based models.This conclusion is supported by multiple human evaluations and automatic metrics.
  • 8 Future Work: Future work should improve both politeness and dialogue quality and may use variational, adversarial, or decoder-regulation techniques.These methods are presented as possible directions rather than evaluated solutions in this paper.
  • 8 Future Work: The models can be generalized to other emotion and personality styles using a style classifier.The paper also discusses extending them to stylistic task-based dialogue while preserving content and style distinctions.
Loading 1805.03162v1…