Source-linked AI summary
Politeness Transfer: A Tag and Generate Approach
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, Shrimai Prabhumoye
TL;DR
The paper addresses how to convert non-polite sentences into polite ones without changing their meaning. It builds a tag-and-generate system and evaluates it on politeness plus five other style-transfer tasks, where it improves content preservation while maintaining or improving transfer accuracy.
Problem
Politeness transfer must preserve sentence meaning, but non-politeness lacks explicit cues and differs from transfers between clearly polarized styles.
Method
The tag-and-generate pipeline identifies or inserts stylistic positions, replaces them with tags, and generates a sentence in the target style.
Results
Across six style-transfer tasks, automatic and human evaluations show stronger content preservation with comparable or better transfer accuracy than existing methods.
Takeaways & Limitations
The task definition and nearly 1.39 million politeness-labeled sentences establish a benchmark for content-preserving politeness transfer.
Takeaways & Limitations
Automatic metrics may not reflect task success because simple adversarial additions can achieve high accuracy and BLEU scores.
Abstract
from arXiv · showhide
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning. We also provide a dataset of more than 1.39 instances automatically labeled for politeness to encourage benchmark evaluations on this new task. We design a tag and generate pipeline that identifies stylistic attributes and subsequently generates a sentence in the target style while preserving most of the source content. For politeness as well as five other transfer tasks, our model outperforms the state-of-the-art methods on automatic metrics for content preservation, with a comparable or better performance on style transfer accuracy. Additionally, our model surpasses existing methods on human evaluations for grammaticality, meaning preservation and transfer accuracy across all the six style transfer tasks. The data and code is located at https://github.com/tag-and-generate.
1 Introduction
The paper frames politeness transfer as converting non-polite sentences into polite ones while preserving meaning, addressing a style that is important but difficult to define and operationalize. It introduces a tag-and-generate pipeline and a large labeled dataset, reporting strong performance across six transfer tasks.
- Politeness transfer converts non-polite sentences into polite sentences while preserving their meaning.
- Politeness is multifaceted and depends on culture, language, and context, making it difficult to define as a style.
- Non-politeness is defined as the absence of both politeness and impoliteness, unlike transfers between clearly opposing style polarities.
- The tagger marks stylistic material or inserts target-style positions, and the generator produces a target-style sentence from that representation.Intermediate outputs are fully realized, making the pipeline interpretable.
- Across six style-transfer tasks, automatic and human evaluations show stronger content preservation with comparable or better transfer accuracy.
- Nearly 1.39 million sentences are labeled for politeness, supporting benchmark evaluation of the new task.
2 Related Work
The paper distinguishes politeness transfer from prior style-transfer work and presents tag-and-generate as a response to limitations in existing methods. Its broader evaluation extends beyond politeness to five additional transfer tasks.
- The paper uses the Enron corpus as a source for style transfer, extending earlier socio-linguistic work on power dynamics, formality, and politeness.
- Prior style-transfer research has focused on sentiment, captions, persona, gender, political slant, and formality rather than politeness.
- Formality and politeness are described as loosely connected but independent styles.
- DRG and its extension preserve source content relatively well but can delete content words, require matching content across styles, and rely on fixed style markers.The retrieval step is also described as time-consuming for large datasets.
- Unlike a sentiment-focused cloze approach, the proposed method can generate the entire sentence in the target style and is evaluated on five additional tasks.
3 Tasks and Datasets
The task targets action-directives and converts them into polite requests using data derived mainly from Enron emails. The datasets and bucket analyses emphasize the contrast between style-neutral content and explicit politeness cues.
- 3.1 Politeness Transfer Task: Politeness transfer targets action-directives, including imperatives and proposal questions, and converts them into polite requests.Gratitude or counterfactual phrasing can make the examples more polite.
- 3.1 Politeness Transfer Task: The Enron data are preprocessed through tokenization, lowercasing, pruning, and removal of duplicates or noisy sentences.
- 3.1 Politeness Transfer Task: The cleaned Enron corpus contains over 1.39 million sentences scored for politeness and divided into ten buckets, P0–P9.The data are split into train, test, and development sets in an 80:10:10 ratio.
- 3.1 Politeness Transfer Task: Sentences scoring over 90% are assigned to the P9 polite bucket, whose training split contains over 270K polite sentences.
- 3.1 Politeness Transfer Task: The P0 and P9 buckets contrast content-heavy words with words closely linked to polite style, supporting the task’s distinction from sentiment transfer.
- 3.2 Other Tasks: The Captions dataset supports transfer from factual style to humorous or romantic style, while Yelp, Amazon, Gender, and Political datasets support other transfer tasks.
4 Methodology
The methodology uses a two-stage tag-and-generate pipeline: a tagger produces a style-agnostic intermediate sentence, and a generator converts it into the target style. The design supports both replacing source-style markers and inserting target-style phrases into style-neutral inputs, making it especially suitable for politeness transfer.
- Two-stage pipeline: The tagger first infers a style-agnostic sentence z(x_i), after which the generator produces the transferred sentence in target style S2.The intermediate representation is intended to preserve content while removing dependence on the original style.
- Tagger variants: Adding [TAG] tokens without removing input phrases is crucial for politeness transfer from non-polite or style-neutral sentences, where new phrases may need to be inserted.This differs from methods that primarily remove source attributes and replace them with target-style attributes.
- Estimating style phrases: Style markers Γ_v are estimated from corpus pairs using n-gram tf-idf comparisons, with phrases above a style-impact threshold selected for training-data construction.An n-gram is more characteristic of S2 when its mean tf-idf is higher in X2 than in X1.
- Tagger variants: The tagger either replaces source-style attributes with [TAG] or adds [TAG] at positions where target-style phrases should be inserted.The replace-tagger suits inputs containing style markers, while the add-tagger handles style-neutral inputs.
- Training the add-tagger: The add-tagger is trained on target-style examples by removing style phrases and replacing them with [TAG], teaching insertion locations for style-neutral inputs.For politeness transfer, only polite sentences are used to train this add-tagger, and the La objective is described as crucial when one style is poorly defined.
- Style-targeted generation: The generator takes tagged, style-agnostic representations and replaces [TAG] tokens with phrases that realize the desired target style.Its training pairs map z(x_i) to sentences in the target style, independently of the original style.
5 Experiments and Results
Experiments evaluate the tag-and-generate approach across six style-transfer tasks using automatic and human measures of content preservation, fluency, and style accuracy. The model generally improves content preservation while maintaining comparable or better transfer accuracy, with performance depending partly on the tagging strategy.
- Evaluation: Automatic evaluation measures style-transfer accuracy and content preservation, using classifiers for accuracy and BLEU, ROUGE, and METEOR for preservation.Politeness uses a classifier trained on an annotated politeness corpus.
- Automatic results: 58.61 BLEU points greater than DRG on Politeness, with higher BLEU, ROUGE, and METEOR scores than DRG, CAE, and BST on Politeness, Gender, and Political datasets.Classifier accuracy is within 1% of DRG for Politeness.
- Automatic results: 5.25, 4.95, and 3.64 BLEU-reference increases on Yelp, Amazon, and Captions, respectively, while Amazon transfer accuracy improves by 14.2%.Yelp and Captions achieve accuracies similar to DRG.
- Evaluation caveat: Automatic metrics can be misleading: a naive sentiment baseline reached 91.3% accuracy and 61.44 BLEU despite not achieving task success.The baseline appends sentiment-bearing phrases, exploiting classifier and n-gram-based evaluation weaknesses.
- Content preservation: Only 6.9% of sentences changed non-tagged words, and some such changes produced more natural and fluent outputs.The model is trained to generate new content primarily at TAG positions.
- Human evaluation: Human evaluations show improved content preservation, target-attribute matching, and grammatical correctness, particularly for the Politeness domain.Ten judges rated content preservation, grammaticality, and target-attribute match on a 1–5 scale.
- Qualitative analysis: Qualitative examples show the model applying politeness strategies including counterfactual modals, shared first-person plural, apologies, and gratitude.The analysis contrasts the model with DRG outputs and discusses strategies such as “Could you please,” “Sorry,” and “thanks.”
6 Conclusion
The paper introduces politeness transfer and a dataset for the task, then presents tag-and-generate as an interpretable approach for content-preserving style transfer. Automatic and human evaluations show stronger content preservation while retaining or improving transfer accuracy.
- The paper introduces politeness transfer and a dataset of sentences curated from Enron email exchanges.The task converts non-polite sentences into polite ones while preserving content.
- Tag-and-generate is a simple, interpretable two-staged pipeline for content-preserving style transfer.
- Automatic and human evaluations show that the approach outperforms state-of-the-art models on content-preservation metrics while retaining or improving transfer accuracy.