Source-linked AI summary
Unsupervised Text Style Transfer using Language Models as Discriminators
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick
TL;DR
Unsupervised text style transfer must generate controllable text from non-parallel data, while binary discriminator feedback can be unstable and insufficient for fluent generation. The paper uses a target-domain language model as a structured discriminator with end-to-end back-propagation, and reports improved performance across three tasks while eliminating negative-sample adversarial steps can stabilize training.
Problem
Binary discriminators in unsupervised style transfer can provide unstable or insufficient feedback for training fluent generators.
Method
The model replaces binary classifiers with a target-domain language model whose token-level probabilities score generated sentences, using continuous sampling to enable end-to-end back-propagation.
Results
The language-model discriminator outperformed traditional binary classifier discriminators on word substitution decipherment, sentiment modification, and related language translation.
Takeaways & Limitations
Structured language-model discrimination can provide more stable, informative generator feedback and can eliminate adversarial training with negative samples.
Takeaways & Limitations
Sentiment-transfer evaluation can encounter over-smoothing, changing less frequent words into more frequent ones.
Abstract
from arXiv · showhide
Binary classifiers are often employed as discriminators in GAN-based unsupervised style transfer systems to ensure that transferred sentences are similar to sentences in the target domain. One difficulty with this approach is that the error signal provided by the discriminator can be unstable and is sometimes insufficient to train the generator to produce fluent language. In this paper, we propose a new technique that uses a target domain language model as the discriminator, providing richer and more stable token-level feedback during the learning process. We train the generator to minimize the negative log likelihood (NLL) of generated sentences, evaluated by the language model. By using a continuous approximation of discrete sampling under the generator, our model can be trained using back-propagation in an end- to-end fashion. Moreover, our empirical results show that when using a language model as a structured discriminator, it is possible to forgo adversarial steps during training, making the process more stable. We compare our model with previous work using convolutional neural networks (CNNs) as discriminators and show that our approach leads to improved performance on three tasks: word substitution decipherment, sentiment modification, and related language translation.
1 Introduction
Unsupervised text style transfer targets controllable generation from non-parallel data, but disentangling style from content and training with discrete tokens remain difficult. The paper replaces binary discriminators with a language-model discriminator and reports more stable training and improved performance across three tasks.
- Non-parallel data is common in text style transfer, despite controllable generation’s applications in dialogue systems and natural language interfaces.
- Unsupervised style transfer must disentangle attributes such as sentiment or orthography from underlying content, even though they interact subtly.
- Discrete word tokens make adversarial generator optimization difficult, so prior methods use REINFORCE or professor forcing after training.
- The proposed discriminator is an implicitly trained target-domain language model that evaluates generated sentences through token-level conditional probabilities.
- Eliminating adversarial steps with negative samples makes training more stable in practice when the language model serves as a structured discriminator.
- The language-model discriminator outperforms a broad set of state-of-the-art approaches on word substitution decipherment, sentiment modification, and related language translation.
2 Unsupervised Text Style Transfer
Prior unsupervised style-transfer systems use encoder-decoder models with style representations and binary adversarial discriminators, trained through reconstruction and distribution-alignment objectives. Their discrete transferred outputs make end-to-end generator training difficult.
- 2 Unsupervised Text Style Transfer: Style transfer uses non-parallel datasets X and Y with different styles, aiming to transfer each sentence toward the other dataset’s style.
- 2 Unsupervised Text Style Transfer: The generative formulation samples style and content representations before generating a sentence from their conditional distribution.
- 2 Unsupervised Text Style Transfer: An encoder extracts a content vector, while a decoder combines that vector with a style representation to generate transferred sentences.
- 2 Unsupervised Text Style Transfer: VAE-style regularization seeks shared content-vector distributions, but prior text models encountered training collapse that prevented the latent variable from capturing sentence content.
- 2 Unsupervised Text Style Transfer: Prior systems add style or real/fake discriminators to constrain generated samples and align content representations across domains.
- 2 Unsupervised Text Style Transfer: The overall objective is a min-max game among the encoder, decoder, and three discriminators, alternating discriminator and encoder-decoder updates.
3 Language Models as Discriminators
The model replaces binary style discrimination with a target-domain language model that scores transferred sentences through token-level probabilities. A continuous Gumbel-softmax approximation makes this signal differentiable for generator training, while experiments examine whether negative-sample adversarial updates are necessary.
- Language Model Discriminator: A target-domain language model replaces the conventional binary classifier by assigning probabilities to real and transferred sentences at the token level.Token-level probabilities provide information about which words contribute to high sentence perplexity.
- Model Architecture: The architecture combines reconstruction with transfer, where the style label is switched and the generator produces a sentence evaluated by the target language model.
- Negative Samples: Setting γ = 0 trains the language model only on real sentences and removes potentially unstable adversarial steps involving negative samples.The authors investigate whether negative samples are necessary because language-model normalization may already lower their probabilities.
- Continuous Approximation: The generator is trained with a continuous approximation: Gumbel-softmax distributions replace discrete sampled words, enabling standard back-propagation through language-model loss.At each timestep, weighted embeddings enter the language model; cross-entropy losses are summed across timesteps.
- Continuous Approximation: The continuous loss compares generator output distributions with language-model output distributions at each timestep, reaching its minimum when the distributions align.
- Training Procedure: Training uses discrete generator samples for language-model discriminator updates but continuous approximations for generator updates.
- Training Stabilization: Because language models prefer short sentences, the experiments normalize loss by length and match transferred-sentence length to the input.
4 Experiments
Experiments evaluate the language-model discriminator on decipherment, sentiment modification, and related language translation. Across these tasks, the model is effective without adversarial training and often improves over CNN-discriminator approaches, though adversarial negative samples can destabilize training.
- Experimental setup: The experiments compare a single language model discriminator with CNN-discriminator approaches across decipherment, sentiment modification, and related language translation.The model operates directly on transferred output sentences, whereas Shen et al. (2017) use three discriminators.
- 4.1 Word substitution decipherment: Negative-sample adversarial training sometimes improves decipherment, but it makes training unstable and can cause the model to diverge easily.This instability prevented consistently better results from incorporating adversarial training.
- 4.1 Word substitution decipherment: Without adversarial training, the language model performs much better than Shen et al. (2017) when fewer than 100% of tokens are changed, and is only 1.5 BLEU worse at 100% change.Combining the language model with CNN discriminators reaches a BLEU score of 52.1 in the 100% change case.
- 4.2 Sentiment Manipulation: The language model outperforms Shen et al. (2017) on sentiment accuracy, content preservation, and fluency, while combining it with a classifier improves accuracy and perplexity over the classifier alone.Sentiment transfer is evaluated using accuracy, BLEU against original sentences, and perplexity.
- 4.2 Sentiment Manipulation: Against DeleteAndRetrieve on human-annotated sentiment-transfer references, the model has similar accuracy, much better BLEU scores, and slightly better perplexity.The comparison uses 500 human-annotated sentences as ground truth.
- 4.3 Related language translation: For related language translation, the language-model discriminator slightly outperforms Shen et al. (2017) on sr–bos and bos–sr, and reaches BLEU1 scores of 81.6 and 85.5 on the easier directions.The reported directions are cn–tw and tw–cn; BLEU4 is near zero for sr–bos and bos–sr.
5 Related Work
The paper situates its method among unsupervised style-transfer and language-model applications, contrasting its discriminator design with prior approaches.
- Hu et al. learn controllable attributes through disentangled representations, while Shen et al. introduce adversarial training for unsupervised text style transfer.
- Shen et al. apply discriminators to encoder representations and decoder hidden states to align their distributions.
- Back-translation is complementary to this method and could be integrated to further improve performance.
- Text GAN progress remains limited because discrete tokens are non-differentiable, motivating methods using REINFORCE to fine-tune trained models.
- Earlier image style-transfer methods align CNN feature covariances, use adaptive instance normalization, or impose cycle consistency.
- Prior language-model applications mainly rerank base-model outputs or provide training supervision, whereas this model uses language models as discriminators in end-to-end distillation.
6 Conclusion
The paper concludes that language models can replace binary discriminators in unsupervised text style transfer, providing more stable training and strong performance across three tasks.
- Language-model discriminators outperform traditional binary classifier discriminators on word substitution decipherment, sentiment modification, and related language translation.
- Language models provide generators with a more stable and informative training signal than binary classifiers.
- Structured language-model discrimination can eliminate adversarial training with negative samples, suggesting a direction for addressing GAN training difficulty.
A Training Algorithms
The training algorithm takes two non-parallel style datasets and iteratively updates the model using language-model objectives until convergence.
- The algorithm inputs a dataset containing two different styles, X and Y.
- Training uses weight parameters λ and γ, temperature τ, and initialized encoder, generator, and language-model parameters.
- The procedure trains the language model using Equation 5 and outputs a text style-transfer model with encoder and generator parameters.
B Model Configurations
The experiments use a GRU encoder-decoder and a separately trained language model, with shared optimization settings and annealed Gumbel-softmax temperature.
- The model uses one-layer GRU encoder and decoder networks, 100-dimensional word embeddings, and 700-dimensional GRU hidden states.
- The language model matches the decoder architecture but does not share parameters with other components and is trained from scratch.
- Training uses Adam with batch size 128, 20 epochs, validation-based hyperparameter selection, and grid search.
- The Gumbel-softmax temperature starts at 1.0, halves each epoch, and stops decreasing at 0.001.
C Sentiment Transfer Examples
Table 5 presents examples of sentiment transfer.
- Table 5 contains sentiment transfer examples.
- The table is presented as an example set for sentiment transfer.
- Its focus is transferred sentences in the sentiment-transfer setting.