PaperlayerView exact revision on arXiv

Source-linked AI summary

Contrastive Learning with Adversarial Perturbations for Conditional Text Generation

Seanie Lee, Dong Bok Lee, Sung Ju Hwang

arXiv:2012.07280v6cs.CL

TL;DR

Teacher-forced seq2seq models do not encounter incorrectly generated tokens during training, and random contrastive negatives can be easy to distinguish. CLAPS generates adversarially perturbed hard positive and negative pairs, and the paper reports improved performance across three text-generation tasks.

  • Problem

    Teacher forcing prevents seq2seq models from seeing incorrectly generated tokens during training, limiting generalization; random non-target negatives can also be readily distinguished.

    Source: §1 · pp. 1–2

  • Method

    CLAPS automatically generates negative examples by perturbing target representations to lower conditional likelihood and positive examples by adding larger perturbations while preserving high likelihood.

    Source: §1 · p. 2

  • Results

    More than 1% performance gain on all tasks according to BLEU scores; CLAPS significantly improves performance on machine translation, summarization, and question generation.

    Source: §1 · p. 3, §4.3 · p. 8

  • Takeaways & Limitations

    The generated hard pairs provide more meaningful positive-negative examples for contrastive learning of conditional sequence generation.

    Source: §1 · p. 2

    An alternative imposter placed near the source embedding yielded less performance gain.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Recently, sequence-to-sequence (seq2seq) models with the Transformer architecture have achieved remarkable performance on various conditional text generation tasks, such as machine translation. However, most of them are trained with teacher forcing with the ground truth label given at each time step, without being exposed to incorrectly generated tokens during training, which hurts its generalization to unseen inputs, that is known as the "exposure bias" problem. In this work, we propose to mitigate the conditional text generation problem by contrasting positive pairs with negative pairs, such that the model is exposed to various valid or incorrect perturbations of the inputs, for improved generalization. However, training the model with naive contrastive learning framework using random non-target sequences as negative examples is suboptimal, since they are easily distinguishable from the correct output, especially so with models pretrained with large text corpora. Also, generating positive examples requires domain-specific augmentation heuristics which may not generalize over diverse domains. To tackle this problem, we propose a principled method to generate positive and negative samples for contrastive learning of seq2seq models. Specifically, we generate negative examples by adding small perturbations to the input sequence to minimize its conditional likelihood, and positive examples by adding large perturbations while enforcing it to have a high conditional likelihood. Such "hard" positive and negative pairs generated using our method guides the model to better distinguish correct outputs from incorrect ones. We empirically show that our proposed method significantly improves the generalization of the seq2seq on three text generation tasks - machine translation, text summarization, and question generation.

1 INTRODUCTION

Teacher forcing leaves seq2seq models unexposed to incorrectly generated tokens, while random contrastive negatives can be trivial to distinguish. CLAPS generates harder positive and negative examples and improves performance across three conditional text-generation tasks.

1 INTRODUCTION

  • Teacher forcing supplies the ground-truth token at each step, so models do not encounter incorrectly generated tokens during training, limiting generalization.This exposure bias can result in low-quality text on unseen inputs.Source: §1 · p. 1
  • Random non-target sequences can be trivial negatives: many positive-negative pairs are distinguishable without training, especially for pretrained models.The problem worsens at smaller batch sizes, which reduce the chance of including meaningfully difficult examples.Source: §1 · p. 2, Figure 2 · p. 2
  • CLAPS generates negative examples by perturbing target representations to minimize conditional likelihood, making them semantically dissimilar yet close in embedding space.These hard negatives are intended to be difficult for the model to distinguish from true targets.Source: §1 · p. 2
  • CLAPS generates positive examples with large perturbations while preserving high conditional likelihood through a KL-divergence constraint.The resulting positives are far from the source representation but retain the target's semantics.Source: §1 · p. 2
  • CLAPS significantly improves seq2seq performance on machine translation, text summarization, and question generation.The paper reports gains of more than 1% on all tasks according to BLEU scores.Source: §1 · pp. 2–3
  • The framework contrasts positive source-target pairs with negative pairs in latent space to expose the model to valid and incorrect outputs.It replaces random negative construction with automatically generated, more difficult pairs for learning meaningful representations.Source: §1 · p. 2

2 RELATED WORK

Related work addresses exposure bias with scheduled sampling, reinforcement learning, and adversarial sequence generation, while contrastive learning has been used across language tasks. Prior adversarial text methods generally target classifier robustness or consistency rather than seq2seq contrastive learning.

Exposure Bias

  • Exposure-bias approaches include scheduled sampling, RL, and GAN-based sequence matching, with reported concerns including tuning effort, training stability, and questions about RL reward optimization.Temperature tuning of MLE language models has also been reported to outperform GAN-based text generation models.Source: §2 · p. 3

Adversarial Perturbation

  • Prior adversarial text methods use perturbations for semi-supervised consistency or classifier smoothness, rather than contrastive positive-negative learning for generation.The cited classifier methods use iterative PGD and are described as computationally prohibitive.Source: §2 · p. 3

Contrastive Learning

  • Contrastive learning separates positive from negative pairs and has been applied in metric learning, computer vision, and multiple NLP tasks.Examples include sentence representations, language modeling, word alignment, caption generation, and machine translation.Source: §2 · p. 3

3 METHOD

The method trains seq2seq models contrastively on source–target representations and adversarial examples in decoder hidden space. Low-likelihood imposters serve as negatives, while likelihood-preserving distant-targets serve as additional positives.

3.1 BACKGROUND: CONDITIONAL TEXT GENERATION

  • Seq2seq models map variable-length inputs to variable-length outputs with an encoder–decoder that maximizes conditional log likelihood, using concatenated source-token hidden representations.The encoder produces M, the concatenated source-token representations.Source: §3.1 · pp. 3–4

3.2 CONTRASTIVE LEARNING WITH ADVERSARIAL PERTURBATIONS FOR SEQ2SEQ

  • Contrastive training pulls source–target representations together and pushes unpaired target representations away, but random negatives can be trivial and text lacks well-defined semantic-preserving augmentation heuristics.The paper identifies these issues as motivation for constructing adversarial positive and negative examples.Source: §3.2 · p. 4

3.3 GENERATION OF IMPOSTERS

  • An imposter lowers the target sequence’s conditional likelihood by perturbing its token hidden states; linearization approximates the intractable minimization, producing a nearby representation that is semantically dissimilar.An alternative that makes the imposter close to the source representation yields less performance gain.Source: §3.3 · p. 5

3.4 GENERATION OF DISTANT-TARGETS

  • A distant-target uses large perturbations to reduce similarity to the source while keeping conditional likelihood high, approximated by a contrastive-loss step followed by KL matching; it becomes an additional positive example.The method uses the perturbed hidden state as a positive for the source sequence.Source: §3.4 · pp. 5–6

4 EXPERIMENT

The experiments evaluate CLAPS on machine translation, text summarization, and question generation, using T5-small and task-specific datasets and baselines. CLAPS improves benchmark performance, while visualizations, decoded examples, and human judgments provide additional evidence about its outputs.

4.1 TASKS

  • Experiments cover WMT’16 RO-EN translation, XSum summarization, and SQuAD question generation, with task-specific perturbation norms and training settings.The SQuAD test set was created by randomly splitting the original validation set.Source: §4 · p. 6, §4.1 · p. 6

4.2 EXPERIMENTAL SETUPS

  • The setup compares T5-based maximum-likelihood, augmentation, contrastive, and smoothing baselines, alongside task-specific models, using BLEU, ROUGE, METEOR, and QG BERT-QA scores.T5-small is used for the encoder and decoder; decoding uses beam search with width 4.Source: §4.2 · pp. 7–8

4.3 EXPERIMENTAL RESULTS

  • CLAPS gains more than 1% BLEU on all three tasks, outperforms baselines on XSum, and improves QG F1/EM.For summarization, contrastive learning with imposters alone also yields a large performance improvement.Source: Table 1 · p. 7, §4.3 · p. 8, Table 2 · p. 8
  • With CLAPS, the t-SNE visualization shows imposters pushed away from target embeddings and distant-targets pulled toward source embeddings.Without contrastive learning, targets and distant-targets are far from source embeddings, while imposters are close to them.Source: §4.3 · pp. 8–9, Figure 4 · p. 9
  • Decoded distant-targets preserve target meaning with a synonym substitution, whereas imposters often differ semantically and are grammatically incorrect.The examples are greedily decoded from the perturbed hidden representations.Source: §4.3 · p. 9, Table 3 · p. 9
  • Human judges preferred CLAPS outputs over T5-MLE for 70% of summaries and 85% of questions.The blind assessment covered 20 summaries and 20 questions, with two model outputs presented in random order.Source: §4.3 · p. 9

5 CONCLUSION

The conclusion presents CLAPS as a contrastive approach to exposure bias that constructs hard positive and negative examples. The authors report gains on three generation tasks and suggest possible applications beyond text generation.

5 CONCLUSION

  • CLAPS contrasts ground-truth input-output pairs with incorrect outputs and constructs hard positive and negative examples through adversarial perturbations.The negatives are semantically dissimilar but close to the input embedding; positives are semantically similar but farther away.Source: §5 · p. 9
  • The authors report improved seq2seq performance on machine translation, question generation, and text summarization.Source: §5 · p. 9
  • The authors suggest CLAPS may apply to other seq2seq domains, including speech recognition, text-to-speech, and video captioning.Source: §5 · p. 9

A EXPERIMENTAL DETAILS

This section describes the experimental setup and evaluation for machine translation, summarization, and question generation, and presents selected qualitative examples for question generation and Romanian-to-English translation.

Dataset

  • The experiments use WMT’16 RO-EN for machine translation, Xsum for summarization, and SQuAD for question generation; SQuAD’s original validation set is split into new validation and test sets.The original SQuAD test set is accessible only through the SQuAD leaderboard.Source: A EXPERIMENTAL DETAILS · p. 14

Preprocessing

  • Inputs and outputs are tokenized with the T5 tokenizer and capped at 128 tokens for translation, 512/128 for summarization, and 384/64 for question-generation inputs/questions.For translation, raw text is tokenized; question-generation inputs concatenate the answer and context.Source: A EXPERIMENTAL DETAILS · p. 14

Implementation

  • Fine-tuning uses pretrained T5-small from the transformers library, Adafactor, batch size 128, and 4 or 8 GPUs depending on the task.The number of in-batch negative examples is 16 or 32, and the hidden size of z matches the T5 hidden-state dimension of 512.Source: A EXPERIMENTAL DETAILS · p. 14

Evaluation

A EXPERIMENTAL DETAILS

  • For the St. John’s Cathedral example, CLAPS paraphrases the reference question about its style, while the context identifies the cathedral as Masovian Gothic.The reference asks what style the cathedral exemplifies; CLAPS asks what style it is a typical example of.Source: A EXPERIMENTAL DETAILS · p. 23
  • The quota and rent examples show CLAPS outputs translating Bucharest’s proposed quota and the annual cost of renting Pogor Club.The quota is given as around 1,500 souls, and the rent as 7,000 euros per year.Source: A EXPERIMENTAL DETAILS · p. 25
  • In narrative examples, CLAPS renders Crina’s name as “The crisis” and the family’s daughter as “their mother,” unlike the references.Source: A EXPERIMENTAL DETAILS · p. 25
Loading 2012.07280v6…