PaperlayerView exact revision on arXiv

Source-linked AI summary

Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation

Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang, Soroush Vosoughi

arXiv:2012.02952v1cs.CLcs.LG

TL;DR

Text augmentation methods can produce unreadable or semantically inconsistent samples, while conditional language models require costly training and sufficient data. Data Boost guides an off-the-shelf language model toward target classes through reinforcement learning during decoding. It improves classification performance across diverse tasks and classifiers, and the authors report that it surpasses several prior augmentation methods.

  • Problem

    Existing text augmentation can have low readability or weak semantic consistency, while conditional language models require costly training and sufficient data for their conditions.

    Source: §1 · pp. 1–2

  • Method

    Data Boost uses reinforcement learning during decoding to guide an off-the-shelf GPT-2 language model toward a specified target class.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Data Boost improves classification performance, is classifier-agnostic, and surpasses several prior augmentation methods across three diverse classification tasks.

    Source: §8 · p. 9

  • Takeaways & Limitations

    The framework produces sentence-level augmentation with greater vocabulary and structural variety, and human evaluations verify readability and label consistency.

    Source: §1 · p. 2

    Gains may be marginal for classes not well modeled by lexical features, such as metaphor, sarcasm, and formality, where syntactic features may matter.

    Source: §7 · p. 9

Abstract

from arXiv · show

Data augmentation is proven to be effective in many NLU tasks, especially for those suffering from data scarcity. In this paper, we present a powerful and easy to deploy text augmentation framework, Data Boost, which augments data through reinforcement learning guided conditional generation. We evaluate Data Boost on three diverse text classification tasks under five different classifier architectures. The result shows that Data Boost can boost the performance of classifiers especially in low-resource data scenarios. For instance, Data Boost improves F1 for the three tasks by 8.7% on average when given only 10% of the whole data for training. We also compare Data Boost with six prior text augmentation methods. Through human evaluations (N=178), we confirm that Data Boost augmentation has comparable quality as the original data with respect to readability and class consistency.

1 Introduction

The introduction identifies a gap between image and text augmentation: common text methods can harm readability or semantic consistency. Data Boost uses reinforcement-learning-guided conditional generation with an off-the-shelf language model to produce class-directed augmentation without training a task-specific LM from scratch.

1 Introduction

  • Existing text augmentation can damage readability or semantic consistency, reflecting the interdependence of syntax and meaning in text.Naive token edits can disrupt word order or remove salient words; synonym replacement may ignore context, while back-translation can cause repetition and lexical shrinkage.Source: §1 · p. 1
  • Off-the-shelf language models generate without task direction, while conditional LMs require costly training and sufficient condition-labeled data.These requirements are especially difficult in low-resource tasks.Source: §1 · p. 2
  • Data Boost turns off-the-shelf GPT-2 into a conditional generator by using reinforcement learning during decoding to guide samples toward a target class.The framework requires neither extra data collection nor training a task-specific LM from scratch.Source: §1 · p. 2
  • Data Boost reports advances across three tasks and five classifiers versus six related works, with sentence-level variety and human-verified readability and label consistency.It also avoids external datasets and separate systems by modifying GPT-2's decoding stage without changing its architecture.Source: §1 · p. 2
  • The introduction contrasts prior word- or phrase-level edits with Data Boost's sentence-level augmentation, which offers greater vocabulary and structural variety.The cited table is described as a demonstration of existing methods on positive sentiment.Source: Table 1 · p. 1, §1 · p. 2

2 Data Boost

Data Boost turns an off-the-shelf language model into a target-label conditional generator by applying reinforcement learning during decoding. Its reward combines class-lexicon salience with a KL constraint, and policy-gradient updates control the strength of generation guidance.

2.1 Conditional Generator

  • Data Boost keeps the language-model architecture unchanged and delays argmax so reinforcement learning can update hidden states between softmax and token selection.The resulting conditional policy decodes toward the target label.Source: §2.1 · p. 3

2.2.1 Reward

  • The token-level reward combines target-class salience gain with a KL-divergence penalty that constrains the conditional policy relative to the unconditional policy.The KL penalty is weighted by β; the method uses an off-policy strategy to reuse unconditional state-action trajectories for multiple updates.Source: §2.2.1 · pp. 3–4
  • Class-specific salient lexicons are built by ranking words using a geometric mean of class-conditioned and word-conditioned frequencies, then selecting the top N.The authors describe this frequency-based approach as simple and efficient, particularly when little training data limits discriminator performance.Source: §2.2.1 · p. 3

2.2.2 Policy Gradient

  • Policy updates use temperature T to control sampling and sum normalized reward gradients over k steps, with k setting the strength of generation control.The update follows a classical SGD rule; η is the learning rate and θc parameterizes conditional hidden states.Source: §2.2.2 · p. 4
  • A dynamic β adjusts the KL penalty against target divergence σ, trading resemblance to unconditional generation against greater room for reinforcement-learning guidance.After optimization, the conditional policy decodes tokens from hidden states shifted toward the specified class.Source: §2.2.2 · p. 4, §2.2.1 · p. 4

3 Tasks & Datasets

The evaluation covers three tweet-classification datasets of differing sizes and label structures, selected to test Data Boost in challenging and low-resource settings. Data are split with class distributions preserved, and BERT performance is reported across repeated experiments.

3 Tasks & Datasets

  • The evaluation uses offense detection, sentiment analysis, and irony classification datasets with approximately 80k, 17k, and 3k examples and four, three, and two classes, respectively.The paper characterizes offense detection and irony classification as low-resource tasks and notes sentiment analysis can overfit with extremely limited training data.Source: §3 · p. 4
  • Figure 2 reports BERT Macro-F1, or F1 for the binary task, with and without Data Boost, averaged over five repeated experiments.The three panels cover offensive language detection, sentiment analysis, and irony detection on tweets.Source: Figure 2 · p. 5
  • Each dataset is split 80% for training and 20% for testing, with the original class distributions maintained across experiments.Samples longer than 30 tokens are filtered out, around 2% of data on average.Source: §3 · pp. 4–5

4 Experiments6

The experiments assess whether Data Boost improves classification, preserves the quality of boosted data, and works across classifier architectures. Results cover three tasks and compare performance across training-data fractions and augmentation ratios.

4.1 Does Data Boost Improve Performance?

  • 12.4% (Offense), 9.1% (Sentiment), and 8.8% (Irony) absolute F1 increases were achieved in the lowest-resource settings tested.Those settings used 1% of original training data for Offense and Sentiment, and 10% for Irony.Source: §4.1 · p. 5

4.2 Does Boosted Data Resemble the Original?

  • In Offense Detection, generated sentences mostly matched the original data distribution in a t-SNE visualization.The visualization used 400 original and generated sentences, with 100 per class.Source: Figure 3 · p. 5, §4.2 · p. 6
  • With 25% original data and 75% generated samples, F1 decreased by only 0.06 absolute versus using 100% original data.Perplexity did not substantially increase at higher boosting ratios.Source: §4.2 · p. 6, Table 3 · p. 6

4.3 Is Data Boost Classifier-Agnostic?

  • Data Boost generally improved all five tested classifiers by 1% to 13% absolute across the reported settings.The classifiers included CNN, Bi-LSTM with attention, Transformer, BERT, and XLNet.Source: Table 2 · p. 6, §4.3 · p. 7

5 Comparison with Related Work

The paper contrasts Data Boost with token-level, synonym-based, and conditional-generation approaches. It reports that Data Boost outperforms other augmentation methods in most experiments, while generating sentences from scratch with GPT-2 and reinforcement learning.

5 Comparison with Related Work

  • Data Boost outperformed other text augmentation methods in the majority of experiments, using GPT-2 to generate samples from scratch with reinforcement learning rather than replacement.The comparison covered six prior methods and used BERT classification and perplexity evaluations.Source: §5 · p. 7
  • CTRL and BART use large-scale conditional LMs, while PPLM perturbs vocabulary distributions during decoding; these approaches had not been explored for text augmentation.The paper identifies fine-grained reinforcement-learning control as especially helpful with multiple labels.Source: §5 · pp. 7–8

6 Human Evaluation

Human evaluation compared original, vanilla GPT-2, and Data Boost samples on label agreement and readability. Data Boost samples generally received higher label-agreement ratings than vanilla GPT-2 and had readability similar to both comparison sources.

6.1 Experimental Design

  • The evaluation involved 178 U.S. adults randomly assigned to assess one of three tasks: irony, sentiment, or offense detection.Participants rated label agreement and readability on seven-point scales after reading samples.Source: §6.1 · p. 8, §6.2 · p. 8

6.2 Procedures

  • Participants read original, vanilla GPT-2, and Data Boost samples without initially being told their labels or versions.After reading, they were shown each sample's actual label and version before rating label agreement and readability.Source: §6.2 · p. 8

6.3.1 Label Agreement

  • Data Boost samples received higher label-agreement scores than vanilla GPT-2 in eight of nine classes, with five differences statistically significant.No significant differences from original samples were found except for the spam and normal classes in Offense Detection.Source: §6.3.1 · pp. 8–9

6.3.2 Readability

  • Readability differences were not significant for six classes; for neutral, abusive, and hateful labels, Data Boost and vanilla GPT-2 samples were rated more readable than originals.The authors note that original samples were generally noisy tweets.Source: §6.3.2 · p. 9, Table 6 · p. 9

7 Limitations

Data Boost's gains can be marginal for classes that are difficult to model with lexical features; syntactic features may matter where token-by-token guidance struggles to compose meaningful augmentation.

7 Limitations

  • F1 increased by 1.3%, 0.9%, and 0.7% for metaphor detection, sarcasm classification, and formality detection, respectively.These results were observed in the extreme-scarcity setting of boosting 1% of the original data to 80%.Source: §7 · p. 9
  • The authors suggest syntactic features may play a role in metaphor, sarcasm, and formality, where extracting explicit lexical features was difficult.They identify token-by-token generation guidance as a challenge for composing meaningful augmentation in these cases.Source: §7 · p. 9
  • The authors expected Data Boost to help most in extreme data scarcity, tested by boosting 1% of the original data to 80%.Source: §7 · p. 9

8 Conclusion

Data Boost uses reinforcement learning with an off-the-shelf language model to guide conditional text generation. The authors report improved, classifier-agnostic performance and gains over several prior augmentation methods, while identifying richer syntactic and positional guidance as future work.

8 Conclusion

  • Data Boost improves classification performance, is classifier-agnostic, and surpasses several prior augmentation methods across three diverse tasks.Source: §8 · p. 9
  • The framework uses off-the-shelf GPT-2 and reinforcement learning to guide conditional generation toward a target class.Source: §8 · p. 9
  • Future work proposes adding syntactic and position features to the reward function to support augmentation of more diverse text types.Source: §8 · p. 9

Related papers

Loading 2012.02952v1…