Source-linked AI summary
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
Jason Wei, Kai Zou
TL;DR
Text classification performance depends heavily on the size and quality of training data, while broadly practical NLP augmentation techniques remain limited. EDA addresses this gap with four simple text-editing operations and improves performance across five classification tasks, especially with smaller datasets.
Problem
High text-classification performance often depends on tedious-to-collect training data, while universal, practical NLP augmentation techniques have not been thoroughly explored.
Method
EDA applies synonym replacement, random insertion, random swap, and random deletion to generate augmented sentences without requiring a trained language model or external datasets.
Results
EDA provides substantial improvements across five benchmark classification tasks for both convolutional and recurrent models, with particularly strong gains on smaller datasets.
Takeaways & Limitations
EDA offers a simple baseline for NLP data augmentation and supports further investigation of universal or task-specific augmentation approaches.
Takeaways & Limitations
With full datasets, average performance gain was less than 1%, and improvements may be negligible when using pre-trained models.
Abstract
from arXiv · showhide
We present EDA: easy data augmentation techniques for boosting performance on text classification tasks. EDA consists of four simple but powerful operations: synonym replacement, random insertion, random swap, and random deletion. On five text classification tasks, we show that EDA improves performance for both convolutional and recurrent neural networks. EDA demonstrates particularly strong results for smaller datasets; on average, across five datasets, training with EDA while using only 50% of the available training set achieved the same accuracy as normal training with all available data. We also performed extensive ablation studies and suggest parameters for practical use.
1 Introduction
EDA addresses the limited practical use of existing NLP augmentation methods by introducing a simple, universal set of text-editing operations. It is evaluated across five benchmark classification tasks and reported to improve performance, especially on smaller datasets.
- Motivation: Existing NLP augmentation methods include back-translation, data noising, and language-model-based synonym replacement.These approaches are described as valid but costly to implement relative to their performance gains.
- Contribution: Table 1 presents sentences generated using synonym replacement, random insertion, random swap, and random deletion.The caption identifies the abbreviations SR, RI, RS, and RD for these operations.
- Contribution: EDA introduces four simple text-editing operations as universal data augmentation techniques for NLP.The paper names the method EDA, or easy data augmentation.
- Contribution: EDA is systematically evaluated on five benchmark classification tasks and reported to provide substantial improvements on all five, particularly for smaller datasets.The evaluation is intended to establish a practical baseline for NLP data augmentation.
2 EDA
EDA generates augmented training sentences by applying one of four simple operations: synonym replacement, random insertion, random swap, or random deletion. The amount of alteration scales with sentence length so longer sentences can absorb more noise while retaining their class label.
- Overview: For each training sentence, EDA randomly chooses and applies one augmentation operation.The operations are designed to generate additional training examples from existing sentences.
- Operations: Synonym replacement substitutes randomly selected non-stop words with randomly chosen synonyms.The number of replacements is n.
- Operations: Random insertion adds a random synonym of a randomly selected non-stop word at a random sentence position, repeated n times.The inserted synonym is derived from a word in the sentence.
- Operations: Random swap repeatedly exchanges the positions of two randomly selected words.The operation is performed n times.
- Operations: Random deletion removes each word independently with probability p.Unlike the other operations, deletion is controlled directly by p.
- Parameters: For synonym replacement, insertion, and swapping, the number of changed words is n=αl, while random deletion uses p=α.Here l is sentence length and α indicates the proportion of words changed; each original sentence produces n_aug augmented sentences.
3 Experimental Setup
The experiments evaluate EDA across five benchmark text-classification tasks using recurrent and convolutional neural networks. The study also varies the fraction of available training data to examine performance on smaller training sets.
- Datasets: EDA is evaluated on five benchmark tasks: SST-2, CR, SUBJ, TREC, and PC.These cover sentiment, reviews, subjectivity, question type, and pro-con classification.
- Models: The study uses both an LSTM-RNN and a convolutional neural network for text classification.The CNN implementation follows Kim (2014).
4 Results
EDA improves text-classification performance across five tasks, with larger gains when training data are scarce. Its operations all contribute, while augmentation strength and quantity require tuning to avoid damaging sentence identity or usefulness.
- Training Set Sizing: 88.6% average accuracy with EDA using 50% of available training data exceeded the 88.3% best average without augmentation using all data.
- Does EDA conserve true labels?: EDA’s augmented sentences closely surrounded same-label originals in latent space, suggesting that they generally conserved their original class labels.
- Ablation Study: EDA Decomposed: All four operations contributed to gains, but excessive alteration reduced performance by changing sentence identity, shuffling word order, or making sentences unintelligible.SR deteriorated at high α, RS declined at α≥0.3, and RD severely hurt performance when up to half the words were removed.
- Ablation Study: EDA Decomposed: α=0.1 appeared to be a cross-operation sweet spot, with improvements more substantial on smaller datasets.
- How much augmentation?: Generating many augmented sentences helped smaller training sets, whereas adding more than four per original sentence was unhelpful for larger training sets.The study varied naug across 1, 2, 4, 8, 16, and 32 and used the results to recommend practical parameters.
5 Comparison with Related Work
Prior augmentation studies often focus on translation or task-specific settings, making direct comparison difficult. EDA achieves gains comparable in magnitude to more complex methods while requiring less implementation effort.
- 3% gain in accuracy was reported on two datasets using a variational autoencoder and attribute discriminator.
- 0.5% gain on five datasets was reported for synonym replacement using a bidirectional language model.
- EDA yields results on the same order of magnitude as these methods without training a language model or using external datasets.
- Related-work comparisons use #datasets, reported Gain, and indicators for language-model or deep-learning requirements and external datasets.
6 Discussion and Limitations
EDA addresses the lack of standardized NLP augmentation by offering simple operations intended as a baseline. The authors frame it as a starting point for future methods, while noting limitations in performance and comparability.
- EDA was introduced to address the lack of standardized data augmentation in NLP compared with vision.
- The simple operations are intended to serve as a baseline for future investigation.
- The authors do not expect EDA to remain the go-to augmentation method and hope it inspires universal or task-specific approaches.
- Less than 1% average performance gain was observed when training with full datasets across the five classification tasks.
- EDA may provide negligible improvement with pretrained models, and comparisons with related work are highly non-trivial because models and datasets differ.
7 Conclusions
The paper concludes that simple data augmentation operations can improve text classification, especially with smaller datasets, although improvements can sometimes be marginal.
- Simple data augmentation operations can boost performance on text classification tasks.
- EDA substantially boosts performance and reduces overfitting when training on smaller datasets.
- The authors identify theoretical analysis of EDA operations as a direction for continued work.
9 Supplementary Material
The supplementary material documents implementation choices for augmentation, embeddings, CNNs, RNNs, and the evaluated datasets. It also points readers to code and dataset summary statistics.
- EDA synonym replacements and random insertions use synonyms generated with WordNet.
- The models use 300-dimensional GloVe word embeddings.
- CNN: The CNN uses 128 size-5 convolutional filters, global max pooling, a 20-unit ReLU dense layer, and softmax output.
- RNN: The RNN uses bidirectional LSTM layers with 64 and 32 cells, dropout p=0.5, a 20-unit ReLU dense layer, and softmax output.
- The supplementary material provides summary statistics for the five text classification datasets, including classes, sentence length, training and testing samples, and vocabulary size.
10 Frequently Asked Questions
EDA is designed as a simple, generalizable alternative to augmentation methods that often require costly model-based implementation. Its practical guidance covers implementation, likely benefits, and important usage boundaries.
- 10.1 Implementation: Synonym replacement uses WordNet as the synonym dictionary.The implementation is described as easy to download.
- 10.2 Usage: EDA may be less useful for sufficiently large datasets and for models pre-trained on massive datasets.These are presented as practical cases in which additional augmentation may not help much or may not be needed.
- 10.2 Usage: EDA uses simple techniques intended to generalize across a range of NLP tasks, while contextual augmentation, noising, GANs, and back-translation may require deep-learning models to generate sentences.The paper notes that alternative techniques may work better depending on the dataset, but often have higher implementation costs relative to expected performance gain.
- 10.2 Usage: EDA can potentially create mislabeled augmented examples when an operation changes a sentence’s class, although the authors report that performance harm was unlikely across five classification tasks.The paper still acknowledges that adverse effects are possible.
- 10.3 Theory: EDA may help prevent overfitting by adding noise and introducing new vocabulary through synonym replacement and random insertion.The authors state that the exact mechanism is difficult to identify and present these as two main reasons.
- 10.3 Theory: Synonym-based insertion is preferred because it is more likely to remain relevant to the context and preserve the sentence’s true label than inserting arbitrary words.The paper frames label preservation as a key requirement for data augmentation.