Source-linked AI summary
AEDA: An Easier Data Augmentation Technique for Text Classification
Akbar Karimi, Leonardo Rossi, Andrea Prati
TL;DR
Text classification often has limited labeled data, while some augmentation methods are complicated or risk information loss. The paper proposes AEDA, which inserts punctuation without deleting or substituting words, and reports that it outperforms EDA across five datasets. The experiments also examine how augmentation quantity affects gains and acknowledge dataset-split differences from the baseline.
Problem
Text-classification applications often lack sufficient labeled data, while existing augmentation methods can be complicated or risk information loss and label changes.
Method
AEDA randomly inserts punctuation marks into input sequences while preserving the original input information and word order.
Results
AEDA outperforms EDA on all five utilized datasets across the reported text-classification experiments.
Takeaways & Limitations
AEDA provides an extremely simple punctuation-based augmentation technique that requires no extra data and improves performance relative to EDA in the reported settings.
Takeaways & Limitations
Comparisons with Wei and Zou (2019) may differ because some datasets required random train/test splits and had different sizes.
Abstract
from arXiv · showhide
This paper proposes AEDA (An Easier Data Augmentation) technique to help improve the performance on text classification tasks. AEDA includes only random insertion of punctuation marks into the original text. This is an easier technique to implement for data augmentation than EDA method (Wei and Zou, 2019) with which we compare our results. In addition, it keeps the order of the words while changing their positions in the sentence leading to a better generalized performance. Furthermore, the deletion operation in EDA can cause loss of information which, in turn, misleads the network, whereas AEDA preserves all the input information. Following the baseline, we perform experiments on five different datasets for text classification. We show that using the AEDA-augmented data for training, the models show superior performance compared to using the EDA-augmented data in all five datasets. The source code is available for further study and reproduction of the results.
1 Introduction
Text classification often lacks sufficient labeled data, motivating data augmentation. AEDA addresses information loss and complexity concerns by inserting punctuation while preserving word order and input information.
- 1 Introduction: Insufficient labeled data can limit text-classification training, making data augmentation a potential solution for improving machine-learning performance.The paper situates this problem across applications including sentiment analysis, toxicity detection, and question answering.
- 1 Introduction: Figure 1 averages results from five seeds after adding 9 augmented sentences to the original training set for each of five text-classification tasks.It compares original, EDA-generated, and AEDA-generated data.
- 1 Introduction: Deep-language-model augmentation can be complicated, while deletion and substitution may lose information, change labels, and mislead the network.These concerns motivate a simpler augmentation strategy that avoids modifying words.
- 1 Introduction: AEDA inserts various punctuation marks into the input sequence while preserving all input information.The method is presented as an extremely simple approach to data augmentation.
- 1 Introduction: AEDA keeps word order intact while shifting words to the right, and the authors report that it helps models avoid overfitting.The reported overfitting result is associated with Figure 1.
2 Related Work
Prior augmentation methods modify words, use back-translation, inject noise, or rely on pretrained models and auxiliary mechanisms. AEDA instead uses punctuation, requires no extra data, and is reported to outperform EDA across model types.
- 2 Related Work: Earlier methods augment input sequences through word substitution, back-translation, contextual language models, or reinforcement learning.These approaches span machine translation, tweet classification, and other NLP settings.
- 2 Related Work: Data noising methods may replace words or insert placeholders, whereas AEDA inserts punctuation characters that commonly occur in sentences.The comparison distinguishes AEDA’s perturbation choice from related noising strategies.
- 2 Related Work: Many related approaches use auxiliary data or complicated language models to produce augmented data.The paper contrasts this requirement with AEDA’s simpler setup.
- 2 Related Work: AEDA needs no extra data and shows superior performance to EDA in RNNs, CNNs, and BERT.The comparison covers both simpler and deeper model families.
3 AEDA Augmentation
AEDA creates augmentations by randomly selecting how many punctuation marks to insert, where to insert them, and which marks to use, while limiting noise.
- 3 AEDA Augmentation: AEDA randomly selects an insertion count between 1 and one-third of the sequence length.This guarantees at least one insertion while avoiding too many punctuation marks.
- 3 AEDA Augmentation: It randomly chooses as many sequence positions as the selected insertion count.Each selected position receives one punctuation mark.
- 3 AEDA Augmentation: Each insertion mark is randomly selected from {".", ";", "?", ":", "!", ","}.The paper notes that excessive noise might negatively affect the model and leaves this for future investigation.
4 Experimental Setup
The experiments reuse the baseline’s codebase, models, and five datasets, with reconstructed train/test splits and repeated runs to compare augmented and original data.
- 4 Experimental Setup: The implementation uses the same codebase as Wei and Zou (2019) without model changes and runs on a GeForce RTX 2070 GPU with 8 GB of memory.This setup is intended to support comparison with the baseline.
- 4 Experimental Setup: The study evaluates SST-2, CR, SUBJ, TREC, and PC, covering sentiment, reviews, subjectivity, question classification, and pros-and-cons data.These are the same five datasets used by the baseline.
- 4 Experimental Setup: Because the baseline’s train and test sets were unavailable, the authors shuffled collected data and divided it into similarly sized splits.CR reviews from three cited sources were combined and converted to binary classes using sentiment annotations.
- 4 Experimental Setup: The experiments use the baseline’s RNN and CNN implementations to compare EDA-, AEDA-, and original-data training fairly.The models are kept consistent across augmentation conditions.
5 Results
Across five text classification datasets, AEDA consistently improves performance over EDA and original training data, with larger gains at smaller training sizes.
- The comparisons use 16 augmentations for Table 1 and 9 augmentations for Figure 2, with scores averaged over 5 runs.
- AEDA boosts performance across all datasets, while EDA can help small datasets but may hurt performance on larger ones.
- 3.2% average absolute improvement occurs with 500 training sentences, compared with 0.5% using the full dataset.
- Across training-set fractions, AEDA outperforms EDA in all tasks and improves over the original data.
6 Ablation Study
The ablation studies examine augmentation count, random-seed variation, and deep-model performance. AEDA improves results with both RNN and BERT models, while gains depend on dataset size and augmentation count.
- Number of Augmentations: One augmentation improves performance by 1.5% to 2.5% across all dataset sizes.
- Number of Augmentations: With more augmentations, the smallest dataset gains almost 4%, whereas the full dataset gains 1%.
- Number of Augmentations: Middle-sized datasets show gains of 2% to 2.5% as augmentations increase.
- Random Initialization: Experiments across 21 seeds on CR and TREC show a trend similar to the averages from 5 seeds.
- Using AEDA with Deep Models: Adding one augmentation per training sample improves BERT performance by 0.66% on SST2 and 0.2% on TREC.
7 Discussion
The authors report discrepancies with prior EDA results, partly because datasets required different train/test splits and had different sizes. They kept other comparison settings consistent.
- Results differ from Wei and Zou (2019), especially regarding EDA’s effect on model performance.
- Some datasets lacked ready-made train and test splits, so the authors randomly divided them.
- Different dataset sizes may produce different results.
- Comparisons used the same library, source code, train/test sets, augmentation count, runs, batch size, and learning rate.
8 Conclusion and Future Work
The paper proposes an easy punctuation-based augmentation technique for text classification and finds that it outperforms EDA across five datasets. Future work will examine which punctuation marks to use, how many to insert, and where to place them.
- AEDA is an easy data augmentation technique for text classification tasks.
- AEDA uses punctuation marks and outperforms EDA on all five utilized datasets.EDA includes random deletion, insertion, and substitution of words.
- Future work will investigate which punctuation marks to add or discard and how many to use.
- Future work will test whether punctuation should be inserted randomly or at more effective positions.
9 Supplementary Material
The supplementary material provides examples of AEDA-augmented data and statistics for the datasets used in the experiments.
- 9.1 Example Augmentations: Table 3 contains examples of augmented data produced with AEDA.
- 9.2 Benchmark Datasets: Table 4 reports statistics for the utilized datasets.
- 9.2 Benchmark Datasets: The reported dataset statistics include class counts, average sentence length, training samples, test samples, and unique words.