Can We Achieve More with Less? Exploring Data Augmentation for Toxic Comment Classification

• Data scarcity still limits high-accuracy machine-learning classifiers, despite advances in algorithms and computing hardware.Source: §1 · p. 1
• The study examines whether data augmentation combined with machine-learning methods can produce high-accuracy classifiers from small datasets.Source: §1 · p. 1
• The authors develop a toxic-comment classifier intended to help moderators address online harassment and cyberbullying.They also aim to reduce annotators’ exposure to large, graphic datasets.Source: §1 · p. 1

• Computer-vision augmentation uses image transformations and mixing to enlarge scarce datasets, and models can benefit when useful information remains extractable.Source: §2 · p. 2
• In NLP, context makes general language transformations difficult because synonyms may be unavailable or change a sentence’s meaning.Prior approaches include context-based replacement, latent-vector-based replacement, Backtranslation [19], and bag-of-words augmentation [25].Source: §2 · p. 2
• The study compares Easy Data Augmentation [25] and Backtranslation [19] across learning methods and analyzes their effects.Source: §2 · p. 2
• The authors report that augmentation choice depends on the model’s hypothesis space and that gains diminish as model expressiveness increases.Source: §2 · p. 2

3.1 Dataset

• The dataset contains approximately 158,000 Wikipedia comments with six binary hate-speech labels, simplified to toxic versus non-toxic for this task.Preprocessing removes special characters and corrects spelling of key words.Source: §3.1 · p. 2

3.2 Training Structure

• The authors reserve a test set, use 5% of the training set as the small-set baseline, and treat the full training set as the oracle.Augmentation is applied to the small training set for comparison with different learning algorithms.Source: §3.2 · p. 2

3.3 Experimental Setup

• The experimental pipeline preprocesses text, compares LR, SVM, and Bi-LSTM on small-set and augmented data, and uses TF-IDF for LR and SVM and GloVe embeddings for Bi-LSTM.EDA expands the training set tenfold and Backtranslation doubles it; Figure 1 depicts the toxicity-classification pipeline.Source: §3.3 · pp. 2–3, Figure 1 · p. 3

3.4 Evaluation Metrics

• Because only 10% of comments are toxic, the study evaluates models with Recall and F1, which account for false positives and false negatives.Source: §3.4 · p. 3

4 Learning Algorithms

• The study compares three learning algorithms of varying complexity under different data-augmentation conditions.Source: §4 · p. 3

4.1 Logistic Regression (LR)

• Logistic regression maps observations to class probabilities with a logistic sigmoid and is included as a simple, commonly used classifier.Source: §4.1 · p. 3

4.2 Support Vector Machines (SVM)

• SVM classifies labeled data using a maximum-margin hyperplane and provides a more expressive, interpretable comparison with LR and Bi-LSTM.Source: §4.2 · pp. 3–4

4.3 Bidirectional LSTM (Bi-LSTM)

• Bi-LSTM reads text forward and backward, combining both representations to capture token sequences that LR and SVM’s bag-of-words models do not.Source: §4.3 · p. 4

5 Data Augmentation Algorithms

• The study focuses on Easy Data Augmentation and backtranslation, two popular data-augmentation methods, and compares their effects across learning methods.Source: §5 · p. 4

5.1.1 Algorithm [25]

• EDA applies synonym replacement, random swap, random insertion, or random deletion to generate variants of a training sentence.The paper describes synonym replacement as a way to introduce words while retaining meaning; swap and insertion perturb token order or placement, and deletion is intended to reduce overfitting.Source: §5.1.1 · p. 4

5.1.2 Implementation Details

• EDA scales the number of tokens modified with sentence length, using α for replacement, swap, and insertion and as the deletion probability.Four operation functions are selected uniformly at random to generate an instance; the paper also allows users to choose how many augmented sentences to generate per sentence.Source: §5.1.2 · pp. 4–5

5.2.1 Algorithm

• Backtranslation translates a training sentence into an intermediate language and back, adding translation noise while producing a semantically similar rephrasing.Figure 2 presents the backtranslation pipeline.Source: §5.2.1 · p. 5, Figure 2 · p. 5

5.2.2 Implementation Details

• The implementation uses the paid Google Translate API and Spanish, French, Hindi, and German as intermediate languages.Source: §5.2.2 · p. 5

6 Results & Discussion

• Across all four methods, EDA improved average F1 by 3% and Recall by 6.3%, while backtranslation improved F1 by 2.3% and Recall by 6%.Source: §6 · p. 6, Table 3 · p. 6, Table 4 · p. 6

6.1 Backtranslation vs EDA

• EDA and backtranslation raised top-feature importance to full-dataset oracle levels; LR and SVM gained substantially from both, while Bi-LSTM improved more with backtranslation.Source: §6.1 · p. 6
• LR and SVM received their largest F1 and Recall boosts from combining backtranslation with EDA, whereas Bi-LSTM received its largest boost from backtranslation alone.Source: Table 3 · p. 6, Table 4 · p. 6, §6.1 · p. 7

6.2 Effect of Different intermediary language for Backtranslation

• The four intermediary languages were selected to contrast morphology and translation-resource characteristics, including Hindi's different etymology and German's extensive translation work.Source: §6.2 · p. 7
• Among Spanish, French, Hindi, and German, Spanish backtranslation produced the best results, with Hindi close behind.Source: §6.2 · p. 7

7 Error Analysis

• In Baseline+Backtranslation (ES), logistic regression assigns “you” low toxicity like hurtful words, possibly because they co-occur in toxic comments; a similar issue appears for Bi-LSTM with EDA.Source: §7 · p. 8
• Words such as “wasted” can be misclassified because they occur with toxic terms or have both toxic and non-toxic meanings, leaving contextual disambiguation unresolved.Source: §7 · p. 8

8 Conclusion and Future Work

• EDA is more favorable for bag-of-words algorithms, while Backtranslation suits models that account for semantic structure.The paper also reports using feature-importance insights to explain how the techniques modulate data and improve performance.Source: §8 · p. 8
• Future work will explore abundant unlabeled data, including positive-unlabeled and unsupervised learning, to make classifiers more robust.Source: §8 · p. 8
• The study's stated social impact includes toxic-comment detection and classification and potentially reducing the amount annotators may need to label.Source: §8 · p. 8

A Plots and Figures

• Figures 4–6 show feature importance for Logistic Regression, SVM, and Bi-LSTM, respectively.Source: Figure 4 · p. 10, Figure 5 · p. 10, Figure 6 · p. 11