Text Data Augmentation Made Simple By Leveraging NLP Cloud APIs

• The “Big Data Wall” is the shortage of data needed to train deep neural networks, posing a challenge for minority-language communities with small corpora and organizations competing with GAFAM.Source: §1 · p. 2
• The amount of data required depends on task complexity, with complex tasks requiring more training data.Source: §1 · p. 2
• Text data augmentation amplifies existing data by creating new examples while keeping meaning invariant.Source: §1 · p. 2
• This work focuses on sentence augmentation, although the techniques are described as applicable to all types of text.Source: §1 · p. 2

• Text augmentation has been less widely used than augmentation in vision and speech, with lexical substitution described as the only widespread technique until recently.Source: §2 · p. 3
• The discussion lists symbolic, discrete, compositional, sparse, hierarchical, noisy, exceptional, and ambiguous properties as challenges for processing text.Source: §2 · p. 3
• Practitioners commonly use hand-crafted rules such as noise injection and regular expressions to augment text.Source: §2 · p. 3
• Much research focuses on generative models and end-to-end learning, including VAEs, GANs, machine translation, and neural lexical replacement.Source: §2 · p. 4
• The paper positions its engineering work as a short-term practical approach distinct from long-term fundamental research.Source: §2 · p. 4

3.1 Basic Assumption

• The work aims to adapt computer-vision-style data transformations to NLP preprocessing and use available NLP Cloud APIs.Source: §3.1 · p. 5

3.2 Attempt to formalize data augmentation

• The paper also states that augmented data should follow a statistical distribution similar to the original data.Source: §3.2 · p. 5
• The authors identify human-perceived plausibility as a constraint: amplified data should not be distinguishable from original data.Source: §3.2 · p. 5
• In supervised classification, augmentation can transform input features while retaining a simple target label.Source: §3.2 · p. 6
• Data augmentation is framed as a semantically invariant transformation: it creates examples from existing data while preserving meaning.Source: §3.2 · p. 6
• For supervised learning, allowed transformations must not change the class label of generated examples.Source: §3.2 · p. 6
• Combining transformations can produce more distant paraphrases while retaining meaning, but each step risks greater semantic drift.Source: §3.2 · p. 7

3.3 Technique 1 - "Textual Noise" Injection

• Textual noise consists of letter changes, additions, or deletions, changes of case, and punctuation modifications.Source: §3.3 · p. 7
• The authors distinguish light from strong textual noise, identify the light-versus-chaos boundary as an open question, and suggest studying noise amount and type.Source: §3.3 · pp. 7–8

3.4 Technique 2 - Spelling Errors Injection

• Spelling-error injection uses a list of common English misspellings, and the authors identify gradual increases in error rates as a question for experimental study.Source: §3.4 · p. 8

3.5 Technique 3 - Word Replacement using thesaurus

• Replacement candidates vary by part of speech, with verbs described as particularly challenging; the method favors hyperonyms and avoids hyponyms.Source: §3.5 · p. 8
• Lexical replacement is difficult because words can have multiple meanings and therefore different synonym sets.Source: §3.5 · p. 9
• The authors choose replacement synonyms by comparing word context with synset definitions and examples, then selecting the most similar sense.Source: §3.5 · p. 9

3.6 Text augmentation by paraphrases generation

• Paraphrases express the same semantic content in an alternative surface form, ideally diverging in form while remaining natural and fluent.They may occur at lexical, phrasal, or sentence level.Source: §3.6 · pp. 9–10

3.7 Technique 4 - Paraphrases generation using regular expressions

• Surface transformations ignore syntax and use simple pattern-matching rules; the paper prefers them for their computational simplicity and efficiency.The section notes that many rules may be partly generalized by factoring their behavior.Source: §3.7 · p. 10
• Contractions and their expansions can preserve meaning only when transformations preserve ambiguity rather than resolving it without contextual justification.For example, “he's” may represent “he is” or “he has”; choosing either expansion without context risks misinterpretation.Source: §3.7 · pp. 10–11
• Many surface transformations depend on the language, and changes requiring grammatical agreement or deeper manipulation cannot be done with simple surface regexes.Such changes require syntactic-level processing.Source: §3.7 · pp. 10–11

3.8 Technique 5 - Paraphrases generation using syntax trees transformations

• The generator parses an input sentence into a dependency tree, transforms that tree using a transformation grammar, then generates a new surface form.The method requires a wide-coverage morphosyntactic parser that produces dependency trees.Source: §3.8 · p. 13
• The prototype uses SyntaxNet parsing through Google Cloud Natural Language API, with rule-based generator code translated from Prolog to Python.The transformation rules were built manually, guided by an established paraphrase typology.Source: §3.8 · p. 13, Comparison between PolyPhrases… · p. 14
• Examples of meaning-preserving tree transformations include switching active and passive voice, replacing a noun phrase with a pronoun, and withdrawing modifiers.For active-to-passive conversion, the example exchanges subject and object groups and modifies the verb before generating the new sentence.Source: Comparison between PolyPhrases… · p. 14

3.9 Technique 6 - Paraphrases generation using back-translation

• Back-translation translates a text from its original language to another language and then back, producing potential paraphrases.The process is applied sentence by sentence.Source: §3.9 · pp. 15–16
• Identical back-translations are rejected; non-identical candidates are retained only when their similarity to the original exceeds an empirically set threshold.The filter may reject good paraphrases because valid paraphrases are not always very similar to the original.Source: §3.9 · p. 16
• For rapid implementation and computational performance, the study measures similarity by word-count difference and sets the threshold at 25%.The authors note that the threshold could be fine-tuned with more experiments.Source: §3.9 · p. 16
• The experiments use online Google translation services rather than the authors' own neural machine translation models, which require considerable computing resources.Poor-quality back-translation is not semantically invariant.Source: §3.9 · pp. 15, 17

4.1 Tools

• The experiments use Google Cloud parsing and translation services, selected for robustness, scalability, ease of deployment, and cost.The toolchain also includes open-source Python libraries for machine learning, preprocessing, graphing, deep learning, and linguistic processing.Source: §4.1 · p. 17

4.2 Task selection - text polarity prediction

• The task is supervised prediction of positive or negative opinion polarity, evaluated on a dataset of 1,000 positive and 1,000 negative IMDB reviews.The experiments use the polarity dataset v2.0.Source: §4.2 · p. 17, §4.3 · p. 18
• The study aims to demonstrate feasibility and a positive augmentation effect, not superiority to the best approaches; its models may still overfit.The objective avoids optimal model tuning, which the authors describe as potentially time-consuming and computationally expensive.Source: §4.2 · p. 18

4.4 Experimental design

• The experimental baseline is performance on original, unaugmented data; scarce computational resources limit the study to proof of concept and leave detailed method exploration for future work.The augmentation pre-processing is followed by model training, with accuracy, training error, and F1 measures examined.Source: §4.4 · p. 18

4.5 Textual data augmentation process

• The augmentation process builds a paratext object to generate the requested number of variants, sampling combinations and recording them to avoid excessive repetition.For these feasibility experiments, each original text generated five variants, only a tiny part of all possible combinations.Source: §4.5 · p. 19

4.6 Models training process

• The tested neural architectures include MLPs, a one-dimensional CNN, LSTM, and biLSTM; models use early stopping without external embeddings, dropout, or hyperparameter tuning.The study averages between 15 and 30 executions and also uses 5-fold, sometimes 10-fold, cross-validation.Source: §4.6 · p. 20

5. RESULTS

• Augmentation combinations add datasets rather than applying one augmentation technique successively to the output of another.Results for successive applications were not available.Source: §5 · p. 20
• The baseline is performance on the original, unamplified data, and the single perceptron's threshold to exceed was around 75%.The baseline varies by neural-network architecture.Source: §5 · p. 21
• The reported accuracy results concern tested neural-network architectures and include single-perceptron and two-hidden-layer MLP labels.The supplied results identify these architecture labels but do not provide their complete table values.Source: §5 · p. 21
• 4.3% to 21.6% is the reported accuracy-increase range across all studied text-augmentation techniques.The accuracy results average 15 to 30 executions, depending on training duration.Source: §5 · pp. 20–21

biLSTM

• The graphs' error bars represent standard deviation.The passage says this presentation highlights the effect of augmentation techniques on accuracy.Source: biLSTM · p. 24
• With textual noise alone, LSTM performance rises from just above chance to more than ⅔ correct predictions.The passage characterizes the chance-level starting point as just a little better than 50%.Source: biLSTM · p. 24
• Recurrent networks, especially biLSTM, give the best results and make good use of combined augmented datasets.The combinations add datasets rather than applying augmentation techniques successively.Source: biLSTM · p. 24
• 2.5% is the reported improvement for XGBoost with augmented data, while recurrent networks—especially biLSTM—give the best results.XGBoost achieves 81% on the original data; the reported account says it improves only slightly with augmented data.Source: biLSTM · p. 24
• The neural networks were not fine-tuned; regularization used added data and early stopping only.No external word vectors, L1/L2 regularization, dropout, or hyperparameter adjustment was used.Source: biLSTM · p. 24

6.1 Randomness of the results

• Neural-network results can vary substantially across training runs, even with the same network and data.The discussion links variability to different training outcomes and notes that ensembles and cross-validation average across models or data subsets.Source: §6.1 · pp. 24–25

6.2.1 A software engineering dream comes true

• NLP cloud services are presented as available, robust, reliable, scalable, inexpensive, ready to use, and often available in many languages.The discussion describes the techniques' principal benefits as practical and software-engineering advantages.Source: §6.2.1 · p. 25

6.2.2 Comparison with emerging approaches

• The proposed techniques modify existing sentences through semantically invariant transformations rather than generating sentences ex nihilo.The discussion presents this as their main advantage over many emerging approaches.Source: §6.2.2 · p. 25

6.3 Drawbacks

• Some techniques require substantial computation and cloud infrastructure, and translation and parsing services introduce dependence on private providers.The experiments used Google Cloud Platform services for practical reasons.Source: §6.3 · pp. 25–26

6.3 The place of data augmentation in the data pipeline

• Augmentation may be the only remedy when data are insufficient to specialize a large generic pretrained model for a task.The discussion identifies this as a fairly common situation despite the potential use of transfer learning.Source: §6.3 · p. 26

6.4 Limits

• The evaluation covers one simple polarity-prediction task and was intended to show feasibility; limited GPU access capped amplification at five variants per original.Five variants represent only a tiny fraction of possible combinations.Source: §6.4 · pp. 26–27

7. Conclusion

• 4.3 to 21.6%: text augmentation increased accuracy in repeated 15- to 30-fold experiments with cross-validation on a simple standard supervised binary-classification task.The authors describe the work as an engineering endeavour conducted with limited computational resources.Source: §7 · p. 27
• The studied techniques include textual noise and spelling-error injection, thesaurus-based word replacement, and paraphrase generation using regular expressions, syntactic-tree transformations, and back-translation.Syntactic-tree transformations and back-translation use NLP Cloud APIs described as robust, scalable, and easy to use.Source: §7 · p. 27
• The authors conclude these practical techniques are likely to help practitioners, engineers, and researchers seeking solutions to the “Big Data Wall” in deep-learning NLP.The stated scope is a simple standard supervised binary-classification task.Source: §7 · p. 27