Source-linked AI summary
Data Augmentation via Dependency Tree Morphing for Low-Resource Languages
Gözde Gül Şahin, Mark Steedman
TL;DR
Low-resource languages have not seen significant improvement from neural methods, with sparse training labels identified as a general reason for low scores. This paper applies dependency-tree cropping and root-centered rotation to low-resource training data, finding POS-tagging gains over non-augmented data for most languages, especially those with rich case marking.
Problem
Neural methods have not significantly improved results for low-resource languages, where sparse training labels are identified as a general reason for low scores.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The authors state that the meaning-preserving methods can be used for tasks such as semantic role labeling, sentiment analysis, and text classification.
Source: §5 · p. 4The study measures basic augmentation techniques rather than optimizing them, and its evaluation is limited to POS tagging.
Source: §6 · p. 5
Abstract
from arXiv · showhide
Neural NLP systems achieve high scores in the presence of sizable training dataset. Lack of such datasets leads to poor system performances in the case low-resource languages. We present two simple text augmentation techniques using dependency trees, inspired from image processing. We crop sentences by removing dependency links, and we rotate sentences by moving the tree fragments around the root. We apply these techniques to augment the training sets of low-resource languages in Universal Dependencies project. We implement a character-level sequence tagging model and evaluate the augmented datasets on part-of-speech tagging task. We show that crop and rotate provides improvements over the models trained with non-augmented data for majority of the languages, especially for languages with rich case marking systems.
1 Introduction
The paper addresses the limited performance of neural NLP methods for low-resource languages, where sparse training labels are identified as a reason for low scores. It proposes dependency-tree cropping and rotation, and reports that both improve POS tagging over non-augmented data for most tested languages, especially those with rich case marking.
1 Introduction
- Neural methods have not shown significant improvement for low-resource languages, where sparse training labels are a generally accepted reason for low scores.For these languages, feature-engineered statistical models still perform better than neural models.Source: §1 · p. 1
- Comparable text-augmentation techniques are less obvious for NLP, and only a small number of studies address NLP augmentation.The passage gives examples in text classification and machine translation.Source: §1 · p. 1
- The paper proposes multilingual dependency-tree cropping and root-centered rotation, adapting image-augmentation ideas to form smaller sentences and synthetic ones.The target languages have small training datasets and are often from families with extensive case marking and relatively free word order.Source: §1 · p. 1
- Crop and rotate improve POS tagging over non-augmented data for most tested languages, especially languages with rich case marking.The evaluation uses low-resource language training sets from the Universal Dependencies project.Source: §1 · pp. 1–2
2 Method
The method generates training examples by cropping dependency links around focused sentence parts or rearranging marked subject/object fragments around the dependency root. A character-level sequence tagger evaluates these examples for POS tagging.
2 Method
- Cropping removes dependency links outside a chosen focus and its subtree to form smaller sentences, preserving local syntactic tags despite possible sentence-level semantic shifts.The examples focus on the subject or either object governed by the predicate.Source: §2 · p. 2
- Rotation rearranges marked subject/object fragments around the dependency root, a strategy suited to languages with extensive case marking and flexible word order.The paper warns that rotation would mostly introduce noise for analytical languages such as English, which lack object/subject marking and have stricter word order.Source: §2 · p. 2
- The authors use a character-level sequence tagger to address rare words and learn morphological regularities in the morphologically rich target languages.Character sequences are composed into word embeddings, which are passed to another bi-LSTM; bidirectional states are mapped to labels and the highest-probability label is assigned.Source: §2 · pp. 2–3
- Training minimizes the negative log-likelihood of the POS-label sequence against gold labels.For each sentence, the model produces a POS tag for each token.Source: §2 · p. 2
3 Experiments and Results
The experiments use low-resource UD v2.1 treebanks to compare augmented training data with original data under a shared character-level tagging setup. Augmentation generally improves results, with gains and differences varying across language groups and treebank conditions.
Experiments and Results
- The experiments use UD v2.1 universal POS tags and include languages with fewer than 120K tokens, excluding those without standard splits and with fewer than 5K tokens.A one-layer bidirectional LSTM setup is used for character composition and POS tagging, with hidden size and character embedding size of 200.Source: §3 · p. 3
- Cropping targets root-linked items with labels of interest; the method limits reordering to first-level chunks and randomly selects n of (n+1)! possible rotations.Each operation is applied to a sentence with probability p; the method treats ungrammatical generated sentences as noise.Source: §3 · p. 3
- With some minor exceptions, augmentation benefits nearly all 20 languages; the largest improvements occur for Irish and Lithuanian, which have the lowest baseline scores and smallest training sets.The Irish dataset has an unbalanced split: 121 training trees and 454 test trees.Source: §3 · p. 3
- −0.35 is the Pearson correlation coefficient between treebank size and improvement from augmentation.The authors also note that tagging results depend on treebank source and language typology.Source: §3 · p. 3
- Both techniques improve scores for Baltic and Slavic languages, while rotation performs slightly better than cropping for Slavic languages and Germanic languages show no repeating gain.The authors attribute the Germanic pattern to a lack of necessary markers and describe Old Church Slavic as an exception because its Bible-derived text contains longer, passive sentences.Source: §3 · pp. 3–4
- Both augmentation techniques improve Uralic and Turkic results, and Tamil’s eight case markers are offered as an explanation for improved augmented-model accuracy.The authors expected a similar Telugu result, but its treebank consists entirely of grammar-book sentences that may lack expressive diversity.Source: §3 · p. 4
4 Related Work
Related work describes NLP augmentation through sentence simplification, word replacement, and synthetic data generation. These approaches have been applied to semantic role labeling, neural models, and semantic parsing.
Related Work
- Vickrey and Koller (2008) used transformation rules to simplify sentences, reporting improved semantic role labeling results when simplified sentences enriched training data.Source: §4 · p. 4
- Zhang et al. (2015) replaced a randomly chosen word with a randomly chosen thesaurus synonym and reported improved test scores for a large neural model trained on augmented data.Source: §4 · p. 4
- Jia and Liang (2016) generated semantic-parsing examples by sampling from an induced grammar, while Fadaee et al. (2017) generated sentence pairs containing selected low-frequency words.Source: §4 · p. 4
5 Discussion
The discussion presents the augmentation methods as meaning-preserving for most tested languages and potentially applicable to several NLP tasks. The evaluation uses POS tagging to make the method’s impact easier to measure and as an indicator for other structured prediction tasks.
Discussion
- The proposed methods are described as meaning-preserving for most tested languages and potentially applicable to semantic role labeling, sentiment analysis, and text classification.Source: §5 · p. 4
- POS tagging was chosen as a simpler evaluation task because measuring impact becomes harder as systems and tasks grow more complex, and POS tags are crucial features for higher-level NLP tasks.Source: §5 · pp. 4–5
- Figure 2 frames a comparison between treebank size and gain by augmentation.Source: Figure 2 · p. 5
- The authors focused on which augmentation technique helps which language, rather than selecting one best model, so they did not use development sets to choose an augmentation model.Source: §5 · p. 5
6 Conclusion and Future Work
The paper concludes that dependency-tree cropping and rotation improve POS tagging over non-augmented data, especially for languages with rich case marking. It presents these as basic techniques and proposes language-aware refinements and evaluation on additional NLP tasks.
6 Conclusion and Future Work
- The study evaluates the techniques on POS tagging across low-resource languages from various language families.Source: §6 · p. 5
- Languages with rich case-marking systems benefit from both cropping and rotation, with rotation yielding slightly better accuracies.For these languages, rotation increases data variance relatively more.Source: §6 · p. 5
- Both techniques provide substantial improvements over non-augmented data when only a tiny training dataset is available.Source: §6 · p. 5
- The evaluation measures basic augmentation techniques rather than creating the best method; proposed refinements include language-aware rotation and testing on other NLP benchmarks.Source: §6 · p. 5