Source-linked AI summary
Low-resource Languages: A Review of Past Work and Future Challenges
Alexandre Magueresse, Vincent Carles, Evan Heetderks
TL;DR
Low-resource language research addresses the fact that most of the world’s languages remain understudied because NLP research focuses on only 20 of roughly 7,000 languages. This review examines over 60 papers and identifies dataset expansion, augmentation, transfer learning, linguistic similarity, robust models, and reliable evaluation as central directions.
Problem
Most NLP research focuses on 20 of the world’s roughly 7,000 languages, leaving the vast majority understudied as low-resource languages.
Method
The paper reviews over 60 low-resource-language papers and synthesizes their recent methods, achievements, and future research directions.
Results
The review finds that recent work primarily expands low-resource-language datasets while applying augmentation and transfer learning adapted to linguistic differences.
Takeaways & Limitations
Future research should improve data quality, exploit linguistic similarities, develop more robust learning models, and increase evaluation reliability.
Takeaways & Limitations
Speech-focused research remains constrained by heterogeneous training data involving varying noise, speakers, accents, and emotions.
Abstract
from arXiv · showhide
A current problem in NLP is massaging and processing low-resource languages which lack useful training attributes such as supervised data, number of native speakers or experts, etc. This review paper concisely summarizes previous groundbreaking achievements made towards resolving this problem, and analyzes potential improvements in the context of the overall future research direction.
1 Introduction
Most NLP research focuses on only 20 of the world’s 7000 languages, leaving the vast majority understudied. This paper reviews methods for low-resource languages and identifies promising future directions and unresolved questions.
- 20 of the world’s 7000 languages receive most NLP research attention, leaving the majority understudied.
- In this paper, low-resource languages are those for which statistical methods cannot be directly applied because of data scarcity.
- Supporting low-resource languages can open economic opportunities, help prevent language extinction, broaden access to original works, and support emergency response.
- The paper reviews recent methods applied to low-resource languages, highlights promising future directions, and identifies unsolved questions.
2 Related work
Earlier reviews of low-resource languages generally focused on specific NLP tasks or dated back decades. Related conferences and workshops have also promoted research in this area.
- Previous reviews targeted specific tasks including part-of-speech tagging, text classification, machine translation, and textual analysis.
- Conferences and workshops such as LREC, AMTA, and LoResMT have placed low-resource languages at the center of their activities.
3 The projection technique
Projection transfers annotations from a high-resource language to a low-resource language, supporting NLP tasks where target-language annotation is difficult or impossible. Its effectiveness depends on alignment quality, structural compatibility, and annotation consistency.
- The review organizes projection, resource collection, linguistic tasks, speech recognition, multilingual embeddings, translation, and classification across its main sections.
- Projection reuses annotations from a high-resource language for a low-resource language whose annotation is difficult or impossible to collect.
- Document-, sentence-, and word-level alignment support information retrieval, machine translation, and linguistic tasks, respectively.
- Word-level aligned corpora are difficult to collect, motivating increased efforts toward automatic alignment.
- One-to-one mappings are exceptional because source and target languages often have mismatching structures.
- Efficient annotation projection requires task-dependent structural closeness and consistency between the source and target annotations.
4 Resource collection
Resource collection for low-resource languages combines new dataset creation with raw-text alignment to higher-resource languages. The reviewed work spans multilingual alignment methods, from lexicon and sentence matching to document-level techniques, while highlighting remaining evaluation and complexity challenges.
- Low-resource information is collected by annotating raw text or gathering raw text and aligning it with a higher-resource language.
- REFLEX-LCTL and LORELEI released annotated corpora for 13 and 34 languages, with LORELEI integrating multiple annotation types in one framework.
- Speech corpora and South African news-headline datasets demonstrated what can be achieved with very limited data and informed future dataset collection.
- Social media, mobile applications, and governmental documents provide potential sources for generating textual content.
- Automatic alignment is time-consuming and requires language expertise, although aligned corpora underpin many low-resource-language techniques.
- Word-level alignment: Lexicon induction methods use dictionary composition, distributional context, cognate assumptions, or Boolean optimization, but pivot methods can fail with lexical variants and polysemy.
- Sentence-level alignment: Sentence-level alignment compares sentences using similarity scores derived from translation and n-gram matching, tf-idf cosine similarity, or multilingual embeddings with word mover distance.
- Document-level alignment: Sentence embeddings combined with relaxed Earth mover’s distance achieved state-of-the-art document alignment on low- and mid-resource languages.
5 Linguistic tasks
The review surveys linguistic tasks for low-resource languages, emphasizing projection, cross-lingual transfer, and methods that reduce dependence on target-language annotations. It also identifies unresolved challenges involving evaluation, source-language choice, language relatedness, and modeling assumptions.
- 5.1 Part-of-speech tagging: POS tagging ranges from unsupervised clustering and parent-language grounding to projected supervised models, including BiLSTM systems adapted with projection layers.Fang and Cohn’s formulations differ in whether hidden states are projected by matrix multiplication or an MLP.
- Future work: Future directions include transferring from multiple high-resource languages, jointly learning POS and dependencies, improving cross-domain NER, and relaxing assumptions about one morphological shift per paradigm.The review also highlights the need for stronger unsupervised POS metrics and a unified assessment of linguistic closeness.
- 5.2 Dependency parsing: Dependency parsing benefits from inferred context-sensitive clusters, multiple sources for grammatically similar languages, parameter sharing, and weighted or carefully selected zero-shot transfer.A weighted projection scheme achieved high UAS across more than 20 languages, while source selection remains an open question.
- 5.3 Named entity recognition, typing and linking: Named-entity recognition and typing commonly project annotations from high-resource languages, while embedding clustering can outperform CRF architectures and handle phrases.A sentence-aligned embedding method requires only sentence alignment and can support other classification tasks with imbalanced class frequencies.
- 5.4 Morphology induction: Morphology induction covers lemmatization and paradigm completion through alignment, trie-based methods, edit-based LSTMs, edit trees, paradigm merging, and transduction.The reviewed paradigm-generation approach performs candidate search, paradigm merging, and generation.
6 Speech recognition
Low-resource speech recognition exploits phones, syllables, and transfer from high-resource languages, while adapting model architectures to scarce target-language data. The reviewed MLP approach improves on common-phoneset transfer by replacing its final layer with a target-language-trained layer.
- Low-resource LVCSR identifies phones to derive message meaning while exploiting phones, syllables, and transfer from high-resource secondary languages.
- Multilayer perceptrons: Thomas et al. train a four-layer MLP on high-resource languages, replace its final layer with a target-language perceptron, then retrain on limited target data.
- Multilayer perceptrons: 30 percent more accuracy than the common-phoneset baseline is reported for the target-language MLP approach.
- Hidden Markov models: HMM design for low-resource speech recognition must consider topology and the scope of speech-modelling units, including phone, syllable, and word levels.
- Future work: Future speech-recognition work targets name and place entity detection and methods to filter and normalize heterogeneous data across noise, speakers, accents, and emotions.
7 Embeddings
Low-resource embeddings require alternatives to resource-intensive training, including shared multilingual representations, transfer across similar languages, and data augmentation. BPE increases cross-language subword overlap for transfer, while augmentation remains constrained by grammatical knowledge and semantic fidelity.
- Embedding training is resource-intensive and incompatible with low-resource data scarcity, motivating multilingual representations, structural transfer, and data augmentation.
- BPE segmentation: BPE segmentation breaks words into subwords and retains common roots, increasing vocabulary overlap between a high-resource parent language and a low-resource target.
- BPE segmentation: More shared tokens provide more overlapping words for aligning embeddings during transfer between parent high-resource and target low-resource languages.
- Data augmentation: Data augmentation creates additional sentences by replacing one word, but replacements must fit the sentence structure, such as replacing a verb with another verb.
- Data augmentation: Even when syntax is preserved, augmentation can alter meaning; this limits its use across low-resource domains and makes it more suitable for machine translation.
- Future work: Current augmentation approaches struggle with extremely low-resource languages because correctness requires precise knowledge of grammar, sentence structure, and translation pairs.
- Multilingual embeddings: Multilingual BiLSTM and multilingual BERT embeddings support zero-shot transfer, including across languages with different scripts and no lexical overlap.
8 Machine translation
Low-resource machine translation addresses scarce parallel data through transfer, multilingual, zero-shot, and graph-based methods. These approaches reuse information across languages, but transfer quality depends on language compatibility and model agreement.
- Motivation: NMT models require large translated or annotated corpora, which are scarce for less common languages and dialects.This data scarcity motivates specialized architectures and cross-lingual methods.
- Transfer Learning: Transfer Learning initializes a low-resource child model with embeddings from a high-resource parent NMT model.The parent is trained on a standard high-resource language pair before its embeddings are reused.
- Transfer Learning: Transfer Learning works best when parent and child languages share similar lexicon and grammar.An ordering model is proposed as one strategy for matching source and target sentence structures.
- Zero-Shot and Language-Model Methods: Zero-Shot Translation trains an encoder on multiple language pairs so it can translate between language pairs unseen during training.For example, English–French and French–Chinese training can support English–Chinese translation.
- Zero-Shot and Language-Model Methods: A target Language Model regularizes NMT decoding by selecting highly ranked NMT words validated by the LM, functioning as knowledge distillation.The LM teaches the NMT model about the target language, but disagreement between the models can still produce incorrect predictions.
- Multilingual Learning: Multilingual Learning uses a universal lexical representation and a mixture of language experts to share word- and sentence-level information across languages.The resulting universal sentence encoder transfers learned embeddings across language pairs.
- Multilingual Learning: Language Graph methods represent languages as vertices and translation pairs as weighted edges, then use distillation across multiple translation paths.Indirect paths can be useful when direct low-resource translation pairs have low accuracy.
9 Classification
Low-resource classification commonly adapts lighter models, enriches architectures with language-specific processing, and expands limited data. Future work emphasizes deeper study of morphology and grammar alongside broader use of existing techniques.
- Sentiment analysis: English-focused opinion-mining models perform poorly on low-resource languages with different grammar, unstructured text, and limited NLP resources.One response is to adapt less resource-dependent baseline models to a particular language.
- Sentiment analysis: Arabic sentiment analysis augments a recursive model with morphological tokenization, sentiment extraction, unsupervised pre-training, and phrase-structure parsing.These additions target language-specific representations and syntactic structure.
- Sentiment analysis: English and Urdu sentiment analysis uses lexicon-based, K-nearest-neighbor, Naive Bayes, decision-tree, and hybrid methods.The approaches combine manually classified sentiment words with machine-learning alternatives.
- Data expansion: Data expansion increases supervised classification resources through adversarial text distortion, feature extraction, and transfer learning from multilingual recognition systems.These methods add data or reusable representations to compensate for limited annotations.
- Miscellaneous: Low-resource readability classification can automate quality analysis in textbooks, while pattern recognition can address Uyghur vowel weakening and suffix-based morphology.The cited approaches use lexical, entropy-based, divergence-based, or syllable features.
- Future work: Future classification research is expected to study low-resource morphological and grammatical traits and apply existing classification methods.The field remains constrained by limited study and experience with these languages.
10 Discussion
The discussion identifies two recurring priorities: expanding datasets for diverse languages and developing a task-specific measure of linguistic closeness. Together, these address data scarcity and the difficulty of selecting languages for transfer or alignment.
- Discussion: The review highlights collecting datasets for more diverse languages and devising a closeness index for languages as recurrent desiderata.These priorities recur across the reviewed NLP tasks.
- Datasets diversity: New low-resource datasets can draw on news or social media, mobile applications for audio and annotations, governmental sources, and collaborative translation collections.Tatoeba 9 provides aligned sentences and translations in more than 350 languages.
- Closeness index for languages: Existing language-comparison resources support morphological and lexical comparison but do not capture grammatical differences.The ASJP word list covers more than 9500 languages and dialects.
- Closeness index for languages: The paper advocates a task-specific linguistic distance that combines morphological and grammatical aspects to guide transfer-language selection.The proposed index would choose an optimal source language for a given target language.
11 Conclusion
The review finds that recent low-resource-language work mainly expands datasets and adapts augmentation or transfer methods to linguistic differences. Future research should improve data quality, exploit linguistic similarities, build robust models, and make evaluation more reliable.
- Conclusion: A review of over 60 papers identifies dataset expansion, augmentation, and transfer learning adapted to language differences as major recent trends.The review covers recent work addressing the field’s fundamental lack of data.
- Conclusion: Future work should improve low-resource data quality, exploit linguistic patterns and similarities, design more robust learning models, and increase evaluation reliability.These directions summarize the paper’s proposed research priorities.