PaperlayerView exact revision on arXiv

Source-linked AI summary

The African Languages Lab: A Collaborative Approach to Advancing Low-Resource African NLP

Sheriff Issaka, Keyi Wang, Yinka Ajibola, Oluwatumininu Samuel-Ipaye, Zhaoyi Zhang, Nicte Aguillon Jimenez, Evans Kofi Agyei, Abraham Lin, Rohan Ramachandran, Sadick Abdul Mumin, Faith Nchifor, Mohammed Shuraim, Lieqi Liu, Erick Rosas Gonzalez, Sylvester Kpei, Jemimah Osei, Carlene Ajeneza, Persis Boateng, Prisca Adwoa Dufie Yeboah, Saadia Gabriel

arXiv:2510.05644v1cs.CLcs.AI

TL;DR

Analysis of the top 10 global and African languages found, per African-language paper in multilingual LLM contexts, 20 global-language papers in Google Scholar, 23 in CORE, 34 in arXiv, and 70 in IEEE. The All Lab combines community-driven All Voices collection, a validated 40-language text-and-speech dataset, and supervised full fine-tuning. Fine-tuning achieves average gains over baseline models of +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU.

  • Problem

    Analysis of the top 10 global and African languages found, per African-language paper in multilingual LLM contexts, 20 global-language papers in Google Scholar, 23 in CORE, 34 in arXiv, and 70 in IEEE.

    Source: §1 · p. 2

  • Method

    The All Lab combines community-driven All Voices collection, a validated 40-language text-and-speech dataset, and supervised full fine-tuning.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    Fine-tuning achieves average gains over baseline models of +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The conclusion presents African-language technological marginalization as severe but not intractable and reports mentorship of fifteen early-career researchers.

    Source: §5 · p. 8

    The validation pipeline lacks native-speaker verification for 73% of languages, and 13 languages have no audio data.

    Source: §6.2 · p. 9

Abstract

from arXiv · show

Despite representing nearly one-third of the world's languages, African languages remain critically underserved by modern NLP technologies, with 88\% classified as severely underrepresented or completely ignored in computational linguistics. We present the African Languages Lab (All Lab), a comprehensive research initiative that addresses this technological gap through systematic data collection, model development, and capacity building. Our contributions include: (1) a quality-controlled data collection pipeline, yielding the largest validated African multi-modal speech and text dataset spanning 40 languages with 19 billion tokens of monolingual text and 12,628 hours of aligned speech data; (2) extensive experimental validation demonstrating that our dataset, combined with fine-tuning, achieves substantial improvements over baseline models, averaging +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU points across 31 evaluated languages; and (3) a structured research program that has successfully mentored fifteen early-career researchers, establishing sustainable local capacity. Our comparative evaluation against Google Translate reveals competitive performance in several languages while identifying areas that require continued development.

1 Introduction

African languages face severe underrepresentation in NLP and research attention. The All Lab combines community-driven data collection, multilingual resources, model evaluation, and mentorship to address this gap.

1 Introduction

  • 88% of African languages are severely underrepresented or ignored in computational linguistics, and approximately 814 face extinction risk.More than 2,000 languages are spoken across Africa.Source: §1 · p. 1
  • 20–70× more papers discuss global than African languages in multilingual LLM contexts across four academic databases.The reported ratios are 20 in Google Scholar, 23 in CORE, 34 in arXiv, and 70 in IEEE.Source: §1 · p. 2
  • The All Lab’s mobile-first All Voices platform supports community-driven multilingual collection and direct translation between African languages without English intermediation.The initiative combines a coordinated research team with systematic data collection, model development, and capacity building.Source: §1 · p. 2
  • 19 billion tokens across 40 languages and 12,628 hours of aligned speech form the Lab’s multi-modal African-language dataset.The paper presents it as the largest multi-modal dataset for African low-resource languages.Source: §1 · p. 2
  • +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU are the reported average improvements over baseline models, alongside mentorship of early-career researchers.The contribution passage also describes structured mentorship programs as developing local research capacity.Source: §1 · p. 2

2 Related Work

Prior African NLP work spans community initiatives, multilingual modeling, and evaluation resources. The paper situates its contribution amid persistent research and coordination gaps.

2 Related Work

  • African NLP research develops through three connected streams: community-driven initiatives, multilingual modeling, and evaluation frameworks.The section frames these streams as shaping current capabilities and exposing gaps.Source: §2 · p. 2

2.1 Community-Driven Research Initiatives

  • Masakhane exemplifies community-led dataset and model research, while institutional projects supply datasets, architectures, speech resources, benchmarks, and capacity-building infrastructure.The cited examples include Lacuna Fund, No Language Left Behind, Common Voice, AI4D, Deep Learning Indaba, and Lanfrica.Source: §2.1 · p. 2

2.2 Advances in African Multilingual NLP

  • Table 1 reports a 20–70× publication-volume disparity between top global and African languages across major academic databases from 2020–2024.The related-work discussion also traces multilingual models from mBERT and XLM-R to broader and African-focused systems.Source: §2.2 · pp. 2–3, Table 1 · p. 3

2.3 Benchmarks and Evaluation Frameworks

  • African NLP benchmarks cover tasks including named-entity recognition, sentiment, translation, reasoning, and question answering, but coordination and knowledge-transfer challenges remain.The section names MasakhaNER, AfriSenti, AFROMT, and IrokoBench among its evaluation resources.Source: §2.3 · pp. 3–4

3 Methodology

The methodology combines community and open-source data, language-aware processing and validation, full fine-tuning, and standardized translation evaluation.

3.1 Datasets

  • All Voices is a mobile-first, open platform for community collection and validation of text and audio, including direct low-resource-language translation without English intermediation.Its workflow uses peer validation, and verified translations can become source material for subsequent translations.Source: §3.1 · p. 4
  • The dataset integrates All Voices translations, curated open-source corpora, and new data collected through community partners.Named sources include NLLB, CCMatrix, Open-Subtitles, MultiCCAligned, ParaCrawl, XLEnt, MultiParaCrawl, LinguaTools-WikiTitles, and CCAligned.Source: §3.1 · p. 4
  • Processing combines general text normalization with language-specific handling such as morphological analysis, script-variant normalization, tone-mark standardization, and language identification.Custom rule sets are developed for specific language families.Source: §3.1 · pp. 4–5
  • Translation validation uses language-specific character-ratio distributions, z-scores, character-overlap checks, and adaptively set thresholds to flag anomalous pairs.Thresholds are based on initial samples of 10,000 pairs per language pair, Kernel Density Estimation, and a modified Tukey method.Source: §3.1 · p. 5

3.2 Model Development

  • Llama-3.2-1B provides the baseline, and the study uses supervised full fine-tuning after preliminary QLoRA experiments yielded insufficient gains for the target languages.The standardized instruction template requests translation from English into the target African language.Source: §3.2 · p. 5

3.3 Evaluation Metrics

  • Translation quality is evaluated with BLEU, METEOR, COMET, ChrF++, TER, and AfriCOMET on FLORES-200 for standardized cross-language comparison.The metrics cover n-gram matching, semantic similarity, character-level patterns, and post-editing effort.Source: §3.3 · p. 6

4 Results

The dataset spans 40 African languages but resources are unevenly distributed across text and speech. Fine-tuning improves translation across evaluated languages, with results varying by language and metric.

4 Results

  • 19,012,019,696 text tokens and 12,628 aligned audio hours cover 40 languages, but the leading languages differ across text and speech.The top three languages by text volume account for a disproportionate share of tokens, while the top three by audio hours capture much of the recorded speech.Source: §4 · p. 6
  • Text resources range from four languages above 2B tokens to a majority below 250M, including Bambara and Luganda.The listed top-tier languages are Amharic, Arabic, Yoruba, and Afrikaans; Bambara has 109.49M tokens and Luganda 121.17M.Source: §4 · p. 6
  • Audio resources are concentrated in four languages above 1,000 hours, while many others have under 100 hours or no audio.The high-resource group includes Kinyarwanda, Luganda, Swahili, and Arabic; the under-100-hour examples include Kikongo, Rundi, Kanuri, Umbundu, and Fang.Source: §4 · p. 6

4.1 Translation Performance Analysis

  • +23.69 ChrF++, +0.33 COMET, and +15.34 BLEU points are the average fine-tuning gains; results are competitive with Google Translate in some languages, while Google leads in others.The comparison covers 22 languages where Google Translate is available; nine collected languages were excluded from evaluation because of insufficient training data or absence from FLORES-200.Source: §4.1 · p. 7, Table 3 · p. 8
  • Yoruba, Arabic, and Twi match or exceed Google Translate in ChrF++, while Google leads for Kinyarwanda and Swahili.Google Translate offers no support for some compared languages; the supplied results do not identify those languages by name.Source: §4.1 · p. 7, Table 3 · p. 8
  • Swahili, Sesotho, and Hausa improve by over 40 ChrF++ points, while Fon, Wolof, and Bambara show limited gains.The authors suggest specialized tokenization or architectural modifications may be needed to address the challenging languages’ linguistic features.Source: §4.1 · pp. 7–8
  • Fine-tuning lowers Translation Edit Rate by an average of 580.13 points, and Swahili reaches a TER score of 23.77 comparable to Google Translate.The authors note that these TER reductions may matter where post-editing cost determines practical viability.Source: §4.1 · p. 8
  • Fine-tuning gains differ across metrics: Ewe gains 30.43 ChrF++ points but only 0.04 COMET, while Arabic gains 0.33 COMET with modest ChrF++ improvement.The authors use these divergences to underscore the importance of evaluating morphologically diverse African languages with multiple metrics.Source: §4.1 · p. 8

5 Conclusion

The African Languages Lab addresses African-language underrepresentation through data collection, model development, and capacity building. Its work combines a large multilingual dataset, improved translation results, and mentorship for early-career researchers.

5 Conclusion

  • The Lab’s contributions include a validated dataset of 19 billion tokens and 12,628 aligned speech hours across 40 languages, alongside translation improvements over baseline models.The reported average improvement is +23.69 ChrF++.Source: §5 · p. 8
  • A structured mentorship program developed fifteen early-career researchers.The conclusion presents capacity building alongside data collection and model development as part of the research initiative.Source: §5 · p. 8
  • The authors conclude that African-language technological marginalization, though severe, is not intractable.They argue that coordinated research, community engagement, and sustained investment in technical infrastructure and human capacity provide a path forward for underserved linguistic communities.Source: §5 · p. 8

6 Limitations

The evaluation is constrained by its model choice and language coverage, while the dataset and platform also have limitations. The authors identify data imbalance, missing audio, validation gaps, and infrastructural needs as boundaries for future work.

6.1 Model Architecture and Scale Constraints

  • Only Llama-3.2-1B was used as the base model, so the experiments may underestimate gains achievable with larger architectures.Performance varies across language families, and the authors suggest optimal model selection may vary by linguistic typology.Source: §6.1 · p. 9

6.2 Dataset Imbalance and Coverage

  • Despite assembling 19 billion tokens, the dataset has a 147,000× disparity between Amharic (2,944.95M) and Fang (0.02M), and 13 languages lack audio data entirely.Native-speaker verification is absent for 73% of languages, potentially missing dialectal variations that affect 28% of evaluated translations with COMET-ChrF++ divergence exceeding 0.3.Source: §6.2 · p. 9

6.3 Platform and Infrastructure

  • All Voices operates primarily through mobile interfaces, potentially limiting participation from communities with different technology preferences or access patterns.The authors also note that its quality-control mechanisms may favor some linguistic varieties; they call for continued investment in computational resources, human expertise, and infrastructure.Source: §6.3 · p. 9

7 Ethics Statement and Broader Impacts

The All Lab describes consent-based collection and managed access alongside mentorship and a roadmap for equitable African NLP development.

7 Ethics Statement and Broader Impacts

  • The authors frame the initiative around public good, community accountability, and equitable technological development.They identify historical exclusion, infrastructural inequities, and linguistic marginalization as relevant contexts.Source: §7 · p. 9

7.1 Data Collection Ethics

  • All Voices uses voluntary, revocable consent, moderation, and native-speaker collaboration to support ethical data collection.Automated validation may miss dialectal nuances or culturally specific meanings.Source: §7.1 · p. 9

7.2 Data Governance.

  • Some dataset components are not fully public because community agreements restrict release; qualified researchers can request managed access.Requests are assessed by research purpose, institutional affiliation, and commitment to ethical use.Source: §7.2 · p. 9

7.3 Capacity Building and Research Development

  • Fifteen early-career researchers across four institutions receive mentorship, project support, and pathways into extended research roles.The program is framed as building sustainable local research capacity and leadership.Source: §7.3 · pp. 9–10

7.4 Societal Impact and Sustainability

  • The authors connect increased digital representation and community-led content creation with education, inequality reduction, and language preservation.They cite 88% of African languages as severely underrepresented or ignored and 814 as facing extinction risk.Source: §7.4 · p. 10

7.5 Philosophical Framework and Future Vision

  • The work grounds equitable NLP development in Ubuntu and proposes broader language coverage, model optimization, and deeper research collaboration.The authors acknowledge limited commercial viability for some technologies and infrastructural constraints.Source: §7.5 · p. 10

A Aggregated Paper Count Approach

Table 4 describes aggregate paper counts and ratios comparing high-resource and African languages from 2020 to 2024.

A Aggregated Paper Count Approach

  • Table 4 covers aggregate paper counts and ratios between high-resource and African languages for 2020–2024.Its stated search terms are “multilingual” “X” “large language models”.Source: Table 4 · p. 21
  • The table’s ratios represent disparity in research visibility, with higher values indicating greater inequality in representation.Source: Table 4 · p. 21
  • The comparison uses paper counts and ratios for the two language groupings, rather than a measure of translation performance.Source: Table 4 · p. 21

B Abstract Translations

The listed figures are titled as translations for ten languages.

B Abstract Translations

Related papers

Loading 2510.05644v1…