Source-linked AI summary
Arabic natural language processing: An overview
Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, Damien Nouvel
TL;DR
Existing Arabic NLP surveys largely omitted Classical Arabic, Arabizi, recent studies, and associated resources. This paper surveys and classifies 90 studies across Arabic varieties and scripts, finding broad but uneven coverage and limited public resources, especially for Arabizi.
Problem
Earlier surveys principally covered MSA and AD in Arabic script, while omitting much Classical Arabic and Arabizi work and associated resources.
Method
The paper analyzes and classifies 90 recent Arabic NLP studies across CA, MSA, AD, and Arabizi in Arabic and Roman scripts, linking studies to public tools and resources.
Results
90 studies were classified, including 67 published between 2015 and 2018; 52 resources and tools were identified, while no Arabizi work was associated with a resource or tool.
Takeaways & Limitations
The survey maps recent Arabic NLP across varieties and scripts while highlighting uneven research coverage and public-resource availability.
Takeaways & Limitations
MSA and AD remain under-resourced compared with more studied languages, and Arabizi resources are not yet publicly available in the surveyed work.
Abstract
from arXiv · showhide
Arabic is recognised as the 4th most used language of the Internet. Arabic has three main varieties: (1) classical Arabic (CA), (2) Modern Standard Arabic (MSA), (3) Arabic Dialect (AD). MSA and AD could be written either in Arabic or in Roman script (Arabizi), which corresponds to Arabic written with Latin letters, numerals and punctuation. Due to the complexity of this language and the number of corresponding challenges for NLP, many surveys have been conducted, in order to synthesise the work done on Arabic. However these surveys principally focus on two varieties of Arabic (MSA and AD, written in Arabic letters only), they are slightly old (no such survey since 2015) and therefore do not cover recent resources and tools. To bridge the gap, we propose a survey focusing on 90 recent research papers (74% of which were published after 2015). Our study presents and classifies the work done on the three varieties of Arabic, by concentrating on both Arabic and Arabizi, and associates each work to its publicly available resources whenever available.
1. Introduction
Arabic NLP spans Classical Arabic, Modern Standard Arabic, and Arabic Dialect, with MSA and AD also appearing in Arabizi. Existing surveys largely omitted Classical Arabic, Arabizi, recent work, and associated resources, motivating this broader survey.
- Language varieties: Arabic comprises Classical Arabic, Modern Standard Arabic, and Arabic Dialect, while MSA and AD may also be written in Arabizi.Arabizi uses Latin characters, numerals, and punctuation for Arabic text.
- Research context: Arabic NLP research addresses morphological analysis, resource building, and machine translation, among other areas.
- Research gap: Earlier surveys principally focused on MSA and AD, often neglecting Classical Arabic and Arabizi.Arabizi research was largely published after 2015, so earlier surveys could not cover it.
- Survey scope: The survey classifies 90 recent Arabic NLP studies and associates them with publicly available tools or resources whenever available.It is organized around Classical Arabic, MSA, AD, Arabizi, synthesis, and future directions.
2. Works on classical Arabic
Work on Classical Arabic in this survey centers especially on Quranic Arabic, covering annotated corpora, semantic analysis, speech identification, search, translation, and verse classification.
- Scope: Classical Arabic studies in the survey focus especially on Quranic Arabic.Table 1 is explicitly devoted to studied works on Classical Arabic and the Quran.
- Resources: Quranic Arabic resources include QAC with morphological, syntactic, and semantic annotation, plus QurSim and QurAna for verse relations and pronoun antecedents.QurAna contains over 24,500 tagged pronouns.
- Speech processing: A Quran reciter recognition system uses MFCC feature extraction followed by neural-network identification.
- Applications: Projects apply Classical Arabic processing to semantic Quran search, multilingual translation, verse-topic classification, and feature-based verse labeling.The Quran analysis project supports semantic search and analysis, while Tanzil produces compliant Unicode Quran text and translations.
3. Works on MSA
MSA work covers core language analysis, corpora, parallel resources, sentiment analysis, dialect identification, translation, and speech-related applications.
- Basic Language Analyses: MSA research includes segmentation, morphological analysis, syntactic dependency parsing, and Universal Dependency resources.Farasa, YAMAMA, CamelParser, NUDAR, and Universal Morphological Lattices represent these efforts.
- Building Resources: MSA resources include large newspaper corpora, Arabic text-image datasets, and parallel corpora spanning multiple languages.TALAA contains 14 million words, ALIF contains 6,532 Arabic text color images, and one parallel corpus contains 1,727,539 aligned document pairs.
- Identification and translation: Arabic identification studies address multidialectal and MSA classification, while machine translation commonly relies on statistical systems requiring parallel corpora.One Arabic-Japanese phrase-based system reported a BLEU score up to 9.38 using aligned documents.
- Sentiment analysis: Sentiment systems combine classifiers such as SVM, Naive Bayes, and MLP with lexical or vector-based features.Reported examples include F1-scores up to 64.5% and 66.6% for two systems.
4. Works on Arabic dialects
Arabic-dialect research emphasizes orthographic conventions, morphological tools, corpora, sentiment resources, dialect identification, and translation across several regional varieties.
- Basic Language Analyses: Dialect research develops orthographic standards and conventions alongside segmentation and part-of-speech tagging.
- Basic Language Analyses: Morphological resources include ADAM, a Gulf analyzer covering over 2,600 verbs, Tunisian adaptations, and manually annotated Moroccan, Yemeni, and Emirati resources.ADAM was evaluated on Levantine and Egyptian Arabic and was comparable to CALIMA.
- Building Resources: Dialect corpora include Shami with 117,805 sentences, Curras with more than 56,000 annotated tokens, and semi-automatically annotated Saudi data.
- Semantic-level analysis: Dialect sentiment resources include NileULex, SANA, SentiAlg, SOCALAlg, and the Tunisian TSAC corpus.TSAC contains 17,060 manually annotated Tunisian Facebook comments.
- Language Identification: Dialect identification covers broadcast speech and written code-switching across multiple dialects and MSA.The reviewed dialects include Egyptian, Levantine, Gulf, and Maghrebi varieties.
- Machine Translation: Dialect machine translation uses phrase-based systems, with one Algerian system reaching a BLEU score up to 67.31 between Algiers and Annaba varieties.
- Semantic-level analysis: Dialect sentiment classification combines lexicons, BOW or Doc2vec representations, and classifiers including SVM, Naive Bayes, and MLP.NileULex integration improved one classification system to an F1-score up to 79%.
5. Works on Arabizi
The surveyed Arabizi research spans analyzers, lexicons, identification, translation, sentiment analysis, and transliteration, with transliteration quality affecting downstream translation.
- Analysis: ASDA is a syntactic analyzer for Algerian Arabizi that extracts verbs, nouns, adjectives, conjunctions, and pronouns.It enriches a basic Algerian dialect–French dictionary with phonological extensions.
- Resources: A bilingual Algerian Arabizi–French lexicon was enriched by adding words with equivalent pronunciations, producing 25,086 entries.
- Identification: Arabizi identification includes word-level Arabizi-versus-English classification and supervised dialect identification in social media.Darwish’s CRF-based system achieved 98.5% accuracy for word-level language identification.
- Machine translation: Algerian Arabizi–MSA machine translation was compared across statistical and neural systems after transliteration.
- Machine translation: BLEU reached 4.26 without transliteration, 6.01 with automatic transliteration, and 10.74 with manual transliteration.The surveyed results therefore report higher translation scores when transliteration is supplied, especially manually.
- Transliteration and sentiment analysis: Arabizi transliteration approaches are classified as rule-based, statistical, or hybrid, while some studies also address sentiment analysis without transliteration.
6. Synthesis and discussion
The synthesis classifies 90 Arabic NLP studies across varieties and scripts, finding dialect work dominant, Arabizi increasingly recent, and resource availability uneven across varieties.
- Overall distribution: 90 studies were classified: 10 on CA, 24 on MSA, 30 on AD, 14 on Arabizi, plus 10 combining MSA/AD and 2 combining AD/Arabizi.67 studies, or 74.4%, were published between 2015 and 2018; 52 resources and tools were identified.
- Resource availability: 52 of 74 CA, MSA, and AD studies, or 70.3%, could be associated with resources or tools, whereas no Arabizi work could be associated with one.
- Overall distribution: Dialectal Arabic received the largest share of work, while 12 of 16 Arabizi studies, or 75%, were published between 2015 and 2018.
- Resource availability: Resources and tools are more abundant for GLF and EGY than for Algerian dialect, for which only PADIC is publicly available.
- Arabizi trends: Recent Arabizi studies increasingly treat Arabizi as a specific dialect form, but the surveyed work does not address identification among Algerian, Tunisian, Egyptian, and other Arabizi varieties.
- Research focus: Resource construction, especially lexicons and corpora, was the most common activity; 50 of 90 studies, or 55.6%, provided resources, and BLA comprised 24 studies, or 26.7%.The survey attributes BLA’s prominence to Arabic’s morphological complexity and the resulting need for preprocessing.
7. Conclusion and perspectives
The survey covers all Arabic varieties in both Arabic and Arabizi scripts and links studies to public resources, while identifying persistent gaps in coverage, availability, and resource development.
- Conclusion: The survey analyzes and classifies 90 recent works covering CA, MSA, AD, and Arabizi in Arabic and Arabizi scripts.
- Conclusion: Its distinctive contribution is associating surveyed works with publicly available tools and resources.
- Open issues: Few studies address CA, leaving many open issues in that variety.
- Open issues: Arabizi is an emerging research area, but its presented resources are not yet all publicly available.
- Open issues: Although MSA and AD receive substantial attention, the survey still considers them under-resourced compared with more extensively studied languages such as English.
- Future perspectives: The authors plan a follow-up survey using the identified public tools and resources on extrinsic data to examine their strengths and weaknesses.