Source-linked AI summary
Arabic Offensive Language on Twitter: Analysis and Experiments
Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, Ahmed Abdelali
TL;DR
Arabic offensive-language detection lacks a large, representative dataset that avoids topic, dialect, and target bias. The paper constructs and analyzes such a dataset with specialized labels, then evaluates state-of-the-art classifiers, including AraBERT and tweet-trained static embeddings. It reports competitive detection results while noting limits in identifying offensive tweets with broad stylistic diversity.
Problem
Arabic offensive tweets comprise only 1-2% of general tweets, making random annotation inefficient and motivating a representative dataset without topic, dialect, or target bias.
Method
The authors use targeted collection and native-speaker annotation to label 10,000 tweets as offensive, vulgar, hate speech, or clean, then analyze usage and train classifiers.
Results
AraBERT and static embeddings trained on tweets produced competitive results on the Arabic offensive-language dataset.
Takeaways & Limitations
The dataset supports analysis of offensive-language topics, dialects, gender distribution, and usage patterns alongside Arabic offensive-language detection experiments.
Takeaways & Limitations
The collection approach captures only a small percentage of general tweets and may not cover greater stylistic diversity.
Abstract
from arXiv · showhide
Detecting offensive language on Twitter has many applications ranging from detecting/predicting bullying to measuring polarization. In this paper, we focus on building a large Arabic offensive tweet dataset. We introduce a method for building a dataset that is not biased by topic, dialect, or target. We produce the largest Arabic dataset to date with special tags for vulgarity and hate speech. We thoroughly analyze the dataset to determine which topics, dialects, and gender are most associated with offensive tweets and how Arabic speakers use offensive language. Lastly, we conduct many experiments to produce strong results (F1 = 83.2) on the dataset using SOTA techniques.
1 Introduction
The paper builds a large Arabic offensive-language Twitter dataset designed to avoid topic, target, and dialect bias, then analyzes Arabic offensiveness and evaluates state-of-the-art classifiers.
- The paper includes offensive-language and hate-speech examples with a disclaimer that they do not reflect the authors’ views.
- The dataset-building method avoids seed lists of offensive words, reducing bias by topic, target, or dialect.
- The authors tagged 10,000 Arabic tweets for offensiveness, vulgarity, and hate speech, producing the largest available dataset at the time.
- The study analyzes which topics, dialects, and user genders are most associated with Arabic offensive language.
- The paper reports strong Arabic offensive-language detection results using state-of-the-art classification techniques and representations.
2 Related Work
Prior work covers offensive-language resources and classifiers, but Arabic offensive-language detection remains relatively nascent and must address substantial morphological and dialectal variation.
- Offensive language is commonly distinguished into vulgar and hateful categories, and prior studies have built annotated corpora and trained classifiers.
- Arabic offensive-language detection is relatively nascent, with earlier studies using labeled tweets, supervised features, stemming, and recurrent neural networks.
- Arabic combines morphologically rich standard and dialectal varieties that differ in lexical, morphological, phonological, and syntactic structure.
- Dialectal segmentation is less reliable than MSA segmentation, while creative spellings and ubiquitous dialectal Arabic in tweets motivate character-level models.
3 Data Collection
The paper collects Arabic offensive tweets using a generic vocative pattern intended to reduce topic and dialect bias, then annotates them with offensive, vulgar, hate-speech, or clean labels. It analyzes their distributions and linguistic usage across topics, dialects, and user demographics.
- Collection: 1–2% of random Arabic tweets were offensive, motivating targeted collection rather than unrestricted random sampling.Three 1,000-tweet samples from different days produced this estimate.
- Collection: The vocative pattern “yA ... yA” was selected because it appears across Arabic dialects and topics without favoring specific offensive expressions.The pattern was used to narrow the search space while retaining broader topical and dialectal coverage.
- Annotation: The annotation scheme assigned tweets one or more labels: offensive, vulgar, hate speech, or clean.Offensive includes explicit or implicit insults or attacks; vulgarity and hate speech are not mutually exclusive.
- Annotation: 10,000 tweets yielded 1,915 offensive, 225 vulgar, 506 hate-speech, and 8,085 clean tweets, with Fleiss’s Kappa values of 0.92, 0.97, 0.96, and 0.97 across validation samples.The reported agreement supports the annotation quality.
- Statistics and User Demographics: Sports and politics were the dominant topics, while Egyptian and Gulf dialects dominated offensive language; vulgar tweets were 71% Egyptian and 13% Gulf.MSA accounted for 15% of offensive tweets overall, none of the vulgar tweets, and 21% of hate-speech tweets.
- Statistics and User Demographics: The analysis also examined hate-speech types, gender distributions, and common offensive-language patterns such as animal name-calling and insults targeting mental abilities.The paper additionally uses valence-weighted word frequencies to visualize prominent offensive terms.
4 Experiments
The experiments compare lexical features, static and contextualized embeddings, and classifiers for coarse offensive-tweet detection, then analyze model errors. Fine-tuned AraBERT performed best overall, while Mazajak embeddings with SVM were competitive.
- Representations and models: The study classifies tweets as offensive or not using lexical features, static embeddings, contextualized embeddings, and several classification models.Representations include polarity-lexicon counts, fastText, Skip-Gram, BERT, and AraBERT.
- Evaluation: Fine-tuned AraBERT yielded the best overall results, followed closely by Mazajak/SVM, with large precision improvements over BERT.The authors suggest that Mazajak’s in-domain tweet training may explain its advantage over BERT.
- Evaluation: SVM achieved the best results among the classifiers evaluated with Mazajak embeddings.
- Error analysis: False negatives included language mixing offensiveness with admiration and implicit offensiveness.
5 Conclusion and Future Work
The paper constructs and analyzes a large Arabic offensive-language dataset designed to avoid topic, dialect, and genre bias, then evaluates strong detection systems. It concludes that AraBERT and tweet-trained static embeddings produce competitive results, while identifying target-specific and cross-dialectal or cross-lingual learning as future work.
- Contributions: The authors built a 10,000-tweet Arabic dataset with labels for offensiveness, vulgarity, and hate speech.The dataset was designed not to favor specific dialects, topics, or genres and was intended for public release.
- Contributions: The dataset analysis examines offensive language by topic, dialect, mode of offensiveness, and author gender, separately considering vulgar and hate-speech tweets.
- Results: AraBERT and static embeddings trained on tweets produced competitive Arabic offensive-language classification results.
- Future work: Future work targets entity- or group-specific offensiveness and cross-dialectal and cross-lingual learning.