Source-linked AI summary
Detecting weak and strong Islamophobic hate speech on social media
Bertie Vidgen, Taha Yasseri
TL;DR
Islamophobic hate speech harms individuals and wider society, yet prior detection research largely treated it as binary despite Islamophobia’s varied forms. The paper develops a three-class classifier and finds that weak Islamophobia is more prevalent than strong Islamophobia in far-right tweets, while context-specific embeddings and SVM perform strongly.
Problem
Prior research largely treated Islamophobic hate-speech detection as binary, although Islamophobia has varied forms and causes considerable harm.
Method
The paper develops a multi-class classifier distinguishing non-Islamophobic, weak Islamophobic, and strong Islamophobic content using theoretically justified features and annotated tweets.
Results
The classifier achieves accuracy of 77.3% and balanced accuracy of 83%, while weak Islamophobia outnumbers strong Islamophobia in 109,488 far-right tweets.
Takeaways & Limitations
Multi-class classification provides detailed quantitative insight into online Islamophobia and can inform study of other forms of hate.
Takeaways & Limitations
The classifier struggles to distinguish weak Islamophobia from strong and non-Islamophobic content.
Abstract
from arXiv · showhide
Islamophobic hate speech on social media inflicts considerable harm on both targeted individuals and wider society, and also risks reputational damage for the host platforms. Accordingly, there is a pressing need for robust tools to detect and classify Islamophobic hate speech at scale. Previous research has largely approached the detection of Islamophobic hate speech on social media as a binary task. However, the varied nature of Islamophobia means that this is often inappropriate for both theoretically-informed social science and effectively monitoring social media. Drawing on in-depth conceptual work we build a multi-class classifier which distinguishes between non-Islamophobic, weak Islamophobic and strong Islamophobic content. Accuracy is 77.6% and balanced accuracy is 83%. We apply the classifier to a dataset of 109,488 tweets produced by far right Twitter accounts during 2017. Whilst most tweets are not Islamophobic, weak Islamophobia is considerably more prevalent (36,963 tweets) than strong (14,895 tweets). Our main input feature is a gloVe word embeddings model trained on a newly collected corpus of 140 million tweets. It outperforms a generic word embeddings model by 5.9 percentage points, demonstrating the importan4ce of context. Unexpectedly, we also find that a one-against-one multi class SVM outperforms a deep learning algorithm.
1 Introduction
Islamophobic hate speech causes harm to individuals and wider society while risking reputational damage for social-media platforms. Because Islamophobia has multiple forms, distinguishing its types offers advantages over treating it as one category.
- Islamophobic hate speech harms targeted individuals and wider society and risks reputational damage for host platforms.
- Islamophobia has been described as racism, stereotyping, prejudice, fear, exploitation, exclusion, and dominance.The paper presents Islamophobia as an essentially contested concept with limited consensus about its core features.
- Prejudice can range from explicit, overt, and direct behavior to implicit, covert, and indirect forms.
- Distinguishing different types of Islamophobic speech offers empirical and theoretical advantages over using a single Islamophobia category.
2 Classification task
The paper classifies social-media content as non-Islamophobic, weak Islamophobic, or strong Islamophobic. Weak and strong categories capture differing degrees and targets of anti-Muslim negativity, including subtle forms that are harder to observe.
- The classification task distinguishes non-Islamophobic, weak Islamophobic, and strong Islamophobic social-media content.
- Strong Islamophobic speech explicitly expresses negativity against all Muslims or calls for prejudicial actions against them.
- Weak Islamophobic speech weakly expresses negativity against all Muslims or explicitly targets a specific subset of Muslims.
- Examples of weak Islamophobia include describing Muslims as culturally different or foregrounding a perpetrator’s Muslim identity in reporting a terrorist attack.
- Subtle Islamophobia is harder to observe because it may only partially manifest anti-Muslim negativity.
3 Previous work
Prior research has mainly used binary classification or distinguished hate targets rather than levels of Islamophobic strength. Strength-based classification is challenging because weak and strong tweets often share linguistic and non-verbal features.
- Previous research largely used binary classification, while this study addresses distinguishing weak from strong Islamophobic hate speech.
- Most existing multi-class research distinguishes different hate targets rather than different strengths.
- Distinguishing hate strength narrows the domain to one target because performing both target and strength classification is difficult.
- Weak and strong Islamophobic tweets often share keywords, grammatical structures, hyperlinks, and emojis.
- Prior studies illustrate varied multi-class performance, including precision of 0.77, accuracy of 78%, and weighted F-score of 0.64.
- Davidson et al. report precision of 0.91, recall of 0.90, and F1 score of 0.90 for three-way hate-content classification.
4 Data
The study combines a large Twitter corpus with a 4,000-tweet annotated training dataset. Tweets were sampled from political-party followers and far-right accounts, then annotated by three expert annotators.
- The researchers collected 140 million tweets from followers of mainstream and far-right UK political parties during 2017 and the first six months of 2018.
- The source data included tweets produced by, and mentions of, 45 far-right accounts.
- The training dataset contained 4,000 tweets sampled across eight sources, including keyword searches for “Muslims” and “Islam.”
- Three experts in UK politics and prejudice annotated all 4,000 tweets blind using shared guidelines.
- Inter-rater agreement was 89.9%, with Fleiss’ kappa of 0.837 and Krippendorff’s alpha of 0.895.
- The final annotated classes were 3,106 non-Islamophobic, 484 weak Islamophobic, and 410 strong Islamophobic tweets before balancing.
5 Input features
The study restricts classifier inputs to theoretically justifiable features and compares text, non-text, and combined representations. Newly trained word embeddings perform best alone, while adding selected non-text features identifies mosque mentions as consistently important.
- Feature selection: The model uses only theoretically justifiable input features to reduce overfitting risk and improve generalizability.The authors emphasize that feature choices should be explainable rather than selected without theoretical justification.
- Compared input features: Three representations are tested: one-hot text encodings, 50 surface-level and derived non-text features, and a combined model.Non-text inputs include sentiment, swear-word counts, parts of speech, named entities, Muslim names, and mosque mentions.
- Evaluation: Ten-fold cross-validation is used to test the Naïve Bayes algorithm.The algorithm is selected because previous research indicates strong text-classification performance and deterministic results.
- Embedding performance: 5.9 percentage points separate the newly trained and pre-trained word-embedding models, with accuracy of 69.13% versus 63.2%.The newly trained embeddings model is the best-performing model among those compared.
- Feature optimization: Mentions of Mosques are consistently important, and the final accuracy-maximizing model adds six non-text features to word embeddings.The final feature set includes mosque mentions, HTML, retweets, conjunctions, locations, and organizations.
6 Choice of algorithm
The study compares six classification algorithms using the newly trained word-embedding representation. SVM achieves the highest reported accuracy and is selected despite the expectation that deep learning might perform better.
- Algorithm comparison: Six algorithms are tested, including Naïve Bayes, Random Forests, Logistic Regression, Decision Trees, SVM, and Deep Learning.The SVM uses a one-against-one multiclass strategy with optimized radial-kernel hyperparameters.
- Results: 72.17% is the SVM accuracy, outperforming Deep Learning by 1.03 percentage points.The two highest-performing methods are SVM and shallow feed-forward Deep Learning.
- Model selection: Low SVM C and gamma values are chosen to maximize generalizability and support empirical applications.The authors describe these hyperparameters as making the classifier suitable for empirical use.
7 Performance
The classifier was evaluated through cross-validation and testing on unseen tweets, showing strong overall performance but difficulty separating weak Islamophobia from adjacent classes.
- Cross-validated performance: 0.807 balanced accuracy provided evidence that the classifier handled class imbalance while limiting misclassifications.Macro-averaged recall was 0.741, precision was 0.739, and F1 was 0.74.
- Performance on unseen data: 100 tweets from each predicted class were combined into a 300-tweet unseen-data evaluation annotated independently by three original annotators.Majority decisions were used, with all three annotators agreeing in 95% of cases.
- Performance on unseen data: 23% of tweets labelled Strong Islamophobic were actually Weak, while 22% of predicted Weak tweets were actually None.The classifier distinguished None from Strong well but struggled with Weak versus both classes.
- Performance on unseen data: None Islamophobic tweets sometimes expressed hatred toward other groups or discussed Muslims without negativity, creating similar input features.These cases complicate classification on unseen data.
8 Application to far right tweets
Applied to 109,488 tweets from far-right accounts, the classifier found that most were not Islamophobic, while weak Islamophobia was more prevalent than strong Islamophobia.
- Application to far right tweets: 57,630 of 109,488 tweets were not Islamophobic.The dataset comprised tweets produced by 45 far-right accounts.
- Application to far right tweets: 36,963 tweets showed weak Islamophobia, considerably more than the 14,895 showing strong Islamophobia.The classifier was used to distinguish the prevalence of different strengths of Islamophobic hate.
9 Conclusion
The study presents a multi-class Islamophobic hate-speech classifier as a promising quantitative method for detailed analysis, while noting that nuanced strength distinctions still need improvement.
- Conclusion: 77.3% accuracy and 83% balanced accuracy indicate promising performance for the multi-class classifier.The authors describe the classifier as an important step toward quantitative insight into online Islamophobia.
- Conclusion: Further work should improve distinctions between hate strengths by enlarging the training dataset and engineering additional input features.The authors identify these as planned avenues for improving classifier performance.