PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards generalisable hate speech detection: a review on obstacles and solutions

Wenjie Yin, Arkaitz Zubiaga

arXiv:2102.08886v1cs.CL

TL;DR

Hate speech detectors generalise poorly to unseen datasets, while existing moderation mechanisms struggle with nuanced and rapidly evolving expressions. This survey synthesises evidence on generalisation, analyses its obstacles and proposed solutions, and concludes that the challenges are intertwined across NLP methods, datasets, and online hate speech.

  • Problem

    Existing hate speech detection models generalise poorly to unseen datasets, limiting reliable detection as online hate speech increases.

  • Method

    The paper systematically searches and synthesises studies on model generalisability, obstacles, and attempts to address them, then proposes future research directions.

  • Results

    Existing models generalise poorly across new datasets, with reported macro-averaged F1 drops exceeding 30 points across Twitter hate speech datasets.

  • Takeaways & Limitations

    Generalisable hate speech detection remains difficult because NLP limitations, dataset biases, and evolving online language are intertwined.

  • Takeaways & Limitations

    Evidence on which dataset properties produce more generalisable models is inconsistent, limiting firm conclusions about training-data choices.

Abstract

from arXiv · show

Hate speech is one type of harmful online content which directly attacks or promotes hate towards a group or an individual member based on their actual or perceived aspects of identity, such as ethnicity, religion, and sexual orientation. With online hate speech on the rise, its automatic detection as a natural language processing task is gaining increasing interest. However, it is only recently that it has been shown that existing models generalise poorly to unseen data. This survey paper attempts to summarise how generalisable existing hate speech detection models are, reason why hate speech models struggle to generalise, sums up existing attempts at addressing the main obstacles, and then proposes directions of future research to improve generalisation in hate speech detection.

Introduction … Generalisability in hate speech detection

Automatic hate speech detection is important because online harmful content is difficult to moderate at scale, yet existing models generalise poorly beyond their original datasets. The survey reviews this problem, analyses its obstacles and attempted solutions, and proposes future research directions.

  • Introduction: Social media’s growth has enabled harmful content, including hate speech, to spread and damage user experience and community civility.Hate speech is introduced as harmful content that directly attacks or promotes hate toward people based on identity-related characteristics.
  • Introduction: Keyword filters miss nuanced hate, while crowdsourced moderation methods such as human review and user reporting do not scale.These limitations motivate automatic detection as an urgent task.
  • Hate speech, offensive language, and abusive language: Hate speech differs from offensive or abusive language through distinctions involving identity-based targeting and intentionality.It also differs from personal attacks, which target individuals without necessarily involving identity, and cyberbullying, which is repeated over time against vulnerable victims.
  • Generalisation: Hate speech detection models mainly use supervised learning, with generalisation performance measuring their ability to predict previously unobserved inputs.Evaluation usually relies on a held-out test set under the assumption that training, test, and future data share a distribution.
  • Generalisability in hate speech detection: Generalisable models must handle the continuing growth and evolution of hate speech across forms, targets, and speakers.This capability is necessary for reducing the harms caused by online hate speech.
  • Generalisability in hate speech detection: Performance significantly drops when models are applied to different hate speech datasets despite impressive results on their respective test sets.This finding indicates that existing test sets may not represent future cases and that model generalisation has been severely overestimated.
  • Generalisability in hate speech detection: Prior research has mainly demonstrated poor generalisability, with only a handful of studies attempting to address particular aspects of the problem.This survey adds a comparative summary, systematic obstacle analysis, review of existing attempts, and suggestions for future research.

Survey Methodology · Sources of search

The literature search was staged separately for each of the paper’s three aims, using Google Scholar, keyword sets, and iterative reference checking. Search terms were refined through conference and workshop proceedings and adapted as new phrasing emerged.

  • Survey Methodology: Literature searches were divided into stages for each of the paper’s three aims.
  • Sources of search: Google Scholar served as the main search engine across the different search stages.
  • Sources of search: References and citations were checked back-and-forth, with iteration counts depending on whether each stage’s search was coarse- or fine-grained.
  • Sources of search: The search used two keyword sets covering hate speech and related harmful-content detection terminology.
  • Sources of search: General keywords included “hate speech”, “offensive”, “abusive”, “toxic”, “detection”, and “classification”.
  • Sources of search: Generalisability-related keywords included “generalisation”, “generalisability”, “cross-dataset”, “cross-domain”, and “bias”.
  • Sources of search: The predefined keyword set was refined by skimming proceedings from relevant recent conferences and workshops, then modified when new term phrasing appeared.

Main literature search stages · Generalisation Studies in Hate Speech Detection

The review used staged literature searches to identify generalisability findings, synthesise obstacles, and select representative studies. Cross-dataset evaluations show substantial performance losses across model types, with generalisation varying according to dataset relationships and properties.

  • Main literature search stages: The initial search used broad keywords, prioritised highly cited peer-reviewed papers in high-impact venues, and considered existing hate-speech and abusive-language surveys.This stage also established the experimental studies later used for error analysis.
  • Main literature search stages: For the comparative generalisability aim, the review searched combinations of general and generalisation-related keywords and included all relevant papers because the topic was sparse.The resulting studies were compared in the generalisation literature review.
  • Main literature search stages: The review identified obstacles by analysing error analyses from experimental studies and comparing results and discussions across generalisability studies, then searching each obstacle category.The obstacle categories became the section headings of the paper’s body.
  • Main literature search stages: During writing, representative studies were selected using novelty, experimental design, error analysis, venue, influence, and attention to hate-speech-specific problems.The findings were structured before representative studies were incorporated.
  • Generalisation Studies in Hate Speech Detection: Cross-dataset testing evaluates generalisability more realistically by testing a model on data different from its training dataset.The method is also called cross-application, cross-domain classification, or cross-domain detection.
  • Generalisation Studies in Hate Speech Detection: More than 30 points in macro-averaged F1 were lost when LSTM and CNN-GRU models were cross-applied across four Twitter hate-speech datasets.The four datasets were Wulczyn, Davidson, Waseem, and Zhang.
  • Generalisation Studies in Hate Speech Detection: Around 30 points in macro-averaged F1 were lost by BiLSTM and GBDT models when transferred from Waseem to Basile, while BERT drops ranged from 2 to 30 points.The BERT evaluation used Waseem, Davidson, Founta, and Zampieri.
  • Generalisation Studies in Hate Speech Detection: Generalisation varies with dataset distributions: similar dataset groups can yield scores above 70, whereas models involving datasets outside those groups achieved around 60 or less.Studies attributed stronger transfer to explicit abuse proportions, less biased sampling, and similar dataset characteristics; dataset-property evidence remains inconsistent.

Obstacles to Generalisable Hate Speech Detection

The paper identifies three factors that may contribute to poor generalisability in hate speech detection: non-standard language, limited and biased datasets, and implicit hate expressions.

  • Non-standard grammar and vocabulary may hinder hate speech detection models’ generalisability.
  • Dataset paucity and biases may contribute to poor generalisability in hate speech detection.
  • Implicit expressions of hate may contribute to poor generalisability in hate speech detection.

Non-standard Grammar and Vocabulary on Social Media

Social-media hate speech uses non-standard language, including obfuscated spellings and context-dependent code words, which complicate generalisable detection. These phenomena contribute to false negatives and expose limitations in standard pre-training, while character, sentence, subword, and domain-specific embeddings offer partial solutions.

  • Challenges: Obfuscated offensive words and ordinary code words such as “Skype,” “Google,” and “banana” can evade moderation and carry hateful meanings.These linguistic phenomena make pre-trained representations unreliable because obfuscated words may be out-of-vocabulary and code-word embeddings may miss context-dependent meanings.
  • Challenges: 30% of Wikipedia false negatives and 43% of Twitter false negatives contained rare or unknown words.Rare words and implicit expressions were identified as two main causes of false negatives.
  • Existing solutions: Character-level features have inconsistent results, with character embeddings potentially overfitting limited training data.Hybrid CNNs performed best in one study, whereas character CNNs performed best in another; simpler character n-gram logistic regression matched sophisticated neural models.
  • Existing solutions: Sentence embeddings partially address out-of-vocabulary words, and multi-task sentence embeddings consistently outperform traditional word embeddings.Universal Sentence Encoder with shallow classifiers helped one team place first at HatEval 2019.
  • Existing solutions: BERT variants combine subword information with contextual word representations and have achieved top performances in recent hate or abusive speech challenges.However, these models were pre-trained primarily on standard English, while embeddings trained on social-media or news data showed a clear advantage.

Limited, Biased Labelled Data

Hate speech detection datasets are difficult to label because annotation requires relevant knowledge and training, while judgments about hate speech are subjective. Consequently, datasets are typically small.

  • Limited, Biased Labelled Data: Annotation requires relevant knowledge and training, making hate speech detection data particularly challenging to acquire.Judging hate speech also requires extra care during annotation processing.
  • Limited, Biased Labelled Data: Because hate speech is an abstract concept and annotation judgments are subjective, datasets are usually not big in size.

Small data size. · Sampling bias.

Small datasets can cause overfitting, while non-random sampling, inconsistent definitions, and annotation schemes make hate-speech datasets biased and heterogeneous. These factors cause models to learn dataset-specific patterns, limiting generalisation to new data.

  • Small data size.: Small datasets can cause overfitting in high-capacity models, harming generalisability; pre-trained embeddings and parameter dropout are standard preventive practices.These methods are commonly used in hate-speech detection to reduce over-fitting.
  • Small data size.: Abusive-language-specific embeddings, such as WSW2V trained on one million tweets, were proposed to address limited hate-speech dataset sizes.WSW2V used tweets collected through white-supremacy-related hashtags and users.
  • Small data size.: Entire-model transfer learning from sentiment analysis consistently outperformed baseline word2vec in Uban and Dinu’s comparisons.The study transferred a classifier pretrained on a large sentiment dataset to the Zampieri and Kumar datasets.
  • Small data size.: Sentiment-embedding approaches differ in training data, label generation, model depth, general embeddings, and topic representations, leaving optimal transfer conditions uncertain.Cao, Lee, and Hoang used only the Davidson and Founta datasets and generated sentiment labels with VADER.
  • Sampling bias.: Less than 3% of social-media content is hate speech or offensive language, prompting datasets to use boosted sampling with simple heuristics.Non-random sampling helps address scarce positive cases during model training but makes datasets prone to bias.
  • Sampling bias.: Keyword searches and hateful-user identification dominate sampling, but selected keywords, users, and additional methods vary substantially across datasets.Sampling may use slurs, neutral words, profanity, hashtags, identified haters, victims, or sentiment classification.
  • Sampling bias.: Datasets apply inconsistent hate-speech definitions and annotation criteria, including different targets, edge cases, hate types, and boundaries from offensive language.These differences create essentially different tasks and training objectives despite substantial overlap in the underlying data.
  • Sampling bias.: 65% of hate speech in the Waseem dataset came from two users whose tweets appeared in training and test sets, encouraging author-style overfitting.Societal biases and minority underrepresentation in data or annotation can also produce broader harms when models are deployed.

Representation bias.

Hate speech detection models exhibit representation bias across dialects, demographic attributes, and identity terms. Recent mitigation efforts target data, external resources, training objectives, and annotation practices, but inconsistent bias metrics make results difficult to compare.

  • Observed biases: Models more often label African-American English as offensive or hateful, while neutral identity terms such as “gay” and “woman” can trigger false positives.Models also more often predict false negatives on “white” English than African-American English.
  • Bias measurement: Bias studies measure associations between identity terms and hateful classes or compare predictions across language styles and demographic attributes.Studies have examined African-American versus white-aligned English and extended comparisons to gender and political identity.
  • Mitigation approaches: Mitigation approaches include augmenting training data, using less biased corpora or debiased embeddings, and adding debiasing objectives or regularisation during training.Examples include balanced identity-term data, transfer learning, weighted loss functions, and regularisation based on feature importance.
  • Evaluation challenges: There is little consensus on bias metrics, and subgroup definitions make results difficult to compare across studies.Reported measures include Error Rate Equality Difference, pinned AUC Equality Difference, Pinned Bias, synthetic test set AUC, and weighted subgroup AUCs.
  • Annotation bias: Including dialect information and annotation guidance reduced white annotators’ tendency to rate African-American English tweets as offensive.Sap et al. addressed annotator bias during dataset construction rather than debiasing completed annotations or model training.

Hate Expression Can Be Implicit

Hate speech can be implicit, conveyed through stereotypes, sarcasm, irony, humour, or metaphor rather than only slurs and profanity. These nuanced expressions are difficult to detect, frequently cause false negatives, and remain challenging for both datasets and models.

  • Hate Expression Can Be Implicit: Hate can be expressed through stereotypes, sarcasm, irony, humour, and metaphor, making keyword-based identification an incomplete proxy.Slurs and profanity are common in hate speech and are therefore widely used as dataset keywords.
  • Hate Expression Can Be Implicit: Implicit hate speech is no less harmful than explicit hate speech and is the most commonly mentioned cause of false negatives in error analyses.Detecting implicit hate requires nuanced interpretation beyond surface-level cues.
  • Hate Expression Can Be Implicit: Implicit hate is difficult to identify because it often requires specific context or real-world knowledge, such as stereotypes.Datasets have incorporated context through original news articles, sentences from the same post, and separate relation labels.
  • Hate Expression Can Be Implicit: The Caselli dataset is so far the only dataset with a standalone “implicit” label, distinguishing implicitly abusive, explicitly abusive, and non-abusive content.The dataset re-annotated the Zampieri dataset and separated abusiveness from offensiveness.
  • Hate Expression Can Be Implicit: Precision of around .234 and recall of 0.098 for Caselli et al. (2020)’s implicit class contrast with precision .864 and recall .936 for non-abusive content, and precision .640 and recall .509 for explicit content.These experiments indicate that predicting implicit abuse or biases remains a major challenge.

Discussion

The discussion argues that hate speech detection generalisability should be evaluated in real-world context, not reduced to cross-dataset or debiased metrics. It also highlights that the field lacks consensus on problem formulation and representative datasets, so no existing dataset is yet a benchmark.

  • Real-world generalisability: Generalisability should not be reduced to cross-dataset performance or “debiased” metrics, because the ultimate goal is real-world applicability.The discussion stresses that models and datasets should be considered in light of their intended use and potential effects on users and wider society.
  • Real-world generalisability: Mathematical metrics quantify performance, but evaluation should remain connected to the problem’s intended context and societal impact.The authors emphasise why models and datasets are developed, how they will be used, and their potential impacts on users and society.
  • Open methodological questions: The field still lacks consensus on whether hate speech detection should be binary, multi-class, or hierarchical, and on how to define its boundaries.Open questions also concern sourcing representative data and deciding what metadata to include, including where offensive content becomes hateful.
  • Open methodological questions: No existing hate speech dataset yet qualifies as a “benchmark dataset,” reflecting the field’s unresolved formulation and data-representation challenges.Hate speech detection is described as a relatively new NLP sub-field, despite drawing on established NLP methods.

Future research · Datasets.

Future research targets both models and datasets to improve the generalisability and practical usefulness of automatic hate speech detection. Dataset-focused priorities include clearer labels, higher-quality and better-documented annotations, representative sampling, and methods that reduce overfitting.

  • Future research: Research should address both model and dataset perspectives to evaluate and improve hate speech detection generalisability.The paper frames these as critical considerations for researchers seeking practical automatic detection.
  • Future research: Clear label definitions should separate hate speech from other offensive and abusive language categories.Future datasets could cover a wider spectrum of abusive language, including personal attacks, trolling, and cyberbullying.
  • Datasets.: Only about two thirds of existing datasets report inter-annotator agreement, highlighting the need for extensive instructions and expert annotators.The paper notes a trade-off between dataset size and annotations with high inter-annotator agreement.
  • Datasets.: Annotator demographics and backgrounds should be reported because perceptions of hate speech depend on the reader’s background.Existing datasets usually report annotator counts and crowdworker status, but seldom demographic information.
  • Datasets.: Future studies should examine sources of annotator disagreement and build datasets containing extensive annotator attributes alongside their judgements.Potential annotator types include authors, linguists, activists, and political experts.
  • Datasets.: Representative sampling should begin with an initial random sample before applying boosting techniques, rather than drawing a filtered sample.Both initial collection procedures and boosting the proportion of positive cases can introduce dataset bias.
  • Datasets.: Overfitting can be reduced by training on multiple datasets or using transfer learning from larger datasets or closely related tasks.The literature includes transfer from sentiment analysis, but synthesis and comparisons of training approaches remain lacking.

Models. · Conclusion

The paper recommends reducing lexical reliance, systematically comparing debiasing methods, and evaluating hate speech models beyond dataset-level metrics. It concludes that poor generalisation reflects intertwined limitations in NLP methods, dataset construction, and online hate speech.

  • Models.: Reducing reliance on lexical features may alleviate overfitting, while linguistic patterns and hate-speech sentiment can inform model design, feature extraction, and preprocessing.Future work should examine how features of different natures can be combined effectively.
  • Models.: Debiasing approaches can reduce model bias against terms or language styles, but systematic cross-study comparisons remain difficult and are needed.Comparisons should apply existing approaches across multiple datasets using consistent attribute definitions.
  • Models.: Model evaluation should extend beyond dataset-level F1/AUC to unseen datasets and in-depth error analysis targeting the specific challenge addressed.Unseen-data evaluation is explicitly identified as important for assessing generalisation.
  • Models.: Hate speech detection models should be studied as components of wider sociotechnical systems rather than as algorithms isolated from their application context.The paper calls for future work examining models in broader contexts of use.
  • Models.: Applications involving human moderators raise questions about using model outputs, effects on moderation bias, and impacts on platform users, requiring interdisciplinary collaboration.The passage specifically asks how moderators can use outputs or post-hoc feature analysis effectively.
  • Conclusion: Existing hate speech detection models generalise poorly to new, unseen datasets because limitations in NLP methods, dataset building, and online hate speech are intertwined.Small datasets increase deep-learning overfitting, while dataset biases transfer to models.
Loading 2102.08886v1…