PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting Online Hate Speech Using Context Aware Models

Lei Gao, Ruihong Huang

arXiv:1710.07395v2cs.CL

TL;DR

The paper addresses the limited use of context in online hate-speech datasets and detection models. It introduces a context-preserving annotated corpus and context-aware logistic regression, neural, and ensemble models. The final ensemble models outperform a strong baseline by around 10% in F1-score.

  • Problem

    Existing hate-speech datasets and detection models largely overlook context, although context is useful for identifying subtle and creative hate speech.

  • Method

    The paper creates a context-preserving annotated corpus and evaluates logistic regression and neural network models that incorporate context information.

  • Results

    Around 10% in F1-score: final ensemble models outperform a strong baseline system.

  • Takeaways & Limitations

    Ensemble models leveraging the strengths of logistic regression and neural networks achieve the best performance for automatic online hate-speech detection.

Abstract

from arXiv · show

In the wake of a polarizing election, the cyber world is laden with hate speech. Context accompanying a hate speech text is useful for identifying hate speech, which however has been largely overlooked in existing datasets and hate speech detection models. In this paper, we provide an annotated corpus of hate speech with context information well kept. Then we propose two types of hate speech detection models that incorporate context information, a logistic regression model with context features and a neural network model with learning components for context. Our evaluation shows that both models outperform a strong baseline by around 3% to 4% in F1 score and combining these two models further improve the performance by another 7% in F1 score.

1 Introduction

The paper argues that context is important for detecting subtle online hate speech, yet existing datasets and models largely overlook it. It introduces a context-preserving corpus and context-aware logistic regression, neural, and ensemble models.

  • Context accompanying hate speech is useful for detection but has been overlooked in existing datasets and automatic models.
  • Context can reveal hatefulness that is unclear from a comment alone, as shown by a comment responding to a news article about rape-law reform.
  • The corpus contains 1528 Fox News comments from 10 complete discussion threads and preserves users, thread comments, and source articles.
  • The paper evaluates logistic regression with context-derived features and neural networks with separate components for modeling context.
  • Around 10% in F1-score: the final ensemble models outperform a strong baseline system.

2 Related Works

Existing hate-speech datasets and detection models largely focus on target text while omitting contextual information. Sampling practices also tend to produce instances from distinct contexts rather than naturally occurring contextual sequences.

  • Most existing hate-speech datasets do not preserve context information.Human-labeled datasets exist, but context is generally absent.
  • Because hate speech is sparse in everyday posts, researchers often bootstrap candidate samples to increase its representation.This differs from relying solely on random sampling.
  • Bootstrapped collection makes instances likely to come from distinct contexts.
  • Most existing detection models use features derived from the target text itself.Examples include n-grams, dependency paths, hateful terms, and paragraph2vec representations.

3 The Fox News User Comments corpus

The Fox News User Comments corpus contains annotated comments from ten manually selected discussion threads and preserves rich contextual information for each comment.

  • The corpus contains 1528 annotated comments, including 435 labeled as hateful.
  • The comments were posted by 678 different users across 10 complete Fox News discussion threads.
  • The ten threads were manually selected as popular discussions during August 2016, with roughly equal numbers of comments.
  • Rich context information, including user information, was retained for every comment.

3.2 Annotation Guidelines

The annotation guidelines define hateful speech by identity-based threats or demeaning language and assign each comment one of two labels.

  • Hateful speech is language that explicitly or implicitly threatens or demeans a person or group based on identity facets.Examples include gender, ethnicity, and sexual orientation.
  • The guidelines cover identity-related characteristics such as gender, ethnicity, and sexual orientation.
  • The corpus uses binary labeling, assigning each comment either hateful or non-hateful.

3.3 Annotation Procedure

Two native English speakers annotated the comments after discussion and practice, achieving high inter-annotator agreement on an initial subset. Disagreements were labeled hateful when either annotator assigned that label.

  • Two native English speakers annotated the online user comments after discussing and practicing before annotation began.
  • Cohen’s Kappa was 0.98 on 648 comments from four discussion threads.
  • When annotators disagreed, comments were labeled hateful if at least one annotator assigned the hateful label.
  • One annotator completed annotation of the remaining 880 comments from six discussion threads.

3.4 Characteristics in Fox News User Comments corpus

Hateful comments in the corpus are often subtle, implicit, and context-dependent, while some user screen names themselves signal hatefulness. Long comments may also contain hate indicators within mostly neutral text.

  • Context dependence: Understanding the associated news context can reveal stereotyping that is not explicit in the original comment.A comment about work requirements for food stamps was interpreted as stereotyping Latino immigrants when paired with its news title.
  • Implicit and creative language: Many hateful comments use subtle and implicit language without clear hate-indicating words or phrases.
  • Long comments: 11% of annotated comments contain more than 50 words, with hateful indicators often concentrated in a small region of otherwise neutral text.This concentration is presented as a characteristic of long comments in the corpus.
  • Disrespectful screen names: Some user screen names indicate hatefulness, making comments posted by those users likely to contain hate speech.The paper gives “commie” as a slur for communists and illustrates this with the username “nocommie11.”

4 Context-aware Online Hate Speech Detection Models

The paper incorporates comment context through feature-based logistic regression and neural networks with separate components for the comment, news title, and username, then evaluates ensembles of both model types.

  • Feature-based logistic regression: Logistic regression extracts word- and character-level n-grams plus lexicon features from the comment, news title, and username.The same four feature types are extracted from the target comment and two context sources.
  • Feature-based logistic regression: Character features use bigrams, trigrams, and four-grams, while word features use unigrams and bigrams.
  • Lexicon-derived features: LIWC represents each word with 125 semantic categories and sums the resulting vectors across a comment or its context.
  • Lexicon-derived features: The NRC emotion feature represents eight emotions and two sentiment polarities in a summed 10-dimensional vector.
  • Neural network models: The neural model uses parallel LSTMs for the target comment, news title, and username, concatenating their outputs before sigmoid prediction.Attention is applied to the bidirectional LSTM encoding the comment, but not to the shorter context inputs.
  • Ensemble models: Ensemble models combine prediction scores from the context-aware logistic regression and neural network models using maximum or average scores.

5 Evaluation

Evaluation used 10-fold cross-validation and multiple metrics. Context-aware logistic and neural models improved performance, while ensembles achieved the strongest results.

  • Evaluation setup: 10-fold cross-validation evaluated accuracy, precision, recall, F1-score, and AUC on the Fox News User Comments Corpus.Both model types used identical training and test folds.
  • Logistic regression models: Adding word-level n-grams and lexicon features improved logistic regression across every reported metric, though the gains were moderate.
  • Logistic regression models: 2.8% AUC and 2.5% F1-score improvements came from using comment features together with both context types in logistic regression.
  • Neural network models: 5.7% AUC improvement resulted from adding attention to the bidirectional LSTM for comment-only neural hate speech detection.
  • Neural network models: News title context produced the best neural-network F1 score, whereas combining news title and username produced the best AUC score.
  • Ensemble models: Around 10% F1 improvement and more than 20% recall improvement over the logistic regression baseline were achieved by the Max Score Ensemble with comparable precision.
  • Ensemble models: Around 7% AUC improvement over the logistic regression baseline was achieved by the Average Score Ensemble.

6 Analysis

Context features substantially improve logistic-regression hate-speech detection, while attention-based neural models capture concentrated and implicit hateful language. Their complementary strengths motivate ensembles that achieve the strongest performance.

  • 6.1 Logistic Regression Models: Around 4% improvement in both F1 score and AUC follows from adding news-title and username features to logistic regression.Text-only feature additions improved F1 by only 1.3% over the baseline.
  • 6.2 Neural Network Models: Attention significantly improves bidirectional LSTM performance by focusing on small regions where hate indicators concentrate.This is especially relevant for long comments.
  • 6.3 Ensemble Models: Both ensemble models significantly improve hate-speech detection, and the two component models perform differently on hateful comments.The ensembles combine predictions from context-aware logistic regression and neural network models.
  • 6.3.1 Strengths of Logistic Regression Models: Character-level n-grams help logistic regression identify hateful comments containing out-of-vocabulary, capitalized, or misspelled words.The model correctly handles the deliberate misspelling “femanazis.”
  • 6.3.2 Strengths of Neural Network Models: Neural networks capture implicit hateful language and stereotyping whose hateful meaning emerges from the composition of otherwise non-hateful words.One example targets religions, while another becomes hateful only when its four words are combined.

7 Conclusion

The paper demonstrates the importance of context for online hate-speech detection through a context-preserving corpus and two context-aware model types. Ensembles leveraging both models achieve the best performance.

  • 7 Conclusion: The paper presents a full-thread hateful-speech corpus and context-aware logistic regression, neural-network, and ensemble models.The ensemble models leverage the strengths of both model types.
Loading 1710.07395v2…