Source-linked AI summary
Hate Speech Dataset from a White Supremacy Forum
Ona de Gibert, Naiara Perez, Aitor García-Pablos, Montse Cuadros
TL;DR
Online hate-speech detection needs better public data because annotation is difficult and existing resources are limited. The paper constructs and studies a manually labeled, publicly available Stormfront dataset, finding that sentences requiring additional context are harder for automatic classifiers and that annotators need less context over time.
Problem
Few public hate-speech resources exist, and annotation is difficult because labeling can depend on definitions, annotator knowledge, and context.
Method
The paper manually annotates approximately 10,000 Stormfront sentences, studies annotation criteria, context needs, vocabulary, and baseline CNN and LSTM classifiers.
Results
Automatic-classifier results are lower when sentences requiring additional context are included, with NOHATE sentences classified more accurately than HATE sentences.
Takeaways & Limitations
The publicly available dataset supports hate-speech detection research while explicitly including difficult cases involving additional context or world knowledge.
Takeaways & Limitations
Sentence-level annotation can miss hate speech conveyed only through combinations of sentences, which the dataset labels as RELATION cases.
Abstract
from arXiv · showhide
Hate speech is commonly defined as any communication that disparages a target group of people based on some characteristic such as race, colour, ethnicity, gender, sexual orientation, nationality, religion, or other characteristic. Due to the massive rise of user-generated web content on social media, the amount of hate speech is also steadily increasing. Over the past years, interest in online hate speech detection and, particularly, the automation of this task has continuously grown, along with the societal impact of the phenomenon. This paper describes a hate speech dataset composed of thousands of sentences manually labelled as containing hate speech or not. The sentences have been extracted from Stormfront, a white supremacist forum. A custom annotation tool has been developed to carry out the manual labelling task which, among other things, allows the annotators to choose whether to read the context of a sentence before labelling it. The paper also provides a thoughtful qualitative and quantitative study of the resulting dataset and several baseline experiments with different classification models. The dataset is publicly available.
1 Introduction
The paper motivates hate-speech detection as increasingly necessary amid expanding online content and presents a publicly available sentence-level dataset from Stormfront.
- Online content growth makes monitoring social-network speech impossible, while cyberbullying, cyberterrorism, and online hate increase the need for identification tools.
- Hate speech is commonly defined as communication disparaging a target group based on characteristics such as race, ethnicity, gender, nationality, or religion.
- Identifying hate speech is difficult because some definitions also include the speaker’s determination to inflict harm.
- Across much of the literature, hate speech is characterized as a deliberate attack on a specific group motivated by actual or perceived identity-related characteristics.
- The paper presents the first public English dataset of sentence-level hate-speech annotations from Stormfront, a large white-nationalist forum, and makes it publicly available.
2 Related Work
Prior hate-speech research uses diverse datasets and task definitions, but few public resources exist; this paper addresses that gap with sentence-level annotation from a white-supremacist forum.
- Few publicly available hate-speech resources exist because researchers commonly collect and label their own datasets.
- Existing resources include multilingual hate vocabulary, toxic Wikipedia comments, and labeled racist, sexist, or harmful tweets.
- Keyword-based approaches can identify a topic but fail to distinguish hateful from clean sentences because both communities may share vocabulary with different intentions.
- Detection studies address subtypes, hate versus offensive language, or hate versus clean comments, using task-specific class schemes.
- Experts identify hate-speech annotation as difficult and emphasize annotator knowledge and detailed instructions as important considerations.
- The paper uses strict guidelines and sentence-level English annotations from white-supremacist forum posts, with access to each sentence’s complete original post.
3 Hate Speech Dataset
The dataset provides sentence-level hate-speech labels from Stormfront, together with metadata that supports reconstructing the conversations containing each sentence.
- Sentence-level annotation isolates the minimum unit containing hate speech and reduces noise from other clean sentences in the same comment.
- 10,568 Stormfront sentences were classified as conveying hate speech or not, with two additional auxiliary classes.
- Each sentence includes post, position, user, and sub-forum identifiers, enabling reconstruction of its surrounding conversations.
3.1 Data extraction and processing
The authors sampled Stormfront posts across diverse sub-forums and users, retained English content, segmented posts into sentences, batched them for annotation, and removed extreme sentence lengths.
- Stormfront content published between 2002 and 2017 was scraped into a database organized by sub-forums and conversation threads.
- A random sample of 22 sub-forums produced posts distributed across sub-forums and users, after which an automatic language detector discarded non-English texts.
- The retained posts were segmented into sentences and grouped into batches of 500 complete posts, approximately 1,000 sentences per batch.
- Manual annotation covered 10,568 sentences in 10 batches, with procedures and guidelines refined during the first two batches.
- 652 sentences shorter than 3 words or longer than 50 words were removed, representing 6.17% of the original dataset.
3.2 Annotation guidelines
The guidelines define HATE through three jointly necessary premises: a deliberate attack on a specific group motivated by an aspect of that group’s identity. They also distinguish context-dependent RELATION cases and exclude non-attacks, irrelevant sentences, and insufficiently informative text.
- HATE: HATE requires a deliberate attack directed at a specific group and motivated by an aspect of the group’s identity.All three premises must hold for a sentence to receive the HATE label.
- HATE: Examples qualify as HATE when insults or negative descriptions target groups based on skin colour or origin.The examples target dark-skinned children and people of Arab origin, respectively.
- NOHATE: NOHATE applies when a sentence is not an attack on a group, including discussion of National Socialism or hostility toward an individual.The examples fail the deliberate-attack or group-targeting premise despite potentially offensive content.
- NOHATE: Pejorative language alone is insufficient evidence of hate speech when the sentence does not clearly constitute a deliberate group-directed attack.The phrase about two Black people holding hands is treated as offensive but not HATE without additional context.
- RELATION: RELATION labels consecutive sentences whose combination conveys hate speech even though the individual sentences depend on one another for interpretation.The label captures cases where group targeting and hostility emerge only across multiple sentences.
- SKIP: The guidelines assign a separate label to non-English or otherwise unclassifiable sentences rather than forcing HATE or NOHATE decisions.The examples include a Norwegian sentence and an irrelevant YouTube phrase.
3.3 Annotation procedure
Annotation guidelines were iteratively developed by three authors, then evaluated on a separate batch using inter-annotator agreement measures. A web tool exposed same-post context and optionally preceding thread posts because context can affect interpretation and speaker intention.
- Guideline development: Three authors first annotated 1,144 sentences using draft HATE, NOHATE, and SKIP categories before revising the guidelines.They discussed the initial annotations and modified the draft, adding the RELATION category.
- Agreement evaluation: A separate batch of 1,018 sentences was annotated by the same three authors under the revised guidelines to calculate inter-annotator agreement.The revised scheme included RELATION alongside the original categories.
- Agreement evaluation: Table 1 reports average percent agreement, average Cohen’s kappa, and Fleiss’ kappa for each batch, together with sentence and category counts.The reported results are described as consistent with similar studies.
- Annotation tool: The annotation tool displayed all sentences from a post and allowed annotators to retrieve earlier thread posts when the immediate context was insufficient.This design supported interpretation of the author’s intention during sentence-level labelling.
- Annotation tool: Prior studies likewise characterize context as important because message classification and speaker intention can depend on it.The paper uses this literature to motivate contextual support during annotation.
3.4 Dataset statistics
The clean dataset is unbalanced across sentence categories, while annotation context is especially important for HATE sentences. Its account distribution is balanced, and vocabulary analyses distinguish HATE from NOHATE and cover all Hatebase categories.
- The clean dataset contains many more NOHATE than HATE sentences.
- HATE sentences require additional context more often, commonly because of unknown slang or unclear offensive targets.
- Across 2,723 Stormfront accounts, sentence contributions are balanced, averaging 0.50 ± 0.42 percent per account.
- A HATE score contrasts each word’s PMI association with HATE against its PMI association with NOHATE.
- The most hateful vocabulary is derogatory or group-targeting, whereas the least hateful vocabulary is more neutral and associated with Internet or temporal expressions.
- HATE vocabulary overlaps Hatebase by 9.28%, versus 6.57% for NOHATE vocabulary, with all eight Hatebase categories represented.
4 Experiments
Baseline experiments evaluate HATE-versus-NOHATE classification on a balanced dataset using standard models and accuracy measures. Performance declines when context-dependent sentences are included; LSTM performs best overall, while SVM remains reasonably effective.
- Experimental setup: The experiments use a balanced set of 2,000 labelled sentences, with 80% for training and 20% for testing.
- Experimental setup: The evaluated classifiers are Bag-of-Words LinearSVM, a simplified CNN with randomly initialized embeddings, and an LSTM with 128 units over 300-dimensional word embeddings.
- Experimental setup: No hyperparameter tuning was performed, and more comprehensive experimentation was left for future work.
- Evaluation: Results report HATE accuracy, NOHATE accuracy, and overall accuracy alongside a majority-class baseline, using true- and false-positive counts.
- Results: Including sentences that required additional context lowers results; NOHATE is classified more accurately than HATE, and LSTM performs best overall.
- Results: Even the simple Bag-of-Words SVM can reasonably discriminate between HATE and NOHATE.
- Error analysis: False NOHATE predictions often reflect missing context or world knowledge needed to identify an implied target or interpret references such as Holocaust denial.
- Error analysis: Manual error analysis identifies false HATE predictions caused by offensive vocabulary used without hateful intent, such as insults aimed at individuals or reports of crimes.
5 Discussion
The discussion highlights the subjectivity and context dependence of sentence-level hate-speech annotation. It also explains the dataset’s granularity and the annotators’ decreasing need for additional context over time.
- Stormfront’s pervasive racism does not make every racist expression hate speech, requiring explicit annotation criteria.The distinction is tied to debates about free speech, tolerance, and civics.
- Annotation remained demanding because criteria such as deliberate attack and group targeting were ambiguous and interpreted differently by annotators.The guidelines were revised after annotators discussed inconsistencies and debated difficult examples.
- Sentence-level labels preserve distinctions that comment-level annotation can obscure, since not every sentence in a comment conveys hate.The dataset provides all sentences per comment with their annotations, allowing researchers to choose how to use them.
- RELATION marks cases where multiple sentences must be understood together as hate speech, but the label was seldom used.It typically links a premise to a following hateful conclusion.
- Annotators needed less additional context over time, reflecting improved ability to distinguish hate speech during the labelling process.Additional context can include the rest of the conversation or the forum-thread title.
6 Conclusions and Future Work
The paper releases a manually labelled Stormfront dataset of approximately 10,000 sentences and studies its annotation and classification properties. It presents the dataset as a starting point for further research, including context and world-knowledge integration.
- The dataset contains approximately 10,000 Stormfront sentences labelled as conveying hate speech or not.
- The paper documents detailed annotation criteria and studies context requirements, hate-speech vocabulary, and baseline automatic classifiers.
- The dataset provides a starting point for discussion and further research on robust hate-speech classification.
- Future work could incorporate online-conversation context and world knowledge, examine RELATION sentences, and further characterize the dataset.