Source-linked AI summary
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, Animesh Mukherjee
TL;DR
Hate-speech detection research has limited evidence on model bias and interpretability, despite concerns about generalization and biased predictions. HateXplain introduces a benchmark dataset whose posts receive class, target-community, and rationale annotations, then evaluates models across performance, bias, and explainability. Models using human rationales reduce unintended bias, but strong classification performance does not ensure plausible or faithful rationales.
Problem
Hate-speech models can fail to generalize, produce biased predictions toward commonly attacked identities, and lack explanations for their decisions.
Method
HateXplain contains around 20K Twitter and Gab posts annotated with hate, offensive, or normal labels, target communities, and human-highlighted rationale spans.
Results
Models using human rationales improve reduction of unintended bias, while models with strong classification performance do not always provide plausible and faithful rationales.
Takeaways & Limitations
HateXplain provides a benchmark for evaluating hate-speech detection beyond classification, including model bias and explanation quality.
Takeaways & Limitations
The work excludes external context such as user profiles and post histories and focuses on English rather than multilingual hate speech.
Abstract
from arXiv · showhide
Hate speech is a challenging issue plaguing the online social media. While better models for hate speech detection are continuously being developed, there is little research on the bias and interpretability aspects of hate speech. In this paper, we introduce HateXplain, the first benchmark hate speech dataset covering multiple aspects of the issue. Each post in our dataset is annotated from three different perspectives: the basic, commonly used 3-class classification (i.e., hate, offensive or normal), the target community (i.e., the community that has been the victim of hate speech/offensive speech in the post), and the rationales, i.e., the portions of the post on which their labelling decision (as hate, offensive or normal) is based. We utilize existing state-of-the-art models and observe that even models that perform very well in classification do not score high on explainability metrics like model plausibility and faithfulness. We also observe that models, which utilize the human rationales for training, perform better in reducing unintended bias towards target communities. We have made our code and dataset public at https://github.com/punyajoy/HateXplain
Introduction
HateXplain addresses generalization, bias, and interpretability problems in hate-speech detection by introducing a dataset with labels, target communities, and human rationales. Model examples show that correct classification can still rely on rationales that differ from human annotations.
- Existing hate-speech models may fail to generalize despite claiming state-of-the-art performance on particular datasets.
- HateXplain collects posts from Twitter and Gab with three annotations: hate, offensive, or normal labels; target communities; and highlighted rationale spans.
- Models can predict hate speech correctly while using rationales that differ from the tokens human annotators considered important.
- The dataset is a benchmark with word- and phrase-level human rationale annotations for hate-speech labeling.
- The paper reports that models using human rationales during training improve performance and reduce unintended bias.
Related work
Prior hate-speech research developed datasets, models, and rationale methods, but often conflated hate with offensive language. HateXplain separates these classes while extending rationale-based annotation to a benchmark dataset with human-level explanations.
- Hate-speech research has produced datasets, detection models, and shared tasks, while online hate speech remains a documented social concern.
- Many prior studies combine hate and abusive or offensive language under one concept, limiting their separation.
- HateXplain uses three classes—hate, offensive, and normal—to distinguish offensive messages that do not qualify as hate speech.
- Earlier rationale research used human-highlighted spans for sentiment classification and later developed methods to generate rationales automatically.
- HateXplain applies rationales to hate speech and provides a benchmark dataset with human-level explanations.
Dataset collection and annotation strategies
HateXplain combines lexicon-based posts from Twitter and Gab with MTurk annotations covering class labels, target communities, and rationale spans. The dataset design uses majority voting, quality controls, and explicit handling of ambiguous or incomplete annotations.
- Dataset collection: Posts were collected from Twitter and Gab using combined hate-speech lexicons, with reposts removed and duplicates filtered.Twitter data came from randomly collected tweets from January 2019 to June 2020; Gab data came from an existing dataset.
- Annotation design: Each post received hate, offensive, or normal labels, target-community annotations, and rationale spans identifying words or phrases supporting the label.Rationale spans were requested only when the majority label was hateful or offensive.
- Annotation quality: Workers were shown examples, warned about hateful content, and screened using MTurk requirements including a 95% HIT approval rate and 5,000 approved HITs.A pilot task also shortlisted annotators who classified posts and identified target communities accurately.
- Label aggregation: Class labels were determined by majority voting, while 919 posts with three different class judgments were excluded from analysis.The reported inter-annotator agreement was Krippendorff’s α = 0.46.
- Target communities: Target communities were assigned by majority vote, required selection by at least two of three annotators, and were retained only when present in at least 100 posts.The resulting dataset included ten communities, including African, Islam, Jewish, Gay, Women, Refugee, Arab, Caucasian, Hispanic, and Asian.
- Rationale annotation: Hateful and offensive posts received rationale annotations from two or three annotators, averaging 5.48 highlighted tokens for offensive and 5.47 for hate speech.Rationales were converted into token-level attention vectors and normalized with a temperature-tuned softmax to form ground-truth attention.
Metrics for evaluation
HateXplain evaluates classification, unintended identity-based bias, and explanation quality as distinct aspects of hate speech detection.
- The benchmark evaluates three-class classification among hate, offensive, and normal speech using accuracy, macro F1-score, and AUROC.
- Bias evaluation measures whether models reduce unintended bias toward target communities using Subgroup AUC, BPSN AUC, BNSP AUC, and generalized mean bias metrics.
- The bias task converts labels into toxic speech—hate or offensive—versus normal speech and evaluates models on the test set.
- Explainability is assessed through plausibility, which reflects human convincingness, and faithfulness, which reflects correspondence with model reasoning.
- Plausibility uses IOU F1-score, token F1-score, and AUPRC for discrete and soft rationale selection.
- Comprehensiveness measures prediction-probability loss after removing predicted rationales, while sufficiency measures whether rationales alone support the prediction.
Model details
The study compares classification models trained with class labels alone against models additionally supervised by human-derived attention, using a shared evaluation setup.
- Each model has a class-label-only version and, where feasible, a version trained with ground-truth attention and class labels.
- For explainability evaluation, the models’ top five tokens are selected as rationales because five is the dataset’s average annotation-span length.
- Table 6 selects tokens for explainability calculations using attention and LIME methods.
- Attention-capable models train attention outputs against ground-truth attention using an attention loss whose contribution is controlled by λ.
- BiRNN and CNN-GRU cannot use the ground-truth-attention training version because they lack an attention mechanism.
- The benchmark evaluates CNN-GRU, BiRNN, BiRNN with attention, and BERT architectures, with attention supervision added to attention-capable models.
- The shared data split is stratified into train, development, and test sets in an 8:1:1 ratio, with development data used for hyperparameter tuning.
Results
Human-rationale supervision slightly improves performance and bias reduction, but the strongest-performing models do not consistently provide the most plausible or faithful explanations.
- Models trained with human rationales perform slightly better on performance metrics and better reduce unintended model bias across all reported bias metrics.
- BiRNN-HateXplain improves all plausibility metrics and comprehensiveness relative to BiRNN-Attn, while BERT-HateXplain [LIME] improves faithfulness but decreases plausibility.
- Community terms included in rationales are effective in reducing unintended bias, although Hispanic communities show more false positives.
- The best performance- and bias-scoring model, BERT-HateXplain [LIME & Attn], does not perform well on plausibility explainability metrics.
- Individual-community bias varies substantially: overall GMB-Subgroup-AUC is approximately 0.8, while Asian communities score approximately 0.7 even for the best model.
Limitations of our work
The study’s models omit external user context and focus exclusively on English-language hate speech.
- The models do not use external context such as profile biographies, user gender, or posting history, although these signals might help classification.
- The study focuses on English and does not consider multilingual hate speech.
Conclusion and future work
HateXplain is introduced as a 20K-post benchmark annotated with labels, target communities, and human rationales. State-of-the-art models can classify well without consistently producing plausible and faithful rationales; future work will incorporate existing datasets.
- HateXplain contains 20K posts from Gab and Twitter annotated with hate, offensive, or normal labels, target communities, and human-marked rationales.
- Models that perform very well in classification cannot always provide plausible and faithful rationales for their decisions.
- Future work: The authors plan to incorporate existing hate speech datasets into the HateXplain framework.
Interface design
The annotation interface separates message labeling from rationale highlighting. Annotators classify each message by speech type and target, then identify text portions that justify the label.
- Interface design: The interface first classifies each message as hate speech, offensive, or normal and records its target.The classification task is shown in Figure 4.
- Interface design: Annotators then highlight portions of the text that could justify the hate-speech or offensive label.The rationale task is shown in Figure 5.
- Interface design: The workflow provides example annotations, highlights, and sample test cases to support the annotation task.
Attention supervision in BERT
The method supervises selected final-layer BERT attention heads with human rationale annotations. It uses attention-based cross-entropy regularization so the [CLS] representation attends toward rationale-relevant words.
- Attention supervision in BERT: BERT attention produces token representations from attention heads, combines heads within each layer, and applies a fully connected layer with skip-connections and layer normalization.Each token’s attention values are based on other tokens in the sequence.
- Attention supervision in BERT: Attention supervision uses x of BERT’s 12 heads in the last layer, with the supervised-head count treated as an experimental hyper-parameter.The selected heads are called supervised heads.
- Attention supervision in BERT: For each supervised head, the model computes cross-entropy between [CLS] attention weights and the ground-truth attention vector.The ground-truth attention vector is illustrated in Figure 7.
- Attention supervision in BERT: The attention-supervision loss averages the supervised-head losses and multiplies the result by the regularizer λ.
- Attention supervision in BERT: Examples report that BERT-HateXplain can attend better than BERT using token-level rationales, with correct predictions in the shown example.The passage also describes wrong or incomplete attention as one reason for incorrect BERT predictions and notes correct attended words for many BERT-HateXplain false positives.
- Attention supervision in BERT: Rationale quality is evaluated using average pairwise Jaccard overlap between ground-truth annotations and randomly generated rationales.The random baseline selects five random tokens per sentence and repeats the trial three times to represent three annotators.