Source-linked AI summary
Predicting the Type and Target of Offensive Posts in Social Media
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, Ritesh Kumar
TL;DR
Existing research often targeted narrow forms of offensive content rather than modeling the problem as a whole. This paper introduces a hierarchical annotation scheme and the OLID dataset for identifying offense type and target, then evaluates machine-learning models and establishes baselines. The experiments show strong offensive-versus-non-offensive detection and better performance for targeted insults than untargeted offenses, while the Other target class remains difficult.
Problem
Prior studies focused on specific offensive-language categories, motivating a broader treatment of offensive content, its type, and its target.
Method
The paper creates OLID with a three-level hierarchical annotation scheme and uses crowdsourcing with quality controls before evaluating machine-learning models at each level.
Results
The CNN achieves macro-F1 0.80 for offensive-versus-non-offensive detection and 0.69 for targeted-insult versus untargeted-offense categorization, while Other-target performance is 0.
Takeaways & Limitations
OLID provides a publicly available resource and baselines for identifying and characterizing offensive language across annotation levels.
Takeaways & Limitations
The Other target class is difficult because it is heterogeneous and has fewer training instances than the Individual and Group classes.
Abstract
from arXiv · showhide
As offensive content has become pervasive in social media, there has been much research in identifying potentially offensive messages. However, previous work on this topic did not consider the problem as a whole, but rather focused on detecting very specific types of offensive content, e.g., hate speech, cyberbulling, or cyber-aggression. In contrast, here we target several different kinds of offensive content. In particular, we model the task hierarchically, identifying the type and the target of offensive messages in social media. For this purpose, we complied the Offensive Language Identification Dataset (OLID), a new dataset with tweets annotated for offensive content using a fine-grained three-layer annotation scheme, which we make publicly available. We discuss the main similarities and differences between OLID and pre-existing datasets for hate speech identification, aggression detection, and similar tasks. We further experiment with and we compare the performance of different machine learning models on OLID.
1 Introduction
The paper addresses fragmented offensive-language research by proposing a hierarchical framework that identifies whether language is offensive, its type, and its target. It introduces OLID, a publicly available annotated tweet dataset, and evaluates models to establish baselines.
- Prior work examined specific forms of offensive language, including abuse, aggression, bullying, toxic comments, hate speech, and offensive language.
- The paper proposes a three-level schema covering offensive-language detection, offense categorization, and target identification.
- OLID is a new large-scale English-tweet dataset annotated for the type and target of offenses and made publicly available.
- Experiments apply different machine-learning models at each annotation level, providing baselines for future comparisons.
- The hierarchical schema captures shared properties across tasks such as cyberbullying and hate speech, supporting multiple offensive-language identification and characterization tasks.
2 Hierarchically Modelling Offensive Content
OLID models offensive content through three hierarchical levels: offensive versus non-offensive language, offense type, and insult or threat target. The schema distinguishes targeted insults from untargeted profanity and separates individual, group, and other targets.
- OLID uses three levels to distinguish whether a tweet is offensive, what type of offense it contains, and whom an offense targets.
- Level A: Offensive Language Detection: Level A labels tweets as Not Offensive (NOT) or Offensive (OFF), including profanity and targeted insults or threats.
- Level B: Categorization of Offensive Language: Level B distinguishes Targeted Insult (TIN) from Untargeted (UNT) offenses, with UNT covering nontargeted profanity and swearing.
- Level C: Offensive Language Target Identification: Level C categorizes insult or threat targets as Individual (IND), Group (GRP), or Other (OTH), including organizations, situations, events, and issues.
- Table 2 lists dataset keywords and the percentage of offensive tweets associated with each keyword.
3 Data Collection
OLID was collected from keyword- and construction-based Twitter searches, with sampling and annotation procedures designed to improve offensive-content coverage and label quality. The dataset’s label combinations were uneven, creating a recurring challenge in obtaining sufficient instances for every class.
- Retrieval: Researchers retrieved OLID tweets through Twitter’s API using keywords and constructions often found in offensive messages.Examples included “she is” and the account-directed construction “to:BreitBartNews.”
- Trial annotation: A 300-instance trial with six experts evaluated the tagset and retrieval method and created test questions for crowdsourced annotation.The trial used nine keywords before the full collection process.
- Sampling: Sampling strategies targeted an offensive-tweet distribution of around 30% because most Twitter content is not offensive.Keywords with low offensive-content rates were excluded, while “he is” was retained to avoid gender bias despite low trial yield.
- Sampling: The full dataset was balanced between political and non-political keyword sources, with 50% of tweets drawn from each group.Tweets within each group were sampled evenly across keywords.
- Annotation: Crowdsourced annotation used experienced Figure Eight workers, test questions, multiple annotations, and majority voting after disagreement.Annotators labeled every tweet at all three levels, and agreement required matching labels across the complete hierarchy.
- Data balance: Producing sufficient instances for every class was a key collection challenge, particularly for Subtasks B and C.The paper notes similar class-sparsity problems in other offensive-language datasets.
4 Experiments and Evaluation
The experiments compare linear and neural models across OLID’s hierarchical tasks using macro-averaged F1 and per-class metrics. Neural models generally outperform the SVM, but the heterogeneous and underrepresented OTH target class remains difficult to identify.
- Models: The study compares a linear unigram SVM, a BiLSTM, and a CNN across the annotation levels.The CNN follows Kim’s architecture and uses the same multi-channel inputs as the BiLSTM.
- Evaluation: Macro-averaged F1 is the primary comparison metric because OLID’s label distribution is highly imbalanced.The evaluation also reports per-class Precision, Recall, F1, weighted averages, and majority and minority baselines.
- Level A: Offensive Language Detection: 0.80 macro-F1: the CNN outperforms the RNN on offensive-versus-non-offensive detection, while neural models substantially outperform the SVM.All models perform significantly better than chance.
- Level B: Offensive Language Categorization: 0.69 macro-F1: the CNN outperforms the BiLSTM when distinguishing targeted insults and threats from untargeted offenses.All models identify targeted offenses better than untargeted offenses.
- Level C: Offensive Language Target Identification: All three target-identification models achieve similar results and surpass random baselines, with a slight performance edge for neural models.The task distinguishes group, individual, and other targets.
- Level C: Offensive Language Target Identification: 0 performance for OTH: every model fails on the heterogeneous other-target class, which has fewer examples than GRP and IND.The paper attributes this to heterogeneous targets and only 395 OTH instances versus 1,075 GRP and 2,407 IND instances.
5 Conclusion and Future Work
The paper presents OLID as a dataset annotating both the type and target of offensive language, with each annotation level used as an independent sub-task. Baseline experiments indicate that the task is challenging but feasible, motivating cross-corpus and multilingual extensions.
- OLID annotates the type and target of offensive language and serves as the official dataset for SemEval 2019 Task 6.
- Each annotation level in OLID constitutes an independent OffensEval sub-task.
- Baseline experiments using SVMs and neural networks show that offensive-language identification is challenging yet doable.
- Future work includes cross-corpus comparisons with aggression and hate-speech datasets and creating similarly annotated datasets for other languages.