Source-linked AI summary
Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, Nicolas Kourtellis
TL;DR
Abusive behavior on Twitter is difficult to study because categories are nuanced and abusive tweets are rare, making reliable large-scale ground truth costly to collect. The paper uses iterative crowdsourcing, boosted sampling, and label analysis to annotate tweets and refine the label set. It reports high agreement, increased representation of inappropriate tweets through boosted sampling, and publicly releases the resulting resources.
Problem
Large-scale abusive-behavior annotation is constrained by confusion among related labels, rare abusive examples, and the need to balance annotation quality with cost and time.
Method
The paper combines iterative crowdsourced multi-label annotation, preliminary rounds for resolving label ambiguity, and boosted sampling to increase minority-class coverage.
Results
55.9% of tweets achieved overwhelming agreement, while boosted sampling produced almost half inappropriate tweets compared with approximately 4% in the random sample.
Takeaways & Limitations
The study provides a methodology, annotation platform, and labeled Twitter dataset for further research on abusive behavior and machine-learning models.
Abstract
from arXiv · showhide
In recent years, offensive, abusive and hateful language, sexism, racism and other types of aggressive and cyberbullying behavior have been manifesting with increased frequency, and in many online social media platforms. In fact, past scientific work focused on studying these forms in popular media, such as Facebook and Twitter. Building on such work, we present an 8-month study of the various forms of abusive behavior on Twitter, in a holistic fashion. Departing from past work, we examine a wide variety of labeling schemes, which cover different forms of abusive behavior, at the same time. We propose an incremental and iterative methodology, that utilizes the power of crowdsourcing to annotate a large scale collection of tweets with a set of abuse-related labels. In fact, by applying our methodology including statistical analysis for label merging or elimination, we identify a reduced but robust set of labels. Finally, we offer a first overview and findings of our collected and annotated dataset of 100 thousand tweets, which we make publicly available for further scientific exploration.
Introduction
The paper addresses the difficulty of defining and labeling abusive behavior at scale, especially across nuanced categories and rare minority classes. It proposes crowdsourced, cost-aware resources and methods to improve label selection, sampling, and annotation quality.
- Challenges: Large-scale abusive-behavior annotation is difficult because categories overlap, minority examples are rare, and quality, cost, and time must be balanced.Workers may struggle to distinguish labels such as hate speech, offensive language, and abusive language, while random sampling yields too few abusive examples.
- Contributions: The paper develops a methodology to reduce confusion among crowdworkers distinguishing nuanced abusive-behavior labels.The approach uses preliminary annotation rounds to identify ambiguity and support label elimination or refinement.
- Contributions: Boosted sampling preserves an unbiased dataset while increasing annotations for minority classes.The method combines random data with tweets likely to belong to minority categories.
- Contributions: The authors design a crowdsourcing platform that optimizes annotation costs.The platform supports large-scale collection while addressing constraints on annotator payment and process duration.
- Contributions: The released resources include an 80k-tweet abusive-behavior dataset, the methodology used to create it, and open-source annotation-platform code.These resources are intended to support further research on abusive behavior.
Related Work
Prior Twitter datasets study several forms of inappropriate speech and harassment using different labels and annotation procedures. The paper identifies an unresolved need for principled, iterative selection of labels for large-scale abusive-behavior annotation.
- Existing datasets: Earlier Twitter datasets used varied labels and methodologies to annotate hate speech, offensive language, trolling, harassment, cyberbullying, and cyberaggression.Examples include distinctions between racism and sexism, hateful and offensive language, and multiple harassment categories.
- Existing datasets: Some prior work focuses on user behavior rather than only the language contained in individual tweets.This frames inappropriate-user detection as a related but distinct task from content classification.
- Research gap: Previous studies often select labels without explaining their choice, use only subsets of labels, or merge categories under broad umbrella labels.The paper treats these practices as limiting the principled characterization of abusive speech.
- Research gap: The paper responds with an iterative methodology that statistically analyzes annotator label selection under time and budget constraints.The final labels are then used for a large-scale crowdsourcing study of 80k tweets.
Overview of Methodology
The study uses crowdsourcing to annotate tweets with multiple possible abusive-behavior categories while controlling annotation cost. Preliminary rounds identify label confusion and guide the design of the main annotation task.
- Multi-label annotation: Crowdsourced workers select from two or more abusive-behavior categories so the study can examine correlations and adjust the final label set.The methodology treats label selection as an empirical design problem rather than fixing a single category scheme in advance.
- Iterative design: Preliminary annotation rounds identify workers’ confusion and support clearer, more accurate, and more consistent main-task labeling.The task wording and available labels are adjusted based on the observed ambiguity.
- Cost management: The platform and workflow are designed to minimize the cost of annotating a large dataset.Cost control is treated as a central constraint alongside annotation quality.
Challenges with Crowdsourced Platforms
Crowdsourced abuse annotation must balance label ambiguity, severe class imbalance, annotation quality, and cost. The methodology addresses these challenges through preliminary rounds, boosted sampling, and calibrated worker decisions.
- High-quality annotation requires balancing task design, label selection, and minimal cost.
- Preliminary annotation rounds identify label confusion and support eliminating ambiguous labels before the main task.This is intended to improve labeling accuracy and consistency.
- Abusive tweets comprise only 0.1%–3% of tweets depending on the label, creating extreme class imbalance.The resulting scarcity can limit machine-learning training examples.
- Boosted random sampling mixes randomly sampled tweets with tweets likely to belong to minority abuse classes.Text analysis and preliminary crowdsourcing help design the boosted set.
- Exploratory rounds use up to 20 annotators per tweet to estimate agreement and calibrate the required number of decisions.
- Worker payment begins at 5¢ per batch of 10 tweets and is adjusted through preliminary rounds.
Crowdsourcing Methodology
The crowdsourcing methodology uses iterative exploratory annotation to tune labels and task parameters before scaling to a larger dataset. A custom platform then supports the final annotation round with greater control over workflow and data.
- The methodology consists of preliminary data preparation followed by iterative annotation rounds.Figures 1–3 represent the three stages.
- A random tweet collection is filtered, augmented with a boosted set, and split into exploratory dataset D1 and large-scale dataset D2.
- The exploratory process tests payment, label design, judgments, worker trustworthiness, and annotation procedures on a small sample of 300 tweets.
- Iterative analysis fine-tunes boosting filters and evaluates whether parameter choices improve annotation quality.The process functions similarly to repeated A/B testing of design choices.
- Three iterations identified a representative label set and an appropriate number of judgments balancing annotation cost and quality.
- The final round annotates D2 with the established labels using a custom platform, database schema, and statistical calculations.
Step 1: Data collection and sampling
Step 1 collects and cleans a large Twitter sample, enriches tweets with metadata, and combines random and boosted sampling to address abuse-class imbalance. It produces D1 for exploration and D2 for final annotation.
- The study collected 32 million tweets from Twitter’s 1% Stream API traffic between 30 March and 9 April 2017.
- Tweets were enriched with content and account metadata, including URLs, hashtags, mentions, emojis, numerals, language, account age, polarity, and subjectivity.
- Preprocessing removes spam and excludes retweets without new content before annotation.The passage also describes additional filtering criteria.
- Because abusive content is a minority relative to normal content, the sampling procedure injects boosted examples into randomly sampled tweets.
- Boosted tweets have strong negative polarity (< −0.7) and at least one offensive word.
- D1 contains 300 tweets for exploratory analysis, while D2 contains 80K tweets for final annotation.
Step 2: Exploratory Rounds
The exploratory rounds tuned the label set and crowdsourcing design by examining annotator confusion, label frequencies, agreement, and label relationships. These analyses led to eliminating Cyberbullying while retaining a reduced set centered on Abusive, Offensive, Hateful, Normal, and Spam.
- Exploratory design: The rounds tuned crowdsourcing parameters on a small dataset to reduce cost while identifying label confusion and representative abuse categories.Researchers varied payment, labels, presentation, judgments, worker trustworthiness, and annotation procedures.
- Annotator agreement: Most annotators disagreed when choosing among the labels in the second round, showing that the distinction among categories was not trivial.The first round showed more overwhelming agreement, but many of its judgments were Normal or Spam.
- Label distributions: Offensive and Abusive were most popular, Hateful and Aggressive were less common, and Cyberbullying was rarely selected.Normal accounted for 53% and Spam for 15% overall, although inappropriate-label percentages excluded them.
- Label reduction: Cyberbullying was eliminated because it was rarely selected and individual tweets provide no evidence of the repetition required by its definition.The resulting decision addressed both its low occurrence and the mismatch between the label’s meaning and tweet-level annotation.
- Label relationships: Correlation and similarity analyses linked Abusive, Offensive, and Aggressive, while Hateful co-occurred with other labels without significant correlation.Offensive-Abusive became the most similar pair in the second round, whereas Hateful remained statistically distinct.
- Validation: After merging labels, almost 70% of tweets reached overwhelming agreement with the reduced label set, while Hateful remained frequent enough to retain.Hateful appeared in almost 7% of judgments in the validation rounds.
Step 3: Large Scale Annotation
The final annotation round used the validated setup to label 80k tweets, producing distributions that remained similar to preliminary rounds and showed strong annotator agreement. Boosted sampling substantially increased abusive and hateful content relative to random sampling, demonstrating its importance for studying minority categories.
- Large-scale annotation: 80k tweets received 5 judgments each using the same setup as the validated round.The final task retained the tested annotation configuration.
- Label distributions: 59% of final judgments were Normal, while Spam reached 22.5%, Abusive 11%, and Hateful 7.5%.The higher Spam share reflected the much larger random subset relative to the boosted subset.
- Annotator agreement: 55.9% of tweets achieved overwhelming agreement, with at least 4 of 5 annotators agreeing on the label.Another 36.6% exceeded 3 of 5 votes, while only 7.5% achieved majority with two annotators.
- Sampling comparison: The boosted sample was substantially richer in Abusive and Hateful content than the random sample.The final dataset contained 70k random and 10k boosted tweets, making the overall inappropriate-content percentage relatively low.
- Sampling comparison: In balanced 10k-versus-10k samples, inappropriate tweets comprised almost half the boosted annotations but only about 4% of random annotations.Abusive tweets were approximately 35% in the boosted sample and Normal tweets approximately 35% in the random sample.
Conclusion
The paper develops and releases a methodology and dataset for annotating abusive and hateful language on Twitter. Empirical analysis guides label selection, while the released resources support future annotation and machine-learning research.
- Conclusion: The study annotates a large-scale Twitter dataset covering multiple facets of abusive and hateful language.The selected labels were drawn from several inappropriate-speech categories and examined through multiple annotation rounds.
- Conclusion: Empirical analysis of label relationships, correlations, similarities, and co-occurrences yielded Abusive and Hateful as the representative labels and eliminated Cyberbullying.The authors merged some labels and removed others after statistical analysis.
- Conclusion: The authors release their methodology, custom annotation-platform code, and final annotated dataset.These resources are intended for annotation studies and machine-learning models.