Source-linked AI summary
SemEval-2023 Task 10: Explainable Detection of Online Sexism
Hannah Rose Kirk, Wenjie Yin, Bertie Vidgen, Paul Röttger
TL;DR
Online sexism detection needs finer-grained explanations than binary or generic toxicity labels provide. The paper introduces EDOS with a hierarchical taxonomy, a 20,000-comment dataset, and benchmark analyses; participant performance was stronger on binary detection than on finer-grained tasks.
Problem
Existing automated sexism classifiers commonly provide only high-level labels, limiting explanations and making moderation mistakes and model weaknesses harder to assess.
Method
The paper organizes EDOS around a hierarchical taxonomy, a diverse fine-grained dataset, expert annotation, baseline models, and analysis of participant systems and errors.
Results
Participant scores were lower and more variable for the four-category and 11-vector tasks than for binary sexism detection.
Takeaways & Limitations
Fine-grained categories and vectors provide a basis for more explainable sexism detection systems.
Takeaways & Limitations
The taxonomy and labels are grounded mainly in Western-centric, English-speaking viewpoints, and the dataset is limited to English.
Abstract
from arXiv · showhide
Online sexism is a widespread and harmful phenomenon. Automated tools can assist the detection of sexism at scale. Binary detection, however, disregards the diversity of sexist content, and fails to provide clear explanations for why something is sexist. To address this issue, we introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS). We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; and iii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task.
1 Introduction
The paper introduces EDOS to address the limitations of opaque, high-level sexism classifiers by combining hierarchical labels with explainable detection. The task contributes a taxonomy, diverse annotated data, and analyses of participating systems.
- EDOS targets the lack of explanations in automated sexism classifiers, which can undermine user trust, moderation efficiency, and model development.
- The task applies a hierarchical taxonomy with binary sexism detection, four sexism categories, and 11 fine-grained sexism vectors.
- The dataset uses diverse content from Reddit and Gab, high-quality annotation by trained women annotators, and a granular taxonomy designed to aid explainability.
- The authors release a 20,000-comment fine-grained dataset, larger unlabelled datasets for adaptation, baseline models, and analyses of task methods, results, and errors.
2 Taxonomy Design
The taxonomy was developed from prior research and empirical data to represent sexism hierarchically and with fine-grained vectors. It defines sexism broadly while distinguishing categories without relying on inferred speaker intent or recipient impact.
- The taxonomy began with a literature review and was refined through grounded-theory analysis of empirical dataset entries.
- Task A: Binary Sexism: Task A defines sexist content as implicit or explicit abuse directed at women because of gender or gender combined with other identity attributes.
- Task A: Binary Sexism: The taxonomy distinguishes sexism from misogyny by including subtle implicit abuse and prejudice in addition to expressions of hate.
- Task B: Category of Sexism: Task B divides sexist content into four conceptually and analytically distinct categories rather than categories based on recipient effects or speaker motivation.
- The taxonomy excludes abuse primarily targeting groups other than women, while annotators flag such cases with an “other target” label.
- Task C: Fine-Grained Vectors: Task C defines 11 fine-grained vectors intended to be mutually exclusive and collectively exhaustive.
3 Dataset Construction
The dataset combines Reddit and Gab data with ensemble sampling, cleaning, prescriptive annotation guidelines, and expert quality control. This design aims to increase diversity and coverage of fine-grained sexism vectors.
- 3.1 Choice of Platform: The authors use Reddit and Gab to address limited community-based diversity in sexism datasets drawn largely from Twitter.
- 3.2 Data Collection: Equal amounts of data were drawn from Reddit and Gab, with 1M-entry pools and 10k labelled entries sampled from each platform.
- 3.2 Data Collection: Cleaning replaced URLs and usernames, removed empty, URL-only, emoji-only, non-English, and duplicate entries before sampling.
- 3.2 Data Collection: The six sampling techniques produced a total dataset of n = 20,000 while targeting coverage of 11 sexism vectors and lexical and topical diversity.
- 3.3 Annotation: Annotators followed comprehensive prescriptive guidelines containing definitions, clarifications, exemplars, edge cases, and general guidance.
- 3.3 Annotation: Nineteen trained annotators who self-identified as women passed a 200-entry screening task covering all 11 vectors, and experts adjudicated disagreements.
- 3.3 Annotation: The data was split into training, development, and test sets in a 70:10:20 ratio.
4 Task Description
EDOS consists of three hierarchical subtasks that move from binary detection to four-way category classification and 11-way vector classification. Systems are evaluated with macro-average F1 and compared against seven baselines.
- Task A is binary classification of sexist versus non-sexist content, while Tasks B and C are multi-class classification tasks.
- Task B predicts four sexism categories, and Task C predicts 11 fine-grained sexism vectors.
- The competition had separate training-development and test phases, with test entries for Task A released separately from Tasks B and C.
- Systems are evaluated using macro-average F1 to account for class imbalance.
- Seven baselines range from majority and uniform prediction through TF-IDF with XGBoost to DistilBERT and DeBERTa-v3 models.
5 Participant Systems and Results
Participant systems predominantly used transformer architectures and task-specific fine-tuning, while stronger submissions often combined multiple models, continued pre-training, or multi-task learning. Performance was more consistent on binary sexism detection than on finer-grained tasks, and additional features were not associated with higher average F1.
- Participant Overview: 599 accounts signed up, with 134 Task A, 87 Task B, and 81 Task C test-phase submissions.The final leaderboard included 84 Task A, 69 Task B, and 63 Task C submissions.
- Leaderboard Results: The top three systems across tasks used multiple models or ensembles, and many applied continued pre-training and multi-task learning.stce and PASSTeam achieved top results on at least two tasks using both strategies.
- Performance by Task: Task B and C scores were lower and more variable than Task A scores, although most participant systems beat the simplest baseline and many beat the continued-pretraining DeBERTa-v3 baseline.Figure 2 describes Task A scores as more tightly distributed than those for Tasks B and C.
- Popular Methods: ∼90% of participants used transformer architectures, while ∼8% used traditional machine learning systems.Common transformer choices included RoBERTa, DeBERTa, BERT, BERTweet, and DistilBERT.
- Popular Methods: More than 90% fine-tuned only on the target task, ∼30% applied further pre-training, and around 40% used starter-kit unlabelled data.About 15% fine-tuned on an auxiliary task and ∼5% used prompt-based learning.
- Performance by Method: Systems without additional features had higher average F1 scores in all tasks, while training type showed the least influence on score distributions.Architecture choices produced the most distinct performance distributions, with transformer and other deep language models outperforming traditional machine learning on average.
6 Error Analyses
The error analyses show that fine-grained sexism detection remains difficult, with systematic confusion among related categories and vectors. Manual inspection also identifies missed implicit, ironic, stereotypical, or slang expressions, alongside annotation ambiguity caused by limited context.
- Binary Label Confusion: 162 of 4,000 Task A test instances were misclassified by all top 10 systems; 61.7% were false negatives and 38.3% were false positives.Among false negatives, Animosity accounted for 41%, Derogation 34%, Prejudiced Discussion 16%, and Threats 9%.
- Category and Vector Confusion: Animosity and Derogation were often confused, while Prejudiced Discussion was commonly misclassified as either category.Threats was least often confused with other categories.
- Category and Vector Confusion: Vector errors reflected both class imbalance and semantic similarity, including confusions between gendered stereotypes and between shared gendered slurs with different usage conditions.The passage distinguishes vectors 2.1 from 3.2 by explicit negative sentiment and 3.1 from 2.2 by causal versus targeted usage.
- Manual Error Inspection: Manual inspection found 76 genuine system misclassifications and 24 annotation errors among 100 instances misclassified by all top 10 teams.The subset was intentionally drawn from especially challenging examples, so its annotation-error rate may exceed that of the full dataset.
- System Errors: Among 76 system errors, 68% were false negatives involving positive-seeming compliments, jokes, slang, rare insults, irony, or rhetorical questions.False positives commonly involved counterspeech, matter-of-fact discussion, or criticism of an individual woman.
- Annotation Errors: Of 24 annotation errors, 17 were false negatives and 7 were false positives, often reflecting confusion between individual-related and generalised statements.False negatives included stereotypes expressed as speculation, while false positives included non-gendered attacks on individual women.
- Grey Areas with Lacking Context: Nine of the 24 annotation errors were considered grey areas because isolated posts lacked context for interpreting meaning, intent, or whether criticism expressed a general sexist view.Annotators sometimes supplied context from personal experience or knowledge of relevant communities.
7 Discussion
The EDOS analysis finds that diverse sexist content shifts errors toward false negatives and that fine-grained prediction remains substantially harder than binary detection. Continued pretraining on unlabelled in-domain data appears effective, while class imbalance and culturally specific annotation constrain interpretation.
- Value of Diverse Data Sampling: More false negatives than false positives arise in EDOS, which the authors attribute to subtle and diverse sexist forms lacking shared keywords.Prejudiced Discussion is given as an example of content that is harder to identify correctly.
- Challenge of Fine-Grained Predictions: Task C reaches only just above 50% maximum F1, whereas binary sexism detection is substantially easier than fine-grained category and vector prediction.The authors argue that binary scores can obscure failures to distinguish forms of sexism with different impacts.
- Efficiency of Continued Pretraining: The best baselines and top participant systems effectively use continued pretraining on the provided unlabelled Reddit and Gab data.The authors encourage future shared tasks to release unlabelled datasets alongside smaller labelled samples.
- Class Imbalance: Substantial imbalance across sexism categories and vectors makes it difficult to determine whether their confusion reflects data properties or class imbalance.The dataset is more balanced at the binary level, with 24% sexism, than many prior hate and abuse datasets.
- Class Imbalance: The dataset was intentionally not rebalanced because different sexism types have different real-world base rates, making imbalance part of the target problem.The authors recommend examining effects on performance and cross-vector confusion when balancing the data.
- Defining “Explainable”: EDOS defines explainability through hierarchical classification of isolated text, while acknowledging other approaches such as span classification.The hierarchy indicates whether content is sexist and identifies its category and vector.
- Value Specificity and Perspective: The taxonomy and labels primarily reflect Western-centric, English-speaking viewpoints and may therefore transmit the values of its creators and annotators.The annotators were UK-residing people who self-identified as women, and the dataset is limited to English.
8 Conclusion
The paper contributes a hierarchical taxonomy, a diverse and finely annotated dataset paired with larger unlabelled data, and analyses of baseline and participant systems. Its analysis supports combining transformer models with continued pretraining and multi-task learning, while showing that some fine-grained sexism cases remain difficult.
- Contributions: The paper introduces a three-level taxonomy covering binary sexism, sexism categories, and fine-grained sexism vectors.The taxonomy was grounded in prior taxonomies and social science literature, then empirically validated with in-domain data from two social media platforms.
- Contributions: The authors create a diverse, high-quality labelled dataset annotated by self-identifying women experts and pair it with a larger unlabelled dataset.The unlabelled data is intended to reduce financial and psychological constraints associated with annotation.
- Contributions: The shared-task analysis demonstrates the success of combining state-of-the-art transformer models with continued pretraining and multi-task learning.
- Contributions: State-of-the-art models still make errors on specific entries and fine-grained sexism vectors, leaving unresolved prediction challenges.
Ethical Risks and Harm Statement
The dataset is designed to support fine-grained sexism detection while acknowledging risks to data subjects, readers, and annotators. Its scope is limited by English-language, platform-specific data and the isolated interpretation of short comments.
- Risks: The dataset contains online sexism examples that may harm targeted women or readers by reproducing stereotypes, prejudiced discussions, or slurs.The paper includes a content warning and modifies some slurs by replacing vowels with asterisks.
- Dataset Purpose: The dataset is intended to train and evaluate fine-grained, explainable sexism detection across explicit and subtle social-media comments.
- Scope: All entries are in English, limiting the dataset’s language coverage and motivating expansion of its sexism vectors to other languages.
- Scope: All entries come from Gab and Reddit, so the speaker demographics approximate those platforms’ users and intentionally include likely male, Western, right-leaning, and far-right users.
- Annotation: The annotation process used 19 screened annotators over two months, with compensation and access to the research team during labelling.
- Annotation: Annotators had varied nationalities, identities, language backgrounds, and social-media use, and all had experience seeing others targeted by online abuse.
- Annotation: The study collected annotator-specific attitudes toward moderation and online-sexism priorities for future analysis of whether attitudes influenced labels.
- Scope: Comments were short-form written social-media texts labelled in isolation, without thread context, user information, or network information.
B Review of Existing Taxonomies
Existing sexism taxonomies differ in construction, scope, naming, and structure. EDOS combines theory and empirical refinement, uses descriptive names at comparable granularity, and organizes sexism into categories containing fine-grained vectors.
- Differences in Construction: Reviewed taxonomies are either theoretically or empirically grounded, while EDOS first draws on theory-based taxonomies and then iterates with in-domain data.
- Differences in Construction: Prior work commonly samples Twitter or Reddit and often relies on keywords, whereas other studies use manosphere communities, lived-experience catalogues, re-annotation, or adversarial data.
- Differences in Scope: Existing taxonomies differ in whether they address sexism or misogyny, how broadly those terms are defined, and which fine-grained vectors they include.
- Differences in Scope: EDOS uses consistently descriptive vector names with roughly equivalent granularity within each taxonomy level.
- Differences in Structure: Most prior taxonomies place vectors directly under sexism or misogyny, whereas EDOS hierarchically separates broad sexism categories from their fine-grained vectors.
- Data Sources: The dataset’s source review identified 81 relevant subreddits and assigned them to four categories based on prior work.
- Data Sources: The reviewed categories include Incels, Men Going Their Own Way, Men’s Rights Activists, and Pick Up Artists, each described by characteristic views or practices.
D Bespoke Sexism Classifier for Data Sampling
The authors trained a bespoke binary classifier to identify sexist content for sampling. It used English and women-targeted entries from seven open-source datasets after standardized preprocessing.
- The sampling ensemble included a bespoke binary classifier trained on English and women-targeted entries from seven open-source datasets.The source datasets covered sexism and/or misogyny under their respective taxonomies.
- Each source dataset was deduplicated and cleaned before training.Preprocessing included whitespace cleanup and conversion of URLs, emoji, and usernames into special tokens such as [URL].
E Crowdworkers vs. Experts Experiments
The authors compared crowdsourced annotation with annotation by trained self-identifying women using a 200-entry gold standard. Crowd majority voting performed well on binary sexism detection but poorly on finer-grained categorization.
- The annotation comparison used a gold standard of 200 entries and tested two alternatives.The alternatives were crowdsourced annotation and annotation by recruited, trained self-identifying women.
- Crowd majority voting labelled nearly all sexist entries correctly for Task A, at 96%.For non-sexist entries, only 4% were labelled correctly.
- Crowd majority voting was correct for 28% of sexist entries in Task B and 14% in Task C.Task B distinguishes four sexism categories, while Task C identifies fine-grained sexism vectors.
- Figure 4 presents a confusion matrix for Task C vectors across the top 10 performing systems.The figure supports examining how systems confuse fine-grained sexism vectors.