Source-linked AI summary
Generating Counter Narratives against Online Hate Speech: Data and Strategies
Serra Sinem Tekiroglu, Yi-Ling Chung, Marco Guerini
TL;DR
Online hate interventions must address hate without the censorship and overblocking concerns associated with deletion, but manual counter-narrative writing does not scale and automated generation lacks sufficient quality data. The paper tests a hybrid author-reviewer collection framework using GPT-2-generated silver data and filtering before expert validation. The authors report that the framework scales to datasets suitable in diversity, novelty, and quantity.
Problem
Method
Results
The authors report that their framework scales to datasets suitable in diversity, novelty, and quantity.
Source: §1 · p. 2Takeaways & Limitations
The framework offers a scalable way to collect counter-narrative datasets while retaining NGO validation and possible post-editing.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Recently research has started focusing on avoiding undesired effects that come with content moderation, such as censorship and overblocking, when dealing with hatred online. The core idea is to directly intervene in the discussion with textual responses that are meant to counter the hate content and prevent it from further spreading. Accordingly, automation strategies, such as natural language generation, are beginning to be investigated. Still, they suffer from the lack of sufficient amount of quality data and tend to produce generic/repetitive responses. Being aware of the aforementioned limitations, we present a study on how to collect responses to hate effectively, employing large scale unsupervised language models such as GPT-2 for the generation of silver data, and the best annotation strategies/neural architectures that can be used for data filtering before expert validation/post-editing.
1 Introduction
Online hate is difficult to distinguish from dangerous speech, while deletion and suspension invite censorship and overblocking concerns; counter-narratives offer a direct, non-aggressive response. Because manual intervention is not scalable and existing collection strategies trade off quantity and quality, the study tests hybrid collection and an author-reviewer framework.
1 Introduction
- Counter-narratives are non-aggressive responses using fact-bound arguments, and NGOs train operators to write them without escalating discussion.The introduction presents them as an alternative to reactive identification and deletion.Source: §1 · p. 1
- The study tests hybrid collection strategies and an author-reviewer framework to obtain high-quality counter-narratives while reducing expert effort.The framework combines niche-sourcing, crowdsourcing, and GPT-2-generated synthetic data; a reviewer filters candidates before NGO validation and possible post-editing.Source: §1 · pp. 1–2
- The authors report that the framework scales to datasets suitable in diversity, novelty, and quantity.Filtered candidates undergo NGO validation and may be post-edited.Source: §1 · p. 2
2 Related Work
- Prior research includes hate datasets and detection methods, alongside studies reporting counter-narratives' effectiveness in hate countering.Work on detection commonly uses lexical, sentiment, or multimodal features.Source: §2 · p. 2
3 CN Collection Approaches
The paper contrasts crawling, crowdsourcing, and nichesourcing, then assesses their quantity, conformity, argument diversity, and lexical diversity. The approaches have different strengths: nichesourcing yields richer, more diverse material, while crowdsourcing offers scale and crawling remains prone to hostile responses.
3 CN Collection Approaches
- Crawling collects naturally occurring hate-speech/counter-narrative pairs, crowdsourcing elicits responses from non-experts in controlled settings, and nichesourcing uses trained NGO operators.These are the three prototypical collection strategies discussed.Source: §3 · p. 2
4 Characteristics of the Datasets
- The paper evaluates dataset quality through conformity to NGO guidelines and lexical and semantic diversity, alongside quantity and non-ephemerality.The table caption identifies CN-type percentages and repetition rate (RR) as the semantic- and lexical-diversity measures, respectively.Source: §4 · p. 2, Table 1 · p. 3
- NICHE has the greatest lexical diversity (lower RR), followed by CRAWL and CROWD.The comparison removes repeated counter-narratives and shuffles examples before calculating RR.Source: §4 · p. 3
- NICHE produces the most diverse material but lacks sufficient volume and is challenging to implement; CROWD provides scale but uses simpler, more stereotyped arguments.The study reports approximately 41K CROWD counter-narratives from more than 900 workers, versus approximately 4K NICHE pairs from about 100 expert operators before augmentation.Source: §4 · pp. 2–3
5 CN Generation through Author-Reviewer Architecture
The proposed author-reviewer architecture mixes crowdsourcing and nichesourcing: an author generates candidate counter-narratives, reviewers filter them, and NGO operators validate and may post-edit the results. Unlike earlier uses of the architecture for generation, this paper uses it for data collection and adds a human reviewer and final post-editing.
5 CN Generation through Author-Reviewer Architecture
- The architecture assigns generation to an author and quality filtering to a reviewer, then sends selected candidates to NGO operators for validation and possible post-editing.It is designed to combine non-expert pre-filtering with expert review while reducing collection effort.Source: §5 · p. 3, Figure 1 · p. 4
- Unlike earlier studies using author-reviewer architecture for NLG, this work applies it to data collection and adds a human reviewer and final post-editing.The framework combines crowdsourcing and nichesourcing.Source: §5 · pp. 3–4
- The authors tested four author configurations, then three reviewer configurations while holding the best author configuration constant.Figure 1 depicts the author producing candidates and reviewers filtering them before NGO validation and possible post-editing.Source: §5 · p. 4, Figure 1 · p. 4
6 The Author: Generation Approaches
To address training-data bottlenecks, the authors fine-tuned GPT-2 on NICHE and CROWD for counter-narrative generation. They conditioned generation on hate speech and used the resulting models to produce new HS-CN pairs.
6 The Author: Generation Approaches
- GPT-2 was proposed to address limited NICHE training data and CROWD repetitiveness, and was fine-tuned on both datasets for counter-narrative generation.The paper describes GPT-2 as a large-scale unsupervised language model capable of generating coherent text.Source: §6 · p. 4
- NICHE training used 5,366 HS-CN pairs and held out 1,288 pairs; CROWD pairs were split 80/20 into 26,320 training and 6,337 test pairs.For CROWD, the authors extracted dialogues with exactly one hate-speech-labeled utterance so the counter-narrative corresponded to that utterance.Source: §6 · p. 4
- At test time, each hate-speech input conditioned generation, using nucleus sampling with p=0.9 to diversify outputs while truncating the less reliable probability tail.The authors also trained transformer baselines and used nucleus sampling for their decoding.Source: §6 · p. 5
3. TRFniche: baseline on NICHE dataset
GPT-2 models produced more diverse and novel responses than the baseline models, and GPTniche had stronger diversity and novelty scores than GPTcrowd. The authors identify GPTniche as the most promising author configuration and use its over-generation to create new HS-CN pairs.
3. TRFniche: baseline on NICHE dataset
- GPT-2 surpassed baselines in RR diversity for NICHE (4.89 vs 3.23) and CROWD (8.93 vs 5.89), and in novelty (0.04 vs 0.46; 0.10 vs 0.70).The passage describes these as baseline-versus-GPT-2 comparisons and reports the novelty pairs in the same order.Source: §3 · p. 5
- GPTniche's RR and novelty scores were more than double GPTcrowd's.The passage does not give the individual score values for this comparison.Source: §3 · p. 6
- GPTniche was the most promising author configuration, and its over-generated coherent HS-CN pairs could be used without supplying a hate-speech input.The authors say this could avoid the ephemerality problem for hate-speech collection.Source: §3 · p. 6
- Conditioning the author module on NICHE test-set hate speech and removing each input's generated response yielded 2,700 new HS-CN pairs for reviewer experiments.Source: §3 · p. 6
7 The Reviewer
The reviewer stage filters generated HS-CN pairs for suitability before NGO-operator validation. Non-expert ratings yielded few high-quality pairs, while ALBERT outperformed BERT across reported metrics and was selected as the machine reviewer.
7 The Reviewer
- The reviewer stage treats suitability as sentence-level confidence estimation and supports expert post-editing by filtering candidates before NGO validation.The paper tested direct expert review, non-expert filtering followed by operator validation, and classifier filtering followed by operator validation.Source: §7 · p. 6
7.1 Human Reviewer Experiment
- Non-experts rated HS-CN suitability on a 0–3 scale, where 0 meant unsuitable, 1 allowed small modifications, 2 meant suitable, and 3 meant an extremely good counter-narrative.Two annotator scores were gathered per pair, and annotators were asked to discard pairs with malformed hate speech.Source: §7.1 · p. 6
- Only 10% of generated pairs met the high-quality Reviewer≥2 threshold, about one third met Reviewer≥1, and the vast majority were discarded.Source: §7.1 · p. 6
7.2 Machine Reviewer Experiment
- The machine reviewer used BERT- and ALBERT-based classifiers trained to assess whether an HS-CN pair was proper.The training set was balanced with 1,373 positive and 1,373 negative examples.Source: §7.2 · pp. 6–7
- ALBERT outperformed BERT in F1, precision, and recall, with a 6% absolute F1-score improvement, so the authors selected ALBERT as the machine reviewer.Source: Table 6 · p. 7, §7.2 · p. 7
8 NGO Operators Experiments
The operator experiment evaluates how generation and filtering affect expert data-collection time, diversity, novelty, and editing effort. Automatic filtering is a viable alternative to human filtering, with trade-offs as filtering becomes more stringent.
8 NGO Operators Experiments
- Five NGO operators evaluated randomized conditions using 20 example pairs per condition and 80 pairs each overall.Pairs were sampled from a pool of 2,700, with stratified sampling and no repeated pairs across subjects.Source: §8 · pp. 7–8
- Automatic filtering saved more operator time than human filtering at score ≥1 and better preserved diversity and novelty than score ≥2, in line with score ≥1.The comparison appears in Table 7.Source: Table 7 · p. 8, §8 · pp. 8–9
- 700 sec (around 12 min) were needed on average for a pair scoring ≥2 from both annotators; only 10% of examples met that condition.More stringent filtering saved more expert time but increased the time required from crowd annotators.Source: §8 · p. 8
- More stringent pre-filtering increased RR and lowered novelty, partly because reviewers more readily agreed on generic responses.Examples included “This is not true” and “How can you say this about an entire faith?”Source: §8 · p. 8
- HTER decreased slightly as pre-filtering became more stringent, but all scores were 0.1–0.2, below the 0.4 acceptability threshold.The authors interpret this as operators modifying only easily amendable CNs.Source: §8 · p. 8
9 Conclusions
The conclusion presents a staged framework for producing counter-narratives while reducing collection effort. It reports promising automatic-classifier results and suggests the framework may also suit other NLG tasks.
9 Conclusions
- The proposed collection pipeline generates silver counter-narratives, filters them with crowd-workers, and then uses expert validation or post-editing.Source: §9 · p. 9
- The study frames this pipeline as addressing limited high-quality data and generic or repetitive outputs in automated counter-narrative generation.Source: §9 · p. 9
- The authors report promising results from replacing crowd filtering with an automatic classifier and suggest possible use in paraphrase generation or text simplification.Source: §9 · p. 9
A Appendices
The appendix captions identify sampled counter-narratives from GPT-2 and transformer models across datasets and reviewer configurations.
A Appendices
- Table 8 presents sampled counter-narratives generated by GPT-2 and TRF models trained on CROWD.Source: Table 8 · p. 12
- Table 9 presents sampled counter-narratives generated by GPT-2 and TRF models trained on NICHE.Source: Table 9 · p. 13
- Table 10 presents sampled GPT-2 counter-narratives filtered under various reviewer configurations.Source: Table 10 · p. 14