Source-linked AI summary
Introducing the Privacy-HSD Trade-off: Hate Speech Detection, but not at the Cost of Privacy
Stephen Meisenbacher, Vlad Garbuz, Chirill Donos, Maxim Dnestreanschii, Gabriel Creanga, Andreea-Elena Bodea, Thomas Lampert, Jana Diesner
TL;DR
Hate speech detection can encode authorship, creating a need to balance detection performance with privacy. The paper formalizes this privacy-HSD trade-off, benchmarks privatization methods, and introduces AGNOSPEECH, whose variants generally preserve HSD performance while achieving higher privacy trade-offs.
Problem
Hate speech detection systems may encode authorship information, motivating privacy-preserving detection that does not degrade privacy protections.
Method
The paper curates author- and hate-labeled forum datasets, probes authorship encoding, formalizes the trade-off, and benchmarks AGNOSPEECH against existing privatization methods.
Results
AGNOSPEECH variants generally maintain HSD classification performance while achieving higher privacy-HSD trade-offs than comparable methods; L2 lowers perplexity and L3 restores coherence.
Takeaways & Limitations
The findings support privacy-preserving HSD and emphasize continued improvement in the design and evaluation of methods balancing privacy with detection.
Takeaways & Limitations
AGNOSPEECH is dataset-dependent, particularly in L2, where saliency measurements derive from a model trained on selected data.
Abstract
from arXiv · showhide
Hate speech is a real and timely threat that affects a large portion of online users, especially youth and minority groups. While building reliable and robust automatic hate speech detection (HSD) systems is paramount, we argue that this must also be balanced with the individual right to privacy. Exploring the intersection of HSD and privacy, we demonstrate that HSD systems might unintentionally achieve performance at the cost of encoding authorship, posing a threat to privacy. Building on these findings, we establish the notion of a privacy-HSD trade-off, which demands a careful balance. We benchmark a series of text privatization methods, as well as our newly proposed domain-specific AgnoSpeech technique, showing that balancing privacy and HSD is difficult but feasible. The findings make a strong case for more research on the trade-offs between privacy and HSD, both of which have tangible implications for the safeguarding of online participation.
1 Introduction
Online hate speech is a pervasive and growing threat, creating an urgent need for automatic hate speech detection that respects privacy. This paper formalizes the privacy-HSD trade-off, shows that HSD can encode authorship, and proposes AGNOSPEECH to preserve detection-relevant cues while removing unnecessary signals.
- Motivation: Nearly 67% of online users have encountered hate speech, including nearly 50% of European Union young adults aged 16 to 29.These figures come from a recent 2025 UN-ESCO report and underscore hate speech’s prevalence online.
- Problem: Automatic hate speech detection should remove hate speech while respecting individuals’ privacy and data-protection requirements.The paper identifies balancing reliable HSD with privacy as intuitive, significant, and complex.
- Contributions: The paper formalizes the privacy-HSD trade-off and demonstrates that unmitigated HSD methods may inadvertently encode authorship and serve as profiling tools.A probing study with two representative datasets examines authorship cues in HSD classification systems.
- Findings: AGNOSPEECH achieves higher privacy-HSD trade-offs than off-the-shelf privatization methods, which cannot reliably balance HSD with privacy, while producing coherent and usable outputs.The findings support domain- and use-case-specific text privatization, particularly for HSD.
- Contributions: AGNOSPEECH preserves hate speech cues for HSD while selectively removing signals unnecessary for detection.The method is designed specifically for text privatization in HSD.
- Evaluation: The study evaluates AGNOSPEECH and comparative methods across three datasets for privacy-preserving HSD.This evaluation directly assesses the privacy-HSD trade-off.
2 Related Work
Related work spans a decade of hate speech detection research and a diverse text-privatization literature, including anonymization, scrubbing, and rewriting. Authorship obfuscation provides a foundation for framing privacy protection as a trade-off with utility, which this work extends to HSD.
- Hate Speech Detection: HSD research has progressed from lexicon-based methods to broader datasets, benchmarks, multilingual systems, and multimodal settings.The paper discusses a representative sample rather than conducting another survey.
- Text Privatization: Text privatization transforms textual data to remove direct or indirect identifiers through anonymization, text scrubbing, and private text rewriting.The precise method depends on the underlying privacy notion; classic anonymization commonly targets PII and named entities by removing or masking them.
- Text Privatization: Evaluating text privatization is complex because privacy in textual data is difficult to define and effectiveness must be assessed against adversaries.Recent work has nevertheless sought to systematize and standardize these evaluations.
- Authorship Obfuscation: Authorship obfuscation removes author-identifying signals to reduce downstream re-identification risk, while authorship attribution and verification can also serve useful purposes.The paper focuses on the negative aspects of authorship attribution and extends prior utility trade-off framing to HSD.
3 Dataset Construction
The study constructs Reddit and Twitter datasets that jointly provide author and hate-speech labels, using constrained sets of frequent authors for privacy and detection experiments. The resulting subsets preserve substantial hate-speech representation while reflecting platform-specific data limitations and release restrictions.
- Dataset Construction: Two base datasets are curated from online forums using published resources filtered to constrained author sets, with both user and hate-speech labels.These datasets support the paper’s experiments and analyses.
- Reddit: The Reddit top-25 subset contains 1154 comments, including 365 (31.6%) marked as hate speech, with author contributions ranging from 32 to 100 comments.A top-50-author subset contains 1795 comments, including 525 (29.2%) hate speech, with at least 21 comments from one author.
- Twitter: The Twitter data simplify racist, sexist, and neither labels into hate and non-hate speech, and 9,523 of 16k tweets were re-hydrated with text and user IDs.Unsuccessful re-hydration attempts imply that the original posts were deleted or moved.
- Twitter: The Twitter top-10-author subset contains 6,792 tweets, including 2039 (30%) tagged as hate speech, with contributions heavily skewed toward the top three authors.The top three contributed 3626, 2040, and 929 tweets, while the 10th contributed 25.
- License and release: The datasets will be publicly released, but the Twitter release omits user IDs and tweet texts because researchers must retrieve them through the X API using tweet IDs.The Reddit datasets will be distributed as created under the original source’s license.
4 Foundations of the PrivHSD Trade-off
The section establishes that hate speech detection models can encode authorship information, creating a privacy risk that motivates privacy-preserving HSD. It then formalizes the privacy-HSD trade-off as a balance between HSD utility losses and privacy gains.
- Baseline HSD models: Three baseline HSD models are trained for Reddit and Twitter using original cleaned datasets, with held-out author-subset samples serving as test sets.The models cover Reddit subsets of 25 and 50 users and Twitter; the datasets contain 22k Reddit and 9.5k Twitter texts.
- Authorship leakage: Linear probing shows that HSD model representations encode author identity, as regressors can predict authors from [CLS] embeddings on held-out texts.The procedure extracts 768-dimensional embeddings, trains multinomial logistic regression, and evaluates author-prediction accuracy; a non-fine-tuned BERT baseline is also tested.
- Authorship leakage: The statistical analyses find considerably high η2 effect sizes across all models, particularly for Twitter, indicating strong author-related variance in HSD predictions.η2 measures the proportion of regression-prediction variance attributable to author identity.
- Adversarial re-identification: A black-box adversary achieves modest but non-negligible improvements over majority-class guessing when re-identifying authors from text, reinforcing that HSD performance and privacy are intertwined.The adversary directly trains an author-classification model and evaluates re-identification using micro-F1 on the test set.
- Trade-off formalization: The privacy-HSD trade-off compares relative HSD utility change with relative privacy change against non-privatized data, using probe accuracy, η2, FPR σ, and adversarial F1.All four privacy metrics are better when minimized, and the resulting PrivHSD score weighs privacy gains against utility losses.
5 A Case Study of Existing Mitigations
The case study benchmarks seven text privatization methods across three curated datasets to assess how well they balance privacy and hate speech detection. Privatized data are used to train HSD models, whose utility, privacy, and coherence are evaluated under the PrivHSD framework.
- Evaluation: For each privatized counterpart of Reddit-25, Reddit-50, and Twitter-10, a new HSD classifier is fine-tuned on privatized training and test data, and test micro-F1 represents Hp.The procedure follows the training setup from Section 4 while replacing the original subsets with privatized versions.
- Methods: Seven methods span entity-based anonymization, differentially private text privatization, and LLM-assisted anonymization, with two configurations used for all but one method.The configurations provide a representative sweep of privatization capabilities.
- Methods: The benchmark includes entity detection and masking tools, word- and document-level DP mechanisms, contextual rewriting, and LLM-based anonymization.Methods include Presidio, GLINER, SANTEXT, DP-MLM, DP-BART, RUPTA, and Privacy Filter.
- Evaluation: Three repeated runs with seeds 41-43 provide average probe accuracies and FPR σ scores, while adversarial micro-F1 supplies the remaining privacy components.The adversarial evaluation uses the retained 20% test splits and seed 42.
- Evaluation: PrivHSD trade-offs are calculated by combining the scores with baseline values, while GPT-2 perplexity separately measures text coherence.The complete results and score breakdowns are reported in Table 2.
6 Towards Privacy-preserving HSD
The paper introduces AgnoSpeech, an author-agnostic text privatization method designed to preserve hate-speech detection while protecting privacy. Its three-tier pipeline removes direct identifiers, retains hate-speech cues, and optionally restores tokens to improve readability.
- Motivation: AgnoSpeech addresses generalized privatization methods’ severe HSD utility loss by optimizing the privacy-HSD trade-off for the HSD context.The method is explicitly designed to preserve utility while protecting privacy rather than applying mechanisms not tailored to HSD.
- Design principle: The method redacts signals unnecessary for hate-speech identification while retaining hate-speech cues and improving readability.This targets the incoherence associated with replacement-free redaction and stricter differential-privacy-based privatization.
- Three-tier pipeline: AgnoSpeech uses three tiers: L0 keeps the input un-privatized, L1 removes direct identifiers, and L2 distills hate-speech signal importance.L1 uses regex detection in the fast variant and adds Microsoft Presidio in the performance variant; L2 uses linear attribution or token saliency.
- Variants: AgnoSpeech provides fast and performance variants, respectively targeting efficiency and throughput versus more computationally expensive modeling.The variants are implemented across the three-tier method.
- Three-tier pipeline: L2’s performance variant removes the bottom 40% of tokens judged unimportant to HSD, using sequential prediction-confidence changes after token removal.The fast variant instead uses linear attribution from a logistic-regression proxy model with unigram and bigram coefficients weighted by corpus TF-IDF.
- Three-tier pipeline: L3 optionally restores pruned tokens randomly according to intensity i ∈[0, 1], trading some author-signal risk for improved readability and coherence.The restoration module is shared by both fast and performance variants.
7 Analysis and Discussion
AGNOSPEECH achieves competitive privacy-HSD trade-offs by preserving HSD signals while protecting privacy, but balancing privacy against coherence, utility, and authorship leakage remains difficult. The discussion identifies persistent limitations, including incomplete mitigation of authorship information and unmeasured usability criteria.
- Results and comparative analysis: AGNOSPEECH L2 achieves competitive trade-offs at lower perplexity, while L3 restores coherence with minimal trade-off degradation.DP-BART and SANTEXT may obtain high privacy scores but suffer extreme coherence and utility loss.
- Results and comparative analysis: AGNOSPEECH maintains HSD classification near Presidio, GLINER, and Privacy Filter while achieving higher PrivHSD trade-offs through domain-specific privacy preservation.This advantage is attributed to preserving the HSD signal while protecting privacy.
- Results and comparative analysis: The performance variant usually sacrifices some HSD for stronger privacy and higher trade-offs, whereas fast remains acceptable in resource-constrained environments.The findings support the more computationally heavy performance variant while retaining fast as a practical option.
- The need for a balance: Extreme privacy optimization can cause up to 20 percentage points of HSD performance loss, showing that acceptable PrivHSD requires balancing privacy with utility.The discussion suggests incorporating perplexity or other quality metrics into utility preservation.
- Negative results and areas for improvement: Methods often mitigate probe accuracy and adversarial F1 but struggle with author-attributable variance, especially on Reddit, leaving authorship and fairness risks.Authorship entanglement is never completely diminished, particularly on Twitter; semantic similarity and grammatical correctness are also not evaluated.
8 Conclusion
The paper establishes the privacy-HSD trade-off as a balance between combating online hate speech and preserving privacy. It benchmarks existing text privatization methods against AGNOSPEECH, demonstrating both the complexity of this balance and the promise of tailored approaches.
- The study establishes the foundations of the privacy-HSD trade-off to address online hate speech while preserving the right to privacy.
- Author-identifiability metrics provide the basis for benchmarking text privatization methods in HSD models.
- Benchmarking existing methods against AGNOSPEECH reveals a complex privacy-HSD balance and demonstrates the merits of tailored methods.
Limitations
The study’s limitations concern constrained author sets, dataset-dependent AGNOSPEECH saliency, reliance on BERT-BASE-CASED, limited privacy-risk coverage, and outdated datasets. The authors call for broader architectures, newer data, improved generalizability, and investigation of additional privacy threats.
- Limitations: Experiments assume constrained author sets, although this models malicious insiders or capable adversaries filtering datasets by frequently writing users.The authors identify this as a plausible real-world use case, including filtering within a subreddit.
- Limitations: AGNOSPEECH depends on dataset-specific L2 saliency measurements, despite remaining effective when trained on Reddit and evaluated on Twitter.Future work should improve its cross-dataset generalizability.
- Limitations: The probing study and evaluations rely solely on BERT-BASE-CASED, motivating tests across diverse model architectures.The authors justify BERT-BASE-CASED as foundational for many subsequent similar models.
- Limitations: The work targets author re-identification but does not consider other privacy risks, including demographic or political-affiliation attribute inference.The authors describe these omitted risks as logical extensions of the work.
- Limitations: The two datasets are established but outdated: Reddit was published in 2021, while Twitter data dates to 2016.Follow-up work should evaluate privacy-preserving HSD methods on newer datasets when available.
Ethical Considerations
The paper frames hate speech detection and privacy as intertwined ethical concerns, since high-performing systems may expose authorship information for profiling or re-identification. AGNOSPEECH begins addressing these concerns together while the study follows dataset licenses and pseudonymizes usernames.
- Privacy risks: High-performing HSD tools may inadvertently entangle authorship information, enabling unintended profiling or re-identification.The authors present AGNOSPEECH as an initial approach to addressing hate speech detection and privacy in tandem.
- Data use: The datasets were used for HSD in accordance with their licenses: CC BY-NC 4.0 for Reddit and CC BY 4.0 for Twitter metadata.
- Harm prevention: Author usernames were converted to integer pseudonymous IDs, while the datasets’ hateful and distressing language warrants caution.