PaperlayerView exact revision on arXiv

Source-linked AI summary

To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

Demetris Paschalides, George Pallis, Marios D. Dikaiakos

arXiv:2610.03324v1cs.CL

TL;DR

Hate-speech moderation must scale while accommodating different definitions, but it is unclear whether structured decision models can do so without task-specific training. HATEDECIDE compares six configurations across four datasets and multiple baselines, testing supplied definitions and decomposed questions. Commercial LLMs significantly outperform all decision models on only one dataset, and neither definitions nor decomposition reliably improves classification.

  • Problem

    Moderation needs efficient classifiers that accommodate differing hate-speech definitions, while structured decision models’ ability to meet these needs without task-specific training remains untested.

    Source: §1 · p. 1

  • Method

    HATEDECIDE evaluates six structured decision-model configurations on four datasets against moderation, zero-shot, commercial, and supervised baselines, varying supplied definitions and direct versus decomposed decisions.

    Source: §1 · p. 1

  • Results

    Commercial LLMs significantly outperformed all decision models on only one dataset; definition provision changed up to 28% of predictions, and decomposition significantly improved classification in 20% of comparisons.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Selected structured decision models offer classification quality and efficiency without task-specific training, but models must be chosen and validated under the policy and data at hand.

    Source: §5 · p. 8, §5.1 · p. 9

    Component answers remain uncertain without corresponding human annotations, and agreement with dataset labels does not establish adherence to a supplied policy.

    Source: §5.1 · p. 9

Abstract

from arXiv · show

The scale of online content makes hate-speech moderation challenging, while Large Language Models (LLMs) enable harmful material to be produced and adapted more easily. Moderation therefore requires efficient classifiers that can accommodate different definitions of hate speech. Recent structured decision models accept natural-language criteria and select among specified answers, raising the question of whether they can meet these requirements without task-specific training. We present HATEDECIDE, an evaluation of six decision-model configurations on four hate-speech datasets against specialized moderation, zero-shot, commercial, and supervised baselines. We examine whether supplying a dataset's definition, or decomposing it into multiple questions, improves classification, and we measure their latency and cost. We find that commercial LLMs significantly outperform all decision models on only one dataset. Supplying definitions changes up to 28\% of predictions without consistently improving classification, and decomposition significantly improves performance in only 20\% of the comparisons. On a diagnostic set of test cases, the best hosted decision model comes within 1.6 macro-F1 points of the best commercial LLM at approximately 97\% lower inference cost. These results identify opportunities for inexpensive moderation, while showing that explicit criteria and additional questions do not reliably improve classification.

1 Introduction

Hate-speech moderation must operate at scale while accommodating differing definitions, but supplied criteria do not consistently improve agreement with dataset annotations. HATEDECIDE evaluates structured decision models against multiple baselines and tests definition provision and question decomposition.

1 Introduction

  • Different legal, platform, and research definitions cover different identities and expressions, so a detector trained under one definition may apply unsuitable criteria in another context.Agreement with dataset annotations does not establish adherence to a supplied definition.Source: §1 · p. 1
  • HATEDECIDE evaluates six structured decision-model configurations on four datasets against moderation, zero-shot, commercial, and supervised baselines, varying supplied definitions and direct versus decomposed decisions.The study measures classification quality, latency, and cost without task-specific training.Source: §1 · p. 1
  • Commercial LLMs significantly outperformed all decision models on only one of four datasets, while the best hosted decision model came within 1.6 macro-F1 points of the best commercial LLM at 97% lower inference cost.Supervised classifiers lost up to 54 points when tested on datasets other than their training data.Source: §1 · p. 2
  • Supplying definitions changed up to 28% of predictions, but significantly improved agreement in only 29% of model-dataset comparisons and reduced it in as many.Definition changes did not consistently improve agreement with dataset annotations.Source: §1 · p. 2
  • Definition-specific decomposition significantly improved classification in 20% of model-dataset comparisons and reduced it in approximately 37%.The best hosted decision model scored within 1.6 macro-F1 points of the best commercial LLM at 97% lower inference cost.Source: §1 · p. 2

2 Related Work

Prior work links hate-speech classification to differences in definitions and annotations, instruction-based moderation, and decomposition into smaller judgments. HATEDECIDE examines how supplied criteria and decision formulation affect classification quality and processing requirements.

Definitions and Cross-Dataset Evaluation

  • Cross-dataset performance differences combine changes in language, sampling, and annotation practices, so they cannot be attributed to definitions alone.HATEDECIDE complements transfer evaluation with comparisons that vary supplied definitions on identical examples, distinguishing annotation agreement from policy adherence.Source: §2 · p. 2

Instruction-Based Moderation Models

  • Natural-language criteria can accommodate different moderation requirements without retraining, but instruction sensitivity does not guarantee reliable adaptation.Prior findings report variation in LLM hate-speech annotations with geographical cues and persona attributes, and more specific definitions do not consistently improve classification.Source: §2 · p. 2
  • Structured decision models restrict outputs to specified answers, and prior comparative evaluation across annotation tasks examined quality, calibration, and cost [10].HATEDECIDE extends this line to hate-speech moderation under varying definitions and decision formulations.Source: §2 · p. 2

Decomposing Hate-Speech Decisions

  • Hypothesis engineering combines natural-language-inference predictions about text aspects into a verdict and has improved zero-shot detection across datasets.Checklist-based xList-Hate uses ten conceptual questions and a decision tree, examining cross-dataset generalization and computational overhead; HATEDECIDE instead uses dataset-derived questions and fixed rules.Source: §2 · p. 2

3 Methodology

HATEDECIDE evaluates structured decision models without task-specific training, varying their criteria, decision formulation, and comparison baselines. The methodology uses four English datasets, defined model configurations, paired comparisons, and macro-F1 with adjusted uncertainty intervals.

3 Methodology

  • Each decision model receives a text, question, and answer options, optionally with a hate-speech definition, and returns a scored answer; the study varies model, definition, and direct versus decomposed formulation.The evaluation compares six configurations with policy-conditioned moderation, zero-shot, commercial LLM, and supervised baselines.Source: §3 · p. 3

3.1 Datasets and Classification Tasks

  • The four English datasets differ in collection, definitions, and available annotations; they support binary classification, while HateXplain also separates offensive language and Dynamic and HateCheck include finer moderation annotations.Texts were deduplicated and linked into groups to prevent supervised training on copies or perturbations of test texts; Dynamic required the largest removals, including 14.7% of training and 8.0% of validation texts.Source: §3.1 · p. 3

3.2 Models

  • The six decision-model configurations comprise Jev, Laya, Decider, two SemIf/Qwen configurations, and Bespoke-Nimble-9B, which differ in how they score answer options.Jev is a hosted API; the other configurations run locally, with Nimble using 4-bit NF4 quantization.Source: §3.2 · p. 4

3.3 Definitions and Decision Formulations

  • The dataset-specific condition supplies documented criteria and label descriptions, so its contrast with the no-definition condition estimates the effect of that instruction package, not the definition alone.For decomposition, binary definitions are split into criterion questions and combined by fixed dataset-specific rules; HateCheck requires both component answers to concern the same target.Source: §3.3 · p. 4

3.4 Evaluation

  • Paired evaluations use identical test examples, and decomposition is compared with direct classification under the same dataset-specific definition.Macro-F1 weights hate and non-hate classes equally; uncertainty is estimated by resampling linked groups 10K times, with Bonferroni-adjusted intervals classifying gains or losses as significant only when they exclude zero.Source: §3.4 · p. 5
  • Efficiency comparisons time complete decisions on identical HateCheck texts, reporting median latency and hosted cost while assigning no monetary cost to local inference.Timing uses one text and one outstanding hosted request at a time, excluding model loading and warm-up.Source: §3.4 · p. 5

4 Results

No model leads on every dataset, and supplying definitions or decomposing decisions does not reliably improve classification. On HateCheck, Jev approaches commercial-LLM quality at substantially lower cost.

4.1 Classification Quality

  • DistilBERT transfer is generally poor: MHS-trained scores fall to .273 on HateCheck, while Dynamic-to-HateCheck is a favorable exception at .845.Source: Figure 2 · p. 6, §4.1 · p. 6
  • Sol leads the out-of-the-box models in three-class macro-F1 at .604, below TF-IDF+LR (.639) and DistilBERT (.629); offensive-class F1 is .316 for Laya and .448 for Sol.Source: §4.1 · p. 6
  • Three of four datasets are led by commercial LLMs, but Decider leads MHS; no model ranks first everywhere.Source: §4.1 · p. 6

4.2 Definition Sensitivity

  • Of 28 definition comparisons, 8 significantly gain and 8 lose macro-F1; definitions alter up to 28% of predictions without reliably improving agreement.Source: Figure 3 · p. 6, §4.2 · pp. 6–7

4.3 Attributes and Decomposition

  • Referenced-group macro-F1 exceeds .92 for every decision model, while none exceeds .70 on direction or .54 on conduct; definitions roughly double Jev’s scores on the harder tasks.Source: Table 5 · p. 7, §4.3 · p. 7
  • Decomposition significantly improves macro-F1 in 6 of 30 comparisons and lowers it in 11; most significant gains reflect fewer false positives, with no supported commercial-LLM gains.Source: Figure 4 · p. 7, §4.3 · pp. 7–8

4.4 Efficiency

  • On HateCheck, Jev is 1.6 macro-F1 points below Sol at approximately 97% lower recorded cost.Source: §4.4 · p. 8
  • Jev’s decomposition cost rises 64% on HateCheck and 139% on Dynamic despite little latency change; additional judgments can therefore raise hosted charges without improving classification.Source: §4.4 · p. 8

5 Discussion and Conclusion

Structured decision models can support hate-speech moderation without task-specific training, but performance depends on the model and dataset. Supplied definitions and decomposition do not reliably improve agreement or classification.

5 Discussion and Conclusion

  • No model led every dataset; commercial LLMs outperformed all decision models only on Dynamic, while supervised classifiers lost up to 54 points outside their training data.The authors advise validating a strong out-of-the-box decision model on a sample annotated under the platform’s own policy and considering false-positive and false-negative costs.Source: §5 · p. 8
  • Supplied definitions changed up to 28% of predictions, significantly improved annotation agreement in only 29% of comparisons, and reduced it in as many.Even CoPE, which was trained to apply supplied policies, lost agreement on three of the four datasets.Source: §5 · p. 8
  • Every decision model exceeded .92 macro-F1 for group identification on HateCheck, but none exceeded .70 for direction or .54 for Dynamic conduct type.Label definitions helped most on the harder direction and conduct tasks, roughly doubling Jev’s scores; the authors recommend validating these attributes before using them for decisions such as threat escalation.Source: §5 · p. 8
  • Decomposition improved 6 of 30 comparisons and reduced 11 of 30, often blocking false positives but sometimes blocking texts labeled hateful by the dataset.No improvement was statistically supported for either commercial LLM; a single miscalibrated question can change many verdicts.Source: §5 · p. 8
  • Decomposition raised median latency by at most 9% for Laya, Jev, and both commercial LLMs, but roughly doubled it for SemIf, Bespoke-Nimble, and CoPE.It raised Jev’s recorded cost by 64% to 139%, depending on question count; the overhead is mainly justified when component answers are needed for review.Source: §5 · pp. 8–9
  • On HateCheck, Jev scored 1.6 macro-F1 points below Sol at 97% lower cost; structured models’ performance depended on the model and dataset.The comparison plots macro-F1 against median complete-decision latency and hosted cost on identical HateCheck examples.Source: §5 · pp. 8–9, Figure 5 · p. 9

5.1 Conclusion, Limitations, and Future Work

  • Validation must distinguish agreement with dataset annotations from adherence to a supplied policy; component-answer correctness remains uncertain without corresponding human annotations.A prediction may follow the supplied definition yet conflict with the dataset label.Source: §5.1 · p. 9
  • The evaluation uses English texts, and HateCheck’s deliberately written examples need confirmation on everyday moderation traffic; prior training exposure is uncertain.The authors also caution that false positives can suppress legitimate speech while false negatives leave targeted communities exposed, and suggest studying referrals of uncertain cases to stronger models.Source: §5.1 · p. 9

A Definitions

The section presents dataset-specific hate-speech criteria, operationalizes them as fixed decomposition questions and rules, and documents how prompts and inference settings implement the evaluation conditions.

A Definitions

  • The dataset definitions are source-grounded paraphrases whose criteria vary in target scope and what qualifies as abuse or hate.They include distinctions such as protected-group targeting, group-directed hatred, and contextual interpretation of offensive language.Source: A Definitions · p. 11

B Decomposition Questions and Rules

  • Decomposition questions and rules were derived from dataset documentation and frozen before evaluation; Dynamic also gates conduct by scope and inferred intent.MHS, HateXplain, and HateCheck classify hate only when both targeting and abuse are present.Source: B Decomposition Questions and Rules · p. 12

C Prompts and Inference Settings

  • In matched direct-versus-decomposed comparisons, both receive the same policy text and differ only in the requested decision; commercial LLMs return all component answers together.Direct classification requests a verdict, whereas local decomposed classification asks each component question separately.Source: C Prompts and Inference Settings · p. 13
  • Commercial LLM requests timed out after 45 seconds; benchmark quality runs used four concurrent requests, while timed and matched runs used one.Their temperature was unset, reasoning was disabled, and routing was restricted to OpenAI without fallbacks.Source: C Prompts and Inference Settings · p. 14

D Prediction-Level Definition Effects

Dataset-specific definitions changed a notable share of decision-model predictions, but those changes did not necessarily translate into substantial macro-F1 movement.

D Prediction-Level Definition Effects

E Generic Definition

The generic definition was not included in the main analysis because its agreement with dataset annotations could not establish whether models applied it correctly.

E Generic Definition

  • The study’s generic definition was written for early experiments and supplied across datasets, but no dataset was annotated under it.Agreement with annotations therefore could not establish whether models applied that definition correctly.Source: E Generic Definition · p. 15
  • The generic definition’s binary macro-F1 effects were mixed in sign, ranging from an 8.3-point loss to a 23.6-point gain.The reported effects are descriptive and were not tested for significance.Source: E Generic Definition · p. 15
  • On HateXplain’s three-class task, generic-definition and Davidson-proxy macro-F1 differed by at most 2.1 points for every decision model.Source: E Generic Definition · p. 15

F Three-Class Results

On HateXplain’s three-class task, all four compared models scored lowest on the offensive class, with out-of-the-box models often confusing offensive and hateful posts.

F Three-Class Results

  • All four models scored lowest on HateXplain’s offensive class.The table reports per-class F1 for the best decision model, best commercial LLM, and two supervised classifiers.Source: F Three-Class Results · p. 15
  • 46% of 548 offensive posts were labeled hate by Laya, while 32% were labeled normal.Source: F Three-Class Results · p. 15
  • Sol labeled 42% of 548 offensive posts as hate and 14% as normal; the principal confusion was offensive versus hateful language.Laya and Sol received the Davidson proxy, while supervised classifiers received no definition and were trained on 5,000 posts.Source: F Three-Class Results · p. 15

G Threshold-Tuned Direct Classification

The section tests whether decomposition improves classification beyond merely shifting a direct classifier’s hate threshold. Against threshold-tuned direct classification, decomposition significantly wins in only one model–dataset pair, while threshold tuning wins in three.

G Threshold-Tuned Direct Classification

  • The control tests whether a calibrated hate-score threshold can reproduce decomposition’s gain using one question, rather than improved judgments.For each model and dataset, the threshold maximizing macro-F1 on 200 labeled validation texts was frozen before testing; thus A* is not out-of-the-box.Source: G Threshold-Tuned Direct Classification · pp. 15–16
  • Decomposition significantly beats threshold-tuned direct classification only for Laya on MHS (+3.6 points); threshold tuning wins in three pairs, and five differences are inconclusive.For Jev on Dynamic, both approaches reach .845; on Dynamic, threshold tuning raises SemIf/Qwen3.5-4B macro-F1 from .674 to .718.Source: G Threshold-Tuned Direct Classification · p. 16, Table 11 · p. 16
  • Table 11 compares direct (A), decomposed (B), and threshold-tuned direct (A*) macro-F1, with B−A* reported in percentage points.Its dagger marks adjusted intervals excluding zero across 18 comparisons; the table caption notes 800 texts per dataset.Source: Table 11 · p. 16

H Efficiency Measurements

Efficiency measurements report 95th-percentile latency and observed throughput for complete decisions on HateCheck. The reported throughput is tied to the measured processing setup, not peak capacity.

H Efficiency Measurements

  • Table 9 reports 95th-percentile latency and observed throughput for direct and decomposed classification across all 3,728 HateCheck texts.Complete wall time includes per-text execution and decomposition’s rule computation, but excludes model loading and warm-up.Source: H Efficiency Measurements · p. 16
  • Measurements used batch size one and hosted concurrency one per model; observed throughput uses complete elapsed processing time, not reciprocal median latency.Accordingly, the recorded rates do not represent peak throughput.Source: H Efficiency Measurements · p. 16
  • BART’s timing reflects its definition-conditioned template, and local inference has no assigned monetary cost.Table 9’s dashes indicate unassigned local cost, not zero cost.Source: H Efficiency Measurements · p. 16

I Selective Routing

The exploratory analysis routes low-margin texts from Laya to a commercial model. Referring 20% to Sol improves macro-F1 on all three reported datasets, whereas routing the same share to Luna lowers HateXplain performance.

I Selective Routing

  • Referring 20% of texts to Sol raises macro-F1 from .658 to .729 on Dynamic, .696 to .721 on MHS, and .730 to .763 on HateXplain.The analysis keeps Laya’s direct answer except for a fixed fraction of texts with the smallest Laya probability margins, which receive the commercial model’s direct answer.Source: I Selective Routing · p. 16
  • Referring the same 20% of texts to Luna lowers HateXplain macro-F1 from .730 to .714.Referral fractions were not selected on development data, and end-to-end cascade latency was not measured.Source: I Selective Routing · p. 16
Loading 2610.03324v1…