PaperlayerView exact revision on arXiv

Source-linked AI summary

SPAR-Hate: An Auditor-Guided Multi-Agent Framework for Bilingual Hate Speech Parsing

Yifan Lyu, Dianqing Lin, Xinran Li, Jiaqi Qiao, Xiujuan Xu

arXiv:2608.22018v1cs.AI

TL;DR

Structured hate parsing must jointly identify targets, arguments, and labels while addressing cultural, linguistic, and social-position differences in interpretation. SPAR-Hate decomposes documents into clause-level units, generates evidence-grounded judgments from three roles, and arbitrates conflicts before reassembling sample-level outputs. Across Chinese and English benchmarks, it improves parsing, with the clearest gains on stricter joint structural metrics.

  • Problem

    Existing hate-speech work pays less attention to cultural, linguistic, and social-group challenges in structured parsing.

  • Method

    SPAR-Hate segments documents into clause-level units, generates candidates from three perspectives, arbitrates them with an auditor, and reassembles structured outputs.

  • Results

    SPAR-Hate improves bilingual parsing on STATE-ToxiCN and TBO across local and API settings, with clearest gains on stricter joint structural metrics.

  • Takeaways & Limitations

    The framework supports bilingual structured parsing while retaining language-specific schemas and concentrating gains on joint structural evaluation.

  • Takeaways & Limitations

    Evaluation mainly covers Chinese and English benchmarks and lacks a full comparison with strong task-specific fine-tuned systems.

Abstract

from arXiv · show

Hate speech detection has recently shifted from coarse-grained classification to structured parsing, where systems must jointly identify hateful targets, arguments, and target-level labels. However, existing studies primarily emphasize benchmark evaluation while paying less attention to the cultural, linguistic, and social-group challenges involved in structured hate speech parsing. To address these challenges, we propose SPAR-Hate, an auditor-guided multi-agent framework for bilingual hate speech parsing. The framework first decomposes documents into clause-level decision units and then generates evidence-grounded judgments from three complementary perspectives: Victim, Moderator, and Cultural Bystander. An evidence-constrained arbitration process resolves conflicts among role-specific predictions and aggregates them into structured sample-level outputs. Experiments on the STATE-ToxiCN and TBO benchmarks show that SPAR-Hate consistently improves bilingual hate parsing across diverse large language models. The framework achieves state-of-the-art results on bilingual multi-tuple extraction tasks, with the largest gains observed under stricter structural evaluation metrics.

1 Introduction

Structured hate parsing must handle culturally situated interpretation and consistent target–argument–label tuples. SPAR-Hate addresses these challenges with multi-perspective, evidence-grounded arbitration and improves bilingual benchmark performance.

  • Structured hate parsing extends beyond classification to predict target-level tuples linking attacked targets, arguments, and labels.
  • Culturally situated hate speech creates errors in tuple prediction and interpretation, especially for implicit, coded, or locally grounded expressions.
  • SPAR-Hate segments documents into clause-level units and generates evidence-grounded judgments from Victim, Moderator, and Cultural Bystander perspectives.
  • An evidence-constrained auditor resolves role-specific conflicts before aggregating clause-level decisions into sample-level predictions.
  • SPAR-Hate improves bilingual parsing across STATE-ToxiCN and TBO settings, with clearest gains on stricter joint structural metrics.

1. A language-decoupled bilingual hate parsing

The paper defines a language-decoupled bilingual tuple formulation and introduces an auditor-guided framework that preserves language-specific schemas while enforcing structured consistency. Experiments and distillation studies report gains concentrated on stricter joint metrics and partial transfer to a smaller student model.

  • 1. A language-decoupled bilingual hate parsing: The bilingual formulation aligns Chinese and English tasks without forcing their original annotation contracts into one schema.
  • 2. An auditor-guided multi-perspective parsing framework: SPAR-Hate models target, argument, and harmfulness judgements through clause segmentation, multi-role candidates, auditor arbitration, and sample-level reassembly.
  • 3. Main experiments, ablations, and distillation studies: Gains concentrate on stricter joint structural metrics, while distillation transfers part of the multi-agent decision process to a smaller student model.

2 Related Work

Prior hate-speech research moved from classification toward fine-grained, rationale-based, and structured analysis as implicitness and multilingual variation exposed model fragility. SPAR-Hate narrows multi-agent collaboration to reproducible target–argument–label decisions supported by structured pseudo-traces.

  • Hate-speech research progressed from classification toward localisation, rationales, and structured tuple parsing to support finer semantic analysis.
  • Coded, indirect, subtle, and language-specific expressions systematically weaken hate-speech model performance and can trigger false hate signals.
  • Explainable detection methods motivate stable alignment among the target, argument, and label in structured parsing.
  • Multi-agent research uses explicit roles, debate, and workflow collaboration for difficult judgements and structured tasks.
  • SPAR-Hate constrains this paradigm to comparable, arbitrable, reproducible target–argument–label decisions with structured pseudo-traces.

3 Methods: The SPAR-Hate Framework

SPAR-Hate parses bilingual hate speech through clause-level decomposition, perspective-guided candidate generation, evidence-constrained arbitration, and sample-level reassembly. It preserves separate Chinese and English annotation contracts while supporting structured target–argument–label outputs and a Chinese group extension.

  • Task Formulation and Output Schema: SPAR-Hate aligns bilingual main-track tuples as attacked target, attack argument, and target-level harmfulness label.The Chinese four-field extension retains attacked-group information, while the main bilingual tracks use three fields.
  • Task Formulation and Output Schema: The framework preserves separate Chinese and English output contracts, including Chinese three-field and four-field evaluation tracks.The Chinese four-field extension retains group, whereas the aligned Chinese main track removes it.
  • Segment and Perspective-Guided Role Generation: SPAR-Hate segments documents into clause-level local units before generating structured candidates from Victim, Moderator, and Cultural Bystander perspectives.Local units retain indices, clause text, focus-marked context, and a soft target anchor for later comparison.
  • Perspective-Guided Role Generation: Strict JSON outputs require arguments to be supporting substrings of focus_text and require textual evidence even for non-hateful judgements.Chinese Cultural Bystander prompts may receive weak lexicon context, but it cannot override grounded judgement.
  • Dynamic Beacon Auditor Engine: The auditor scores candidates, clusters them by structured fields, triages agreement and defects, applies soft beacons, and deterministically selects and refines outputs.Cluster scores incorporate member quality, supporting-role counts, and any lexicon-aware bonus; bounded regeneration may address defective roles.
  • Sample-Level Multi-Target Reassembly: Phase 4 aggregates final clause-level tuples by sample_id and normally concatenates them while preserving clause-level provenance rather than forcing exact deduplication.This restores internally made clause-level decisions to the sample-level format required by evaluation.

4 Experiments

Experiments evaluate SPAR-Hate on bilingual Chinese and English benchmarks using separate task protocols, baselines, model settings, and structural metrics. Results show consistent gains, especially on stricter joint metrics, while ablations, distillation, and qualitative analysis examine the framework’s components and residual errors.

  • Datasets and setup: Experiments use Chinese STATE-ToxiCN and English TBO, with TBO’s public test set deterministically repartitioned into 3200/800 train/test subsets.STATE-ToxiCN retains its official split; preprocessing otherwise covers cleaning, schema normalization, and split freezing.
  • Evaluation protocol: Evaluation preserves benchmark-specific protocols: Chinese reports Hard and Soft Macro-F1, while English emphasizes exact tuple consistency.Chinese metrics distinguish exact span-and-field agreement from overlapping spans under the same target-centered structure.
  • Main results: Under a fixed local 14B backbone, SPAR raises average scores from 29.83 to 32.82 on ZH-main and from 29.59 to 37.51 on EN-main.The clearest gains occur on stricter joint structural metrics rather than from scale alone.
  • Main results: Across tracks and API settings, improvements concentrate on higher-order joint metrics, including Targeted, NTA, TA, T-A Pair, T-A-H Tri, and Quad.SPAR-DeepSeek-v4-Pro achieves the best reported ZH-main average of 39.52, while SPAR-GPT-5.4 retains strong English TA and NTA performance.
  • Component ablations: Removing the segmenter improves Chinese Target Hard F1 from 48.73 to 52.34 but reduces T-A-H Tri Hard F1 from 8.71 to 7.02, while removing Phase 3 lowers bilingual averages.Removing Phase 3 lowers the ZH average to 31.71 and the EN average to 36.72.
  • Distillation: A 4B distilled student exceeds the 14B 0-shot baseline on all three tracks and remains close to the 14B SPAR teacher on EN-main.It scores 33.67 versus 32.82 on ZH-main, 30.38 versus 28.95 on ZH-quadruple, and 37.33 versus 37.51 on EN-main.

5 Conclusion

SPAR-Hate combines clause-level segmentation, role-conditioned generation, auditor arbitration, and sample-level reassembly for bilingual hate speech parsing. Its largest gains occur on stricter joint structural metrics, with further evidence of language-specific schema adaptation and partial student transfer.

  • SPAR-Hate combines clause-level segmentation, role-conditioned generation, auditor arbitration, and sample-level reassembly for bilingual hate speech parsing.
  • The framework’s gains are largest on stricter joint structural metrics.
  • The Chinese four-field extension suggests adaptation to language-specific schema.
  • Distillation results suggest partial transfer of the teacher’s structured decision process to a smaller student model.

Limitations

SPAR-Hate’s evaluation is concentrated on Chinese and English benchmarks and does not yet establish performance across broader languages, cultural settings, or strong task-specific fine-tuned systems.

  • SPAR-Hate is evaluated mainly on Chinese and English benchmarks.
  • The evaluation does not cover a broader range of languages and cultural settings.
  • The study lacks a full comparison against strong task-specific fine-tuned systems.
  • The results are better read as validation of an interpretable structured parsing framework than as a complete performance ceiling.

Ethical Considerations

The paper addresses ethical risks in structured hate-speech parsing through restrained data exposure and traceable processing, while emphasizing safeguards for real-world deployment.

  • The study uses public Chinese and English benchmark datasets containing abusive, discriminatory, and potentially traumatic expressions.
  • The paper retains only representative examples and recommends minimal exposure during data processing, visualization, and manual analysis.
  • Structured outputs and traceable intermediate processes increase transparency but may also expand the operational reach of moderation systems.
  • Real-world deployment requires human review, appeals, threshold calibration, error auditing, and continuing bias evaluation across languages, groups, and cultural settings.
  • Baseline adaptations normalize outputs, align field names and label spaces, and apply limited interface-required post-processing without candidate arbitration or reranking.
  • The mainline arbitration configuration reuses quality and similarity thresholds across the dynamic-beacon family rather than retuning them separately for each benchmark.

E Additional Qualitative Case Studies

The qualitative case studies illustrate how grounded arbitration resolves conflicting target and harmfulness judgments across Chinese and English examples. They also show how intermediate arbitration information is preserved for structured distillation.

  • The case studies illustrate conflict resolution under grounded arbitration in one Chinese and one English example.
  • Chinese case study: Victim and Cultural Bystander identify “Beijingers” as the attacking target, while Moderator selects the mentioned victim group.
  • Chinese case study: The auditor follows grounded consensus on the focus text and downweights the Moderator reading.
  • English case study: At sample level, Phase 4 retains two harmful tuples from g1, (shit, shit, 1) from g2, and one non-harmful tuple from g3.
  • English case study: For the current focus, the auditor adopts the Victim’s harmful interpretation and downweights the non-harmful readings from Moderator and Cultural Bystander.
  • Distillation schema: Each distillation row preserves decomposition, role candidates, conflict diagnosis, and final arbitration so the student learns a structured decision process.

F.2 Filtering and Dataset Statistics

The distillation pipeline constructs grounded teacher-trace subsets and progressively trains students from structured reasoning toward strict JSON output. The supplied passages also identify the associated training and evaluation tables.

  • Reported artifacts: The supplied tables cover teacher-trace statistics, a condensed distillation row, student-model setup, Chinese four-field results, and complete bilingual ablations.The Chinese extension is reported in Table 16, while bilingual ablations are reported in Table 17.
  • Filtering and dataset construction: Teacher traces are filtered for recoverable identifiers and serialisable packages, then selected into a strict HQ reasoning subset using groundedness, tuple validity, and audit confidence.The subset is expanded into reasoning-plus and JSON-only variants under the same grounding constraints.
  • Student training: Student training follows a three-stage teacher-only curriculum: strict HQ reasoning, reasoning-plus coverage, and strict JSON alignment.Training uses only the train split, excludes ground-truth assistant supervision, and disallows manual prefill.

G Full Bilingual Ablation Results

The bilingual ablations show that the framework’s phases and roles affect structural parsing differently across evaluation dimensions. Removing early segmentation can improve target-only recall while weakening stricter structure, whereas removing arbitration reduces aggregate performance.

  • Phase ablations: Removing Phase 1 often raises target-only recall but weakens stricter structural metrics.This indicates that ablation effects differ between isolated target detection and joint structural parsing.
  • Phase ablations: Removing Phase 3 lowers the overall average on both bilingual tracks.The reported effect appears across both evaluation tracks rather than only one language.
  • Role ablations: Role removal has a milder aggregate effect but still weakens exact structural parsing in the main configurations.The role components therefore contribute more clearly to structural exactness than to aggregate score changes.
  • Chinese four-field extension: In the Chinese four-field extension, retaining group changes are most visible in the higher-order Quad metrics.Among local models the 35B system is strongest overall, while SPAR shows clearer API gains on stricter parsing than target detection.
Loading 2608.22018v1…