Source-linked AI summary
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, Maarten Sap
TL;DR
Multilingual safety research has limited language coverage and safety scope despite multilingual LLM deployment and elevated non-English safety risks. The paper introduces POLYGUARD, trained on large multilingual datasets combining naturally occurring interactions and machine-translated safety data. POLYGUARD outperforms existing open-weight and proprietary safety detectors, while the authors note potential incorrect predictions and harmful-content risks in the released datasets.
Problem
Safety research has focused primarily on English, while multilingual LLMs can generate more harmful content in non-English languages and existing systems cover limited languages or safety dimensions.
Method
The authors release POLYGUARD, PGMix, and PGPrompts, combining naturally occurring multilingual interactions with machine translations and labeling prompt harmfulness, response harmfulness, and response refusal.
Results
POLYGUARD outperforms existing open-source and proprietary safety detectors across English-only and multilingual safety and toxicity benchmarks.
Takeaways & Limitations
The datasets and models provide a starting point for building robust multilingual safety detectors and advancing multilingual safe AI systems.
Takeaways & Limitations
POLYGUARD may occasionally produce incorrect predictions, and the released datasets contain unsafe content that may facilitate harmful-content creation.
Abstract
from arXiv · showhide
Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release POLYGUARD, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the corresponding training and evaluation datasets. POLYGUARD is trained on POLYGUARDMIX, the largest multilingual safety training corpus to date containing 1.91M samples across 17 languages (e.g., Chinese, Czech, English, Hindi). We also introduce POLYGUARDPROMPTS, a high quality multilingual benchmark with 29K samples for the evaluation of safety guardrails. Created by combining naturally occurring multilingual human-LLM interactions and human-verified machine translations of an English-only safety dataset (WildGuardMix; Han et al., 2024), our datasets contain prompt-output pairs with labels of prompt harmfulness, response harmfulness, and response refusal. Through extensive evaluations across multiple safety and toxicity benchmarks, we demonstrate that POLYGUARD outperforms existing state-of-the-art open-weight and commercial safety classifiers by 5.5%. Our contributions advance efforts toward safer multilingual LLMs for all global users.
1 Introduction
Multilingual LLM deployment has outpaced safety research, which remains narrow in language coverage and safety scope. POLYGUARD addresses these gaps with a multilingual, multi-task safety detector and large supporting datasets.
- Multilingual LLMs can produce more hate speech, disinformation, and harmful content when prompted in non-English languages.
- Existing multilingual safety efforts are constrained by limited language coverage, narrow safety categories, weak multilingual performance, and simplistic safe/unsafe classification.
- POLYGUARD is a fine-tuned model for multi-task multilingual safety detection and moderation across five input dimensions.
- PGMix and PGPrompts contain 1.91M and 29K prompt-output pairs, respectively, across 17 languages, with labels for prompt harmfulness, response harmfulness, and response refusal.
- PGMix combines naturally occurring multilingual interactions with machine-translated WildGuardMix data to increase diversity, while POLYGUARD outperforms existing open-source and proprietary safety detectors.
- The datasets and models are presented as a starting point for robust multilingual safety detectors and safer multilingual AI systems.
2 Dataset
The paper builds multilingual safety datasets from translated and naturally occurring interactions, then annotates and validates them with multiple automated judges and human reviewers. The resulting resources span 17 languages and cover varied safety categories.
- PGMix contains 1.91M interactions, while PGPrompts contains 29K translated samples for multilingual safety-classifier training and evaluation.
- The datasets cover 17 languages spanning multiple language families and writing systems.
- The annotation pipeline combines English safety classifiers, LLM judges, existing binary labels, and conflict resolution using Llama3.1-405B-Instruct.
- ITW samples cover unsafe-content types different from those in WildGuardMix, while multilingual samples are annotated with GPT-4o because Llama-Guard-3-8B performs poorly on them.
- Human validation across 16 languages rated translation quality at 81.15 on average and found source-target safety-label agreement of Krippendorff’s α = 0.94.
3 POLYGUARD: A 17-Language Safety Moderation Tool
POLYGUARD is built by fine-tuning multilingual instruction models on PGMix. Its unified text-to-text format jointly assesses prompt harmfulness, response harmfulness, and response refusal.
- POLYGUARD fine-tunes Qwen2.5-7B-Instruct and Ministral-8B-Instruct-2410, with an additional Qwen2.5-0.5B-Instruct model.
- The models use Low-Rank Adapters and a unified text-to-text format for comprehensive safety assessment.
- The assessment jointly labels prompt harmfulness, response harmfulness, and response refusal, including violated categories for unsafe prompts or responses.
4 Results & Research Questions
POLYGUARD is evaluated against open-source and proprietary safety detectors across in-distribution, multilingual, out-of-distribution, and code-switched benchmarks. Results generally favor POLYGUARD, while robustness depends on combining translated and In-The-Wild data and retaining training diversity.
- In-distribution PGPrompts benchmark: POLYGUARD models achieve state-of-the-art performance on POLYGUARDPROMPTS, with Qwen2.5 performing marginally better and PG Smol outperforming DuoGuard.
- Out-of-distribution multilingual benchmarks: POLYGUARD models outperform baselines on most Multilingual Guardrail Test Suite datasets and achieve the best average performance on Patronus AI safety benchmarks.Perspective API and Google Moderation outperform PG on RTP-LX and XSafety, respectively.
- Robustness across data distributions: Training with translated and In-The-Wild samples produces greater robustness across in-domain and out-of-distribution evaluation benchmarks.Models trained only on In-The-Wild data perform well on somewhat in-distribution benchmarks but do not generalize to PGPrompts.
- English, translated, and code-switched data: POLYGUARD models outperform baselines across code-switched evaluation settings, although code-switching causes considerable performance degradation for nearly all models.Aegis 1.0 also shows a performance drop from English to translated data, followed by an increase on code-switched data that remains below English performance.
- Translation quality: Removing low-quality translations does not necessarily improve performance, with contrasting trends for Qwen2.5 and Ministral.The authors hypothesize that low-quality translations may help Qwen2.5 perform on low-quality benchmark text.
- Translation artifacts: Evaluation on manually created Aya RedTeaming samples provides no empirical evidence that POLYGUARD relies only on machine-translation artifacts.
5 POLYGUARD Runtime Comparison
POLYGUARD includes models intended for different deployment settings, including a lightweight model for latency-sensitive use. The 0.5B model offers high throughput, whereas the larger models are slower than similarly sized Llama Guard models; multilingual moderation also outperforms English-only moderation after translation.
- Runtime efficiency: The 0.5B POLYGUARD model supports on-device or latency-critical applications through high throughput.The models were tested on 7,419 Aya RedTeaming samples using an NVIDIA L40S GPU and VLLM.
- Runtime efficiency: The 7B and 8B POLYGUARD models run comparatively slower than similarly sized Llama Guard counterparts.
- Training-data effects: Removing In-The-Wild data generally degrades model performance by reducing training-data diversity.
- Translation-based moderation: PG Qwen2.5 achieves higher recall than an English-only Llama-Guard-3-8B setup after translating Aya RedTeaming inputs into English: 0.916 vs. 0.706 in French, 0.926 vs. 0.669 in Russian, and 0.952 vs. 0.681 in Spanish.
6 Background & Related Work
Prior multilingual safety work is constrained by English-centric data, incomplete safety coverage, limited language coverage, and simplified binary task formulations. These constraints motivate broader multilingual safety datasets and detectors.
- Research gap: Multilingual safety research has focused primarily on English despite LLM deployment across diverse global languages.The paper links this focus to potential exposure of global users to harmful content and privacy violations.
- Research gap: Existing efforts often cover only a subset of safety dimensions or a narrow set of languages, while some multilingual systems struggle on multilingual benchmarks.
- Research gap: Many safety frameworks perform a single classification task with binary safe/unsafe labels, which does not capture the varied forms of harmful content across cultural and linguistic contexts.
7 Conclusion
The paper presents multilingual training and evaluation resources alongside POLYGUARD safety detectors. Across varied benchmarks and data distributions, the detectors perform better or comparably to existing open-weight and proprietary systems.
- Contributions: POLYGUARDMIX contains 1.91M user-LLM interactions across 17 languages, while POLYGUARDPROMPTS contains 29K samples for evaluating safety guardrails.
- Conclusion: POLYGUARD detectors perform better or comparably to existing open-weight and proprietary safety detectors across evaluation benchmarks from different data distributions.
Ethics Statement
POLYGUARD may occasionally produce incorrect predictions, and its released datasets contain unsafe content that could inadvertently facilitate harmful-content creation.
- POLYGUARD may occasionally produce incorrect predictions when used as a moderation tool.
- POLYGUARDMIX and POLYGUARDPROMPTS contain unsafe or harmful content that may inadvertently facilitate harmful-content creation.The authors state that dataset-use restrictions are planned as a safety measure.
A Translation Pipeline and Quality Evaluation
The translation pipeline combines chunking, iterative refinement, and GPT-4o-based quality evaluation, with human validation across 16 languages. TowerInstruct generally outperforms NLLB for supported languages, while Hindi translations are poor when Tower does not support the language.
- Translation pipeline: The training-data pipeline chunks source text, translates chunks independently, and concatenates them to process long texts within translation-model context limits.
- Translation pipeline: The test-data translation framework performs initial translation, error analysis, and refinement to preserve semantic accuracy and cultural nuances.
- Quality evaluation: GPT-4o evaluates translated test data on a three-point scale ranging from entirely correct to bad translation.The judge is prompted to reason about source and target sentences before producing a verdict.
- Quality results: TowerInstruct generates higher-quality translations than NLLB for supported languages, whereas Hindi quality is poor because Tower does not support Hindi.
- Human validation: Human validation uses 50 data points per language, three annotators per point, and Krippendorff’s α for safety-label agreement.The study recruits annotators through Prolific and reports validation results across 16 languages.
F Influence of low-quality translated data
The study trains a translation-quality classifier and finds that filtering low-quality translated data has model-dependent effects rather than consistently improving performance. Qwen2.5 generally degrades after removal, while Ministral improves.
- Classifier training: The translation-quality classifier is trained on 60,346 samples and reaches 82% accuracy on a 25,863-sample validation set.
- Performance influence: Removing low-quality translated data does not necessarily improve performance across models.
- Performance influence: Qwen2.5 performance degrades for most datasets after low-quality samples are removed, whereas Ministral performance improves.The authors hypothesize that noisy translations may bridge gaps between training and evaluation data.
G Limitations
The work is limited by translation and automated-labeling errors, an incomplete safety taxonomy, and lack of coverage for low-resource languages. Removing low-quality data also does not consistently improve performance.
- Limitations: Automatic translation can introduce deviations in toxicity and safety risks through incorrect translations and hallucinations.
- Limitations: Automated safety-category annotations may inherit biases from the classifiers and language models used to produce them.The authors use a panel of models to mitigate, but do not eliminate, this limitation.
- Empirical boundary: Removing low-quality training data does not necessarily improve model performance.
- Limitations: The MLCommons Safety Taxonomy may omit potential harms and differ from categories preferred by other researchers.
- Limitations: POLYGUARD datasets and classifiers do not extend to low-resource languages because suitable high-quality multilingual models are unavailable.