PaperlayerView exact revision on arXiv

Source-linked AI summary

X-Guard: Multilingual Guard Agent for Content Moderation

Bibek Upadhayay, Vahid Behzadan, Ph. D

arXiv:2504.08848v1cs.CRcs.AI

TL;DR

Existing guardrails lack transparency and remain vulnerable to low-resource-language and code-switching attacks, while multilingual safety data are scarce. X-Guard combines multilingual routing and translation with a reasoning safety evaluator, and reports effective classification across languages and on a Sandwich-Attack test. The authors also note that translation limitations constrain recognition of native slang and culturally specific harmful content.

  • Problem

    Existing safety systems can lack transparency and struggle with low-resource-language and code-switching attacks, while multilingual guardrail development is constrained by scarce open-source data.

    Source: §1 · p. 2

  • Method

    X-Guard combines language detection and routing, translation, and a safety evaluator trained with supervised finetuning and GRPO, using jury-judged multilingual data.

    Source: §1 · pp. 2–3

  • Results

    X-Guard achieved 83% accuracy versus Llama-Guard-8B’s 62% on the reported Sandwich-Attack test and was reported as competitive across languages.

    Source: §3.3 · p. 5, §4 · p. 5

  • Takeaways & Limitations

    The reported results support X-Guard as a transparent moderation agent for multilingual contexts, including low-resource-language and code-switching attacks.

    Source: §3.3 · p. 5, §6 · p. 8

    Google Cloud Translation biases and translationese carry over to the agent and limit its capacity to understand native slang and identify culturally specific harmful content.

    Source: §4 · p. 6

Abstract

from arXiv · show

Large Language Models (LLMs) have rapidly become integral to numerous applications in critical domains where reliability is paramount. Despite significant advances in safety frameworks and guardrails, current protective measures exhibit crucial vulnerabilities, particularly in multilingual contexts. Existing safety systems remain susceptible to adversarial attacks in low-resource languages and through code-switching techniques, primarily due to their English-centric design. Furthermore, the development of effective multilingual guardrails is constrained by the scarcity of diverse cross-lingual training data. Even recent solutions like Llama Guard-3, while offering multilingual support, lack transparency in their decision-making processes. We address these challenges by introducing X-Guard agent, a transparent multilingual safety agent designed to provide content moderation across diverse linguistic contexts. X-Guard effectively defends against both conventional low-resource language attacks and sophisticated code-switching attacks. Our approach includes: curating and enhancing multiple open-source safety datasets with explicit evaluation rationales; employing a jury of judges methodology to mitigate individual judge LLM provider biases; creating a comprehensive multilingual safety dataset spanning 132 languages with 5 million data points; and developing a two-stage architecture combining a custom-finetuned mBART-50 translation module with an evaluation X-Guard 3B model trained through supervised finetuning and GRPO training. Our empirical evaluations demonstrate X-Guard's effectiveness in detecting unsafe content across multiple languages while maintaining transparency throughout the safety evaluation process. Our work represents a significant advancement in creating robust, transparent, and linguistically inclusive safety systems for LLMs and its integrated systems.

1 Introduction

Existing guardrails can identify harmful content but often obscure their safety decisions, while multilingual systems face low-resource and code-switching attacks and a shortage of open-source data. X-Guard combines transparent multilingual moderation with judge-diverse data curation, a 132-language dataset, and translation and safety-evaluation models.

1 Introduction

  • Current guardrails can moderate harmful content but provide little transparency, while multilingual systems struggle with low-resource-language and code-switching attacks amid limited open-source data.Existing reasoning guards may also inherit bias from a single judge LLM.Source: §1 · p. 2
  • X-Guard combines language routing, translation, and safety evaluation to moderate content across multilingual contexts, including code-switching attacks.The agent is designed to provide safety assessments across diverse linguistic contexts.Source: §1 · pp. 2–3
  • A jury of judges supplies safety labels and categories through voting, providing a more diverse evaluation than relying on a single evaluator.The jury approach is intended to mitigate biases from individual LLM providers.Source: §1 · pp. 2–3
  • The authors created and publicly released a multilingual safety dataset with 5 million data points spanning 132 languages.They also custom-finetuned mBART-50 and released an X-Guard 3B model trained with supervised finetuning and GRPO.Source: §1 · p. 3

2 Data Collection and Synthetic Data Generation.

The authors assembled safety data from existing datasets and generated harmful responses, then used multiple judges to add assessments and rationales. They translated about 39,000 responses into 132 languages, producing 5 million translation data points.

2 Data Collection and Synthetic Data Generation.

  • The authors generated harmful responses from Salad-Bench and ALERT prompts with an uncensored LLM and combined data from four other safety datasets to create 100K+ examples.Examples include prompts, responses, and prompt-response pairs.Source: §2 · p. 3
  • Multiple open-source LLMs and moderation systems assessed sampled data, providing reasoning, safety labels, and violated categories.The jury marked text unsafe when at least two judges agreed, selected the most common violation category, and used an assessment aligned with the jury label.Source: §2 · p. 3
  • The authors translated about 39K ALERT and Salad-Bench responses into 132 languages, yielding 5 million translation data points from other languages to English.Source: §2 · p. 3

3 X-Guard Agent

X-Guard routes multilingual text through language detection, translation when needed, and safety evaluation that returns an assessment and label. Its evaluation reports safety-classification results across 132 languages, comparisons with other guards, and a Sandwich-Attack test.

3 X-Guard Agent

  • The agent detects language and Sandwich-Attack style, routes English directly to safety evaluation, and translates other languages before evaluation.Code-switched text is translated one language at a time; the evaluator returns an assessment, safety label, and, when applicable, category code.Source: §3 · p. 3

3.1.1 Training Translation Model

  • mBART50-many-to-one was finetuned with one million translation data points to extend its English translation capacity from 49 pretrained languages to 132.The authors selected it for its pretrained many-to-English capability.Source: §3.1.1 · p. 4

3.1.2 Training X-Guard

  • The safety evaluator was supervised-finetuned on 100K examples and then trained with GRPO on 76K examples to predict labels and categories.Training outputs include a reasoning assessment, a safety label, and category codes when applicable; reward functions address output format, safety labels, and categories.Source: §3.1.2 · p. 4

3.2 X-Guard Agent Evaluation

  • Translation metrics averaged across languages improved after finetuning for both languages previously supported by mBART-50 and newly supported languages.The reported metrics were BLEU, chrF, and translation edit rate.Source: §3.2 · p. 4
  • 70.38% accuracy and 70.44% F1-score were achieved by X-Guard 3B on 65K samples spanning 132 languages; English-only accuracy was 97.20%.Category identification weighted F1-scores were 52.37% across the multilingual dataset and 68.53% on English-only text.Source: §3.2 · p. 4
  • The 3.7B agent outperformed the compared models in English and remained highly competitive across other languages against Llama-Guard-3 8B.Comparisons with Llama-Guard covered only its supported languages; comparisons with GuardReasoner and ThinkGuard were English-only, and category classification was excluded because taxonomies differed.Source: §3.2 · p. 5

3.3 Defense against Sandwich-Attack

  • 83% accuracy for X-Guard versus 62% for Llama-Guard-8B was reported on the 100-example multilingual Sandwich-Attack test.The test used Nepali, Persian, Maithili, Arabic, and Vietnamese, with prompts balanced between safe and unsafe content.Source: §3.3 · p. 5

4 Discussions

X-Guard’s 3.7B-parameter agent is reported to outperform the larger LG-8B model in English while remaining competitive across other languages, and to address code-switching attacks. The discussion also highlights training behavior and limitations tied to synthetic judgments and translation.

4 Discussions

  • The 3.7B-parameter agent achieves higher English accuracy than LG-8B and remains competitive across other languages.Table 3 compares the agent with Llama-Guard-3-1B/8B across languages.Source: §4 · p. 5, Table 3 · p. 6
  • Table 4 reports safety-label and category-classification metrics for English versus the other 132 languages.The discussion says a single multilingual classifier is feasible, but limited high-quality reasoning data and the curse of multilinguality would be bottlenecks.Source: Table 4 · p. 6, §4 · p. 7
  • X-Guard is described as handling input-output filtering in more than 130 languages and defending against code-switching attacks such as Sandwich Attack.Table 5 compares the agent with LG-8B on Sandwich Attack data made from Nepali, Persian, Maithili, Arabic, and Vietnamese questions.Source: §4 · p. 6, Table 5 · p. 7
  • The discussion identifies synthetic jury evaluations as a limitation: judge-model biases may remain, and labels may differ from human annotations.Different judge LLMs assigned varying safety labels to the same response.Source: §4 · p. 6
  • Google Cloud Translation may introduce biases and translationese, limiting understanding of native slang and culturally specific harmful content.These translation limitations are described as carrying over to the agent.Source: §4 · p. 6
  • Performance depends directly on translation; the translator expanded from 50 pretrained languages to 132 with limited fine-tuning data, potentially invoking the curse of multilinguality.The authors propose specialized language-specific translation modules as future work.Source: §4 · p. 6
  • Pre-SFT helped the model learn formatting earlier, while safety labels and categories continued to be learned during GRPO training.Figure 2 plots three reward values over 500 training steps (one epoch).Source: Figure 2 · p. 6, §4 · p. 7

5 Related work

Related work spans safety classifiers, reasoning-based evaluators, parameter-efficient fine-tuning, collaborative guardrails, and lightweight approaches. The surveyed work also includes benchmark and training datasets for safety evaluation.

5 Related work

  • Llama-Guard evolved from an input-output, multi-label safety filter into later versions, with Llama-Guard-3 also adding vision capabilities.ShieldGemma and ShieldGemma-2 are also noted as image-capable models; proprietary training details have not been fully disclosed.Source: §5 · p. 7
  • Parameter-efficient safety methods apply PEFT, LoRA, QLoRA, or CPO to fine-tune smaller models for moderation and reasoning tasks.The surveyed tasks include harmful-prompt and response detection, refusal measurement, and question-answer safety assessment.Source: §5 · pp. 7–8
  • Reasoning-based approaches pair safety labels with explicit rationales, critiques, or stepwise safety-token predictions.Wen et al. (2025) use labels and critiques; Liu et al. (2025) reason before assigning categories; Zhu et al. (2025) use [SAFE], [UNSAFE], and [RE-THINK] pivot tokens.Source: §5 · pp. 7–8
  • Collaborative guardrails include multi-agent safety judgments, runtime guardrail code, and systems combining category-specific models with logical rules.The surveyed methods also include KNN-based and energy-based components.Source: §5 · p. 8
  • Lightweight approaches range from adding refusal examples to prompts to distilling Llama-Guard-3-8B into a 435M-parameter DeBERTa model.The smaller model uses augmented jailbreak examples to improve harmful-instruction detection.Source: §5 · p. 8
  • The literature includes safety datasets and benchmarks ranging from 30K jury-annotated samples to 92K labeled adversarial examples.SALAD-Bench evaluates safety capabilities, attack vulnerabilities, and defenses.Source: §5 · p. 8

6 Conclusion and Future Works

X-Guard combines multilingual safety data, judge-based evaluation, and translation with a 3B evaluator to address harmful content, including code-switching attacks. The conclusion identifies limitations and future directions while calling for more inclusive and robust AI safety research.

6 Conclusion and Future Works

  • X-Guard combines enhanced safety data, a jury of judges, a 5-million-point dataset spanning 132 languages, and a translation-plus-evaluation architecture.The architecture uses a custom-finetuned translation module and an X-Guard 3B evaluator.Source: §6 · p. 8
  • The authors identify the cascade method as a limitation and propose larger models, single-model multilingual classification, and further language and reasoning improvements.Future work also includes expanding language coverage and improving performance in ultra-low-resource languages.Source: §6 · p. 8
  • The authors aim for their work to contribute to research on more inclusive and robust AI safety systems.The conclusion says the datasets, code, and models are being publicly released to support further research.Source: §6 · pp. 8–9

A.1 mBART-50 covered Langauges

The mBART-50 language list includes languages from Arabic and Czech to Vietnamese and Chinese, alongside many others.

A.1 mBART-50 covered Langauges

  • The listed languages include Arabic, Czech, German, English, Spanish, Estonian, Finnish, French, Gujarati, Hindi, Italian, Japanese, Kazakh, and Korean.This is an excerpt from the list of covered languages.Source: §A.1 · p. 12
  • The list also includes Lithuanian, Latvian, Burmese, Nepali, Dutch, Romanian, Russian, Sinhala, Turkish, Vietnamese, and Chinese.These entries appear in the supplied language list.Source: §A.1 · p. 12
  • Other listed languages include Afrikaans, Bengali, Persian, Hebrew, Indonesian, Georgian, Khmer, Malayalam, Mongolian, Marathi, and Pashto.The list further includes Portuguese, Swedish, Swahili, Tamil, Telugu, Thai, Tagalog, Ukrainian, Urdu, Xhosa, Galician, and Slovene.Source: §A.1 · p. 12

A.2 Limitations

The authors describe limitations in X-Guard’s proof-of-concept maturity, translation-dependent cascade, multilingual data and performance, judge bias, reasoning evaluation, compute, model scale, and cultural-language coverage.

A.2 Limitations

  • The multilingual content-moderation agent is a proof of concept and requires further training and calibration for production use.This is the stated boundary on its readiness for deployment.Source: §A.2 · p. 12
  • Because safety evaluation depends on translating harmful context, translation quality affects moderation and adds another potential point of failure.The authors contrast this with having the safety evaluator as the only failure point.Source: §A.2 · p. 12
  • Expanding the translation model from 50 to 132 languages may have degraded performance across linguistic domains, and limited training data may affect safety accuracy.The authors describe the multilinguality effect as a possibility, not a demonstrated certainty.Source: §A.2 · p. 12
  • The synthetic dataset may retain judge-specific biases; both judges used were Meta models and reflect Meta safety-policy biases.The authors recommend drawing judges from varied providers for a more balanced jury.Source: §A.2 · p. 12
  • The reward function checks safety labels and categories but does not evaluate the quality of the model’s generated reasoning.It also constrains the number of sentences inside the think tags.Source: §A.2 · p. 12
  • The approach requires substantial computational resources, potentially limiting deployment in resource-constrained environments.Source: §A.2 · p. 12
  • The authors could not train a single decoder-based multilingual classifier; their 3B-model translation experiment performed poorly across 100-plus languages.They tested only a 3B model because of computational limitations.Source: §A.2 · p. 12
  • Google Cloud Translation may introduce linguistic artifacts and biases, while nuanced cultural contexts, native slang, and idioms remain challenging.The authors say these issues limit comprehensive understanding of harmful content across native linguistic contexts.Source: §A.2 · p. 12

A.3.1 mBART50-many-to-en

The mBART-50-many-to-en translation model was trained with multi-GPU DeepSpeed Zero-1 optimization and a dataset of over 5.2 million examples.

A.3.1 mBART50-many-to-en

  • Translation-model training used DeepSpeed Zero-1 across 8 GPU devices through Hugging Face Accelerate, with per-device batches of 48 and 16-step gradient accumulation.Source: §A.3.1 · p. 12
  • The training configuration used a learning rate of 3e-5 and weight decay of 0.01.Source: §A.3.1 · p. 13
  • The dataset contained 5,240,720 text examples, including 3,078,561 training examples and 162,159 validation examples.Source: §A.3.1 · p. 13

A.3.2 X-Guard

This section describes X-Guard’s training and moderation materials, and illustrates its multilingual translate-then-classify approach to unsafe-content evaluation. It also identifies the evaluation metrics and comparison scopes and shows examples involving imperfect translations and safe versus unsafe sexual content.

A.3.2 X-Guard

  • The Qwen-2.5-3B model receives supervised fine-tuning and one-epoch GRPO training with three reward functions implemented using vLLM.SFT uses a curated dataset; the reported GRPO configuration uses per-device batch size 5, gradient accumulation 4, and learning rate 5e-7.Source: §A.3.2 · p. 13

A.4 Results

  • Tables 6–9 list X-Guard safety-label and category classification metrics for all languages except English and for English-only evaluation.The supplied table captions do not include metric values.Source: Table 6 · p. 14, Table 7 · p. 14, Table 8 · p. 14
  • Tables 10–13 list English-only classification metrics for Llama-Guard-3-8B, ThinkGuard-8B, and GuardReasoner-3B.The supplied captions do not show metric values or establish a comparison winner; Table 11 notes that Llama-Guard categories are mapped to the paper’s definitions.Source: Table 10 · p. 15, Table 11 · p. 15, Table 12 · p. 15

A.5.1 Unsafe Content Categories

  • The unsafe-content taxonomy covers privacy violations, violence, self-harm, criminal planning, hate, controlled substances, and weapons, distinguishing harmful assistance from some informational discussion.For example, it disallows help planning violence or crimes while allowing some historical or general discussion; it also protects non-public identifying information and references privacy laws.Source: §A.5.1 · pp. 15–17
  • The category guidance also addresses profanity, sexual content, harassment, child safety, and threats, while allowing specified educational or contextual discussion.Examples include allowing sexual education and school sex education while prohibiting sexual exploitation of minors, and permitting contextual discussion of discrimination or harassment.Source: §A.5.1 · pp. 18–19

A.6 Examples

  • The examples label an ambiguous “bomb that beeyach” phrase and a historical bomb-making query unsafe under O7, while a concert-bombing request is labeled unsafe under O4.The first example’s rationale notes ambiguity but flags the reference to explosives as potentially harmful.Source: §A.6 · pp. 20–21
  • English examples label requests about cocaine use and soliciting prostitution unsafe, but classify a question about sex with one’s wife as safe.The unsafe examples receive categories O4 and O11, respectively; the spouse question receives no category.Source: §A.6 · pp. 22–23
  • Nepali and Persian requests are classified as unsafe, while imperfect Maithili translation still captures intent to damage a building.The examples assign the Nepali request category O4 and the Persian request category O2: Violence.Source: Figure 4 · p. 24, Figure 5 · p. 25, Figure 6 · p. 26
  • X-Guard classifies a Malayalam threat and a Vietnamese Sandwich-Attack prompt as unsafe under O2: Violence.The Vietnamese prompt embeds the murder question in its middle sentence and passes through language detection, routing, translation, and safety evaluation.Source: Figure 7 · p. 26, Figure 8 · p. 27
Loading 2504.08848v1…