Qwen3Guard Technical Report

1 Introduction

• Existing guard models face conflicting safety policies and evaluate only complete responses, preventing timely intervention during streaming generation.The report identifies the need for guards adaptable to varying real-world safety standards and interactive use.Source: §1 · p. 2
• Qwen3Guard introduces safe, controversial, and unsafe labels to accommodate safety judgments that depend on context or policy.The three-tier classification is intended to support differing moderation requirements across deployment scenarios.Source: §1 · p. 2
• Generative Qwen3Guard frames safety classification as instruction following, while Stream Qwen3Guard adds a token-level head for real-time monitoring during generation.Both variants are offered in 0.6B, 4B, and 8B sizes for different resource constraints.Source: §1 · p. 2
• Generative Qwen3Guard reportedly outperforms existing state-of-the-art models on unsafe prompt and response detection across diverse languages.The report also says it substantially improves model safety as an RLAIF feedback signal while preserving helpfulness; Stream Qwen3Guard supports on-the-fly intervention without model retraining, with modest performance degradation versus Generative Qwen3Guard.Source: §1 · p. 2
• Qwen3Guard supports 119 languages and dialects for global and cross-lingual safety moderation.The evaluation spans English, Chinese, and multilingual datasets.Source: §1 · p. 2

Safety Policy

• The policy defines unsafe or cautionary dialogue to guide guard-model alignment and ensure consistent training-data annotation and evaluation.It also provides criteria for interpreting the Guard’s evaluation results.Source: §2 · p. 3
• The policy detects harmful or eliciting inputs and harmful outputs, aims to cover recognized societal and ethical concerns, and allows severity tiers to reflect application-specific risk tolerance.The tiers are Safe, Controversial, and Unsafe.Source: §2 · p. 3

Safety Categories

• Covered categories include violent content, non-violent illegal acts, sexual content or acts, personally identifying information, suicide and self-harm, and unethical acts.Examples include weapon instructions, hacking, unauthorized disclosure of sensitive personal information, hate speech, harassment, and misinformation regarding ethics.Source: §2 · p. 3
• The policy also covers politically sensitive falsehoods posing public-deception or social-harm risks, and unauthorized uses of copyrighted works.The specified political category concerns demonstrably untrue claims about government actions, historical events, or public figures.Source: §2 · p. 3
• Jailbreak is an input-only category for attempts to override system prompts or model conditioning; output classification instead depends on the generated response’s potential harm.Source: §2 · p. 3

Severity Levels

• Unsafe content is generally harmful across most scenarios, Controversial content may depend on context or application disagreement, and Safe content is generally safe across most scenarios.Source: §2 · p. 3

3 Generative Qwen3Guard

• Supervised fine-tuning of instruction-tuned Qwen3 foundation models enables Generative Qwen3Guard to perform content moderation generatively.The foundation models are cited as Yang et al. (2025a).Source: §3 · p. 4

3.1 Task Definition

• Safety classification is framed as instruction following: given moderation instructions and target content, the model generates an assessment in a predefined format.Inputs include safety policies, categories, dialogue context, and formatting requirements; response assessments also indicate whether the assistant refused.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Data Collection

• The curated training dataset contains over 1.19M positive and negative samples from human annotation and synthetic generation.Table 1 presents the language distribution as percentages of total data volume.Source: §3.2 · p. 4, Table 1 · p. 4
• Prompt synthesis uses a fine-grained policy taxonomy, category-specific seed prompts, keyword-guided generation, and paired safe/unsafe examples to broaden coverage and reduce reliance on superficial cues.The paired examples share similar surface structures, helping prevent labels from being associated with irrelevant wording.Source: §3.2 · p. 5
• Response data combines human-authored and model-generated content, including unsafe outputs synthesized with base models and reasoning outputs from open-source models.The reasoning sources include QwQ, the Qwen3 series (Yang et al., 2025a), DeepSeek-R1, and distilled DeepSeek variants.Source: §3.2 · p. 5
• Voting over preliminary labels from multiple Qwen models yields safety-level labels with an F1 score exceeding 0.9 on a manually annotated validation set.Category and refusal labels are assigned using the most frequently predicted output across models.Source: §3.2 · p. 5
• To address multilingual data scarcity, the authors translated original content into 15 additional languages and checked quality with language-mixing detection, an LLM judge, and random manual review.The translations used Qwen-MT.Source: §3.2 · p. 5

Training

• Generative Qwen3Guard is trained with vanilla supervised fine-tuning on instruction-tuned Qwen3 models.Source: §3.3 · p. 5
• Training data has few controversial examples because the category is ambiguous, and annotation noise may confuse learning and generalization.Source: §3.3 · p. 5

Building Controversial Labels

• Rebalancing Safe/Unsafe samples shifts model tolerance: doubling Safe examples makes the model more permissive, motivating ratios intended to approximate the controversial decision boundary.Source: §3.3 · pp. 5–6
• The pipeline trains contrasting models on one dataset partition, applies them to the other, and labels conflicting predictions Controversial through majority voting.The process is reversed across the two partitions to produce controversial labels throughout the training dataset.Source: §3.3 · p. 6, Figure 3 · p. 6

Label Distillation

• After label construction, Qwen3-32B serves as the distillation teacher, with models trained on one subset refining annotations in the other to reduce errors.Source: §3.3 · p. 6

3.4.1 Main Results

• Qwen3Guard-Gen leads 8 of 14 public English benchmarks, and its 0.6B model rivals or exceeds guards more than 10× larger.The English results cover prompt and response classification.Source: Table 2 · p. 7, Table 3 · p. 7, §3.4.1 · pp. 7, 9
• Qwen3Guard-Gen significantly outperforms previous guard models on the Think benchmark for moderation of internal reasoning traces.The Think set contains manually annotated reasoning traces and responses generated from Beavertails prompts using existing reasoning models.Source: §3.4 · p. 6, Table 3 · p. 7, §3.4.1 · p. 9
• Qwen3Guard-Gen outperforms prior guards on 6 of 10 major languages for prompts and leads all 10 for responses, including limited-coverage languages.The multilingual evaluation includes RTP-LX prompt classification and PolyGuard-Response classification.Source: Table 5 · p. 8, Table 6 · p. 8, §3.4.1 · p. 9
• Figure 4 shows policy inconsistency is more pronounced in prompt than response classification, reflecting differing risk-tolerance approaches across benchmarks.WildGuard-7B aligns with Aegis but is overly conservative on OpenAIMod.Source: Figure 4 · p. 9, §3.4.1 · p. 9
• The controversial label can be treated as unsafe for stricter Aegis guidelines or safe for more permissive OpenAIMod guidelines.The paper reports that choosing the strict or loose mode improves scores over omitting the label on most datasets, especially ToxicChat and OpenAIModeration.Source: §3.4.1 · p. 9, Table 8 · p. 10, §3.4.2 · p. 10
• Category classification performs strongly across most harm categories, but rare Copyright cases are more difficult to classify.The evaluation uses an additional manually annotated test set of unsafe prompts and responses.Source: §3.4.1 · p. 9, Figure 5 · p. 10
• Qwen3Guard-Gen achieves refusal-detection results comparable to WildGuard-7B on XSTest and WildGuardTest.Source: §3.4.1 · p. 9, Table 7 · p. 10

3.4.2 Ablation Study

• Distillation improves average prompt scores by +0.47/+1.10 points and response scores by +0.5/+0.76 points.The report attributes the gains to filtering noisy annotations and making category decision boundaries more separable.Source: §3.4.2 · p. 10, Table 9 · p. 11

3.5 Application I: Safety RL with Generative Qwen3Guard

• Safety RL uses Qwen3-4B to improve robustness to harmful or adversarial prompts while seeking strong safety without simplistic blanket refusals.The application uses Generative Qwen3Guard’s response safety assessment as a reward signal.Source: §3.5 · p. 11

3.5.1 Reward Design

• Guard-only reward maximizes safety by rewarding responses classified “Safe” by Qwen3Guard-4B-Gen and treating “Unsafe” or “Controversial” as not safe.This formulation does not explicitly account for helpfulness or refusal behavior.Source: §3.5.1 · p. 11
• Hybrid reward jointly targets safety, helpfulness, and low refusal, using Qwen3Guard-4B-Gen judgments and WorldPM-Helpsteer2 helpfulness scores (Wang et al., 2025).The design addresses the risk that safety-only optimization could lead the model to refuse all queries.Source: §3.5.1 · p. 11

3.5.2 Experiment Settings

• GSPO (Zheng et al., 2025) training uses 13.7k thinking-mode and 6.7k non-thinking-mode samples after filtering WildJailbreak prompts with uniformly safe or unsafe responses.Qwen3-4B generated eight responses per prompt in both modes; the training data came from the Vanilla Harmful and Adversarial Harmful categories.Source: §3.5.2 · p. 11
• Evaluation uses 2,000 harmful and 210 benign WildJailbreak prompts, with safety judged by Qwen3-235B-Instruct-2507 and WildGuard rather than Qwen3Guard.WildGuard also classifies refusal rate; general utility is tested on Arena-Hard-v2, AIME-25, LiveCodeBench-V6, and GPQA.Source: §3.5.2 · p. 12

3.6 Experiment Results

• WildGuard safety rises from approximately 60 to over 97 under Hybrid Reward across both thinking modes, while arena-hard-v2 response quality marginally improves.The report concludes that Hybrid Reward produces a safer, more helpful model that retains high general capability.Source: §3.6 · p. 12
• Guard-only reward reaches near-perfect safety but has an extremely high refusal rate and slightly lowers arena-hard-v2 win rate, without noticeably affecting AIME25, LCB-v6, or GPQA.Source: §3.6 · p. 12
• During RL, Hybrid Reward avoids over-refusal while steadily and reliably improving safety measured by Qwen3Guard-Gen-4B, as shown in Figure 6.Source: Figure 6 · p. 12, §3.6 · p. 13

4.1 Architecture

• Two independent heads attached to the pretrained Qwen3 backbone’s final transformer layer classify response and query safety from the last hidden state.Each pathway predicts harm severity and safety category; incoming prompts are assessed alongside the LLM, and generated tokens are forwarded for per-token moderation and possible interruption.Source: §4.1 · p. 13, §4 · p. 13, Figure 7 · p. 13

4.2 Data Collection

• To create token-level labels from coarse unsafe or controversial sample labels, the method tests response prefixes using ensemble-generated rollouts and verifies flagged prefixes with an LLM judge.The rollout assessment uses an 85% safety-violation threshold, and judge verification avoids attributing risk to a token based only on harmful continuations.Source: §4.2 · p. 14

4.3 Training

• Training jointly optimizes query and response classification heads with cross-entropy, computing query loss at the end-of-query token and response loss at every generated token.Category loss is included only when the ground-truth risk level is unsafe or controversial.Source: §4.3 · pp. 14–15

4.4 Evaluation

• Stream Qwen3Guard’s average performance drop versus Generative Qwen3Guard is around two points, while it remains competitive with prior guard models despite using partial context under strict latency constraints.The evaluation covers English and Chinese harmful-prompt and response benchmarks and multilingual prompt and response tasks.Source: §4.4 · p. 15
• Nearly 86.0% of response-only cases have the first flagged token within the human-annotated sentence, while with thinking content, detection occurs within the first 128 tokens in approximately 66.8% of cases.The evaluation uses sentence-level human labels and filters samples with inconsistent annotations.Source: §4.4 · p. 18
• Stream Qwen3Guard processes each token without reprocessing prior tokens, so its processing time scales nearly linearly with response length while Generative Qwen3Guard incurs substantially higher overhead as responses grow.The comparison simulates Generative Qwen3Guard by resubmitting the accumulated response after each 32-token chunk.Source: §4.4 · p. 18, Figure 9 · p. 19

4.5 Application II: Real-time Safety Intervention with Stream Qwen3Guard

• CARE with Stream Qwen3Guard raises the Qwen3-235B-A22B-evaluated safety score from 47.5 to 85.7 in non-think mode and from 43.8 to 72.0 in think mode, while response quality also increases.CARE selectively rolls back and intervenes on outputs likely to be harmful, preserving normal interactions; Wait Tokens are higher in think mode due to more opportunities for detected violations and correction.Source: §4.5 · p. 19, Table 16 · p. 19

5 Related Work

• Most guard models monitor inputs and outputs with instruction-following classifiers trained on discrete safety categories, while related work also extends moderation to multilingual and multimodal settings.The cited guard-model examples include Inan et al. (2023), Jiang et al. (2024), Zeng et al. (2024), and Ghosh et al. (2025); multilingual and multimodal examples include Upadhayay et al. (2025), Verma et al. (2025), Kumar et al. (2025), and Gu et al. (2024).Source: §5 · p. 20
• DuoGuard improves safety enforcement through reinforcement learning, while GuardReasoner improves classification accuracy by explicitly modeling the reasoning behind safety judgments.These works illustrate recent efforts to add explicit reasoning capabilities to guard models (Deng et al., 2025; Liu et al., 2025).Source: §5 · p. 20
• Prior token-level methods use sentence-level annotations or indirect learning, whereas Stream Qwen3Guard estimates incomplete-sentence labels from Generative Qwen3Guard predictions across multiple rollouts.The passage cites Sharma et al. (2025), Xuan et al. (2025), and Li et al. (2025a) as prior token-level approaches.Source: §5 · p. 20
• Zhang et al. (2024) introduce a controversial label with rule-based annotations, while Qwen3Guard derives such labels through ensemble voting; Zeng et al. (2024) instead adjust thresholds dynamically.Source: §5 · p. 20

• Qwen3Guard adds a controversial category to binary safety labels and offers generative classification and token-level monitoring for intervention during incremental generation.The controversial category supports moderation decisions that can vary across regions, platforms, or use cases.Source: §6 · p. 20
• Experiments show strong performance across safety benchmarks spanning English, Chinese, and multilingual datasets.Source: §6 · p. 20
• Qwen3Guard is not a complete safety solution; the authors call for safer model architectures and training, and dynamic inference-time interventions that adapt to emerging risks.They envision safety frameworks that adapt alongside AI capabilities and shifting cultural contexts.Source: §6 · p. 20

Vulnerability to Adversarial Attacks

• Sophisticated paraphrasing, obfuscation, or context manipulation may bypass Qwen3Guard, degrading performance under targeted attacks despite robustness on standard benchmarks.The report identifies adversarial prompt engineering as a risk shared with most guardrail models.Source: Limitations · p. 20

Fairness and Bias in Moderation Decisions

• Training-data biases may lead Qwen3Guard to disproportionately flag contextually appropriate content from some demographic, linguistic, or cultural groups as unsafe or controversial.The report says this warrants attention to fairness and inclusivity, especially in multilingual and multicultural contexts.Source: Limitations · pp. 20–21

Limited Generalization

• Stream-Guard’s latency and partial-context constraints can reduce robustness on ambiguous phrases early in generation and raise error rates in out-of-distribution or low-resource-language scenarios.These limitations may reduce reliability in highly dynamic or noisy environments.Source: Limitations · p. 21

Insensitivity to Regional and Cultural Nuances

• Without geolocation-aware or policy-adaptive mechanisms, Qwen3Guard may miss regional legal frameworks, social norms, or cultural sensitivities in localized deployments.Source: Limitations · p. 21

• The authors commit to addressing these limitations through continued research and iterative improvements to the safety classifiers.Source: Limitations · p. 21

6.1 Instruction Template of Generative Qwen3Guard

• Figures 10 and 11 show the detailed prompts used for Generative Qwen3Guard prompt and response classification.Source: Figure 10 · p. 22, Figure 11 · p. 22

6.2 Categories Classification Results of Stream Qwen3Guard

• Stream Qwen3Guard distinguishes different safety categories in most unsafe scenarios, with confusion matrices shown for unsafe prompts and responses.The evaluation used the same category dataset as Generative Qwen3Guard.Source: §6.2 · p. 22, Figure 12 · p. 23

6.3 Illustrative Example of Sentence-Level Annotation for Stream Qwen3Guard’s Latency Evaluation

• Sentence-level latency annotations address token-level annotation difficulties and low inter-annotator agreement; the example marks the second of four sentences as the first unsafe instance.Annotators identified the sentence where the response first exhibited unsafe characteristics.Source: §6.3 · p. 23, Table 17 · p. 23

6.4 A Case Study of Safety RL

• After safety reinforcement learning, Qwen3-4B-SafeRL avoids harmful advice while still trying to fulfill a WildJailbreak request without outright refusal.Source: §6.4 · p. 23