PaperlayerView exact revision on arXiv

Source-linked AI summary

SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

SingGuard Team

arXiv:2606.22873v3cs.CVcs.CL

TL;DR

Existing multimodal guardrails often rely on fixed taxonomies, limiting policy adaptation across changing deployment settings. SingGuard conditions multimodal safety judgments on runtime rules and achieves state-of-the-art average F1 across six benchmark families, while policy-following accuracy rises from 0.6465 to 0.7415 under runtime shifts.

  • Problem

    Existing guardrails typically use fixed taxonomies or narrow interaction settings, limiting adaptation when safety policies change at deployment time.

  • Method

    SingGuard uses runtime natural-language policies for rule-by-rule multimodal safety assessment, with fast, hybrid, and slow inference regimes.

  • Results

    SingGuard achieves state-of-the-art average F1 across six benchmark families, while dynamic-policy accuracy improves from 0.6465 to 0.7415.

  • Takeaways & Limitations

    Runtime-policy conditioning supports broad multimodal moderation while combining policy-following with low-latency and interpretable inference options.

  • Takeaways & Limitations

    SingGuard depends on clear active policies, and its benchmark cannot exhaust real-world safety policies or emerging abuse patterns.

Abstract

from arXiv · show

Vision-language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from multimodal question answering, assistant responses, and cross-modal composition, while moderation policies may vary across products, regions, and deployment stages. Most existing guardrails either rely on fixed taxonomies or target only a narrow set of interaction settings, which limits their adaptability when safety rules change at deployment time. We present \textbf{SingGuard}, a policy-adaptive multimodal guardrail model family for safety assessment in multimodal conversations. SingGuard treats the active policy as a runtime input: given natural-language rules, it checks the target content against the active policy rule by rule and predicts both the safety label and the triggered rule. To balance efficiency and interpretability, SingGuard supports fast, hybrid, and slow inference regimes along a fast-to-slow reasoning spectrum, ranging from direct safety judgments to policy-grounded deliberation. We further optimize this behavior with fast--slow decoupled reinforcement learning. We also introduce \textbf{SingGuard-Bench}, a multimodal guardrail benchmark with 56{,}340 examples spanning 80+ fine-grained risk types across multimodal QA, adversarial attack, and dynamic-rule evaluation settings, including cross-modal joint-risk cases where each modality is harmless in isolation but their composition implies unsafe intent. Across six benchmark families (35 datasets), SingGuard achieves state-of-the-art average F1 in every family. Dynamic-rule evaluation further shows improved policy-following accuracy from 0.6465 to 0.7415 under runtime policy shifts. Our code is available at https://github.com/inclusionAI/Sing-Guard.

1 Introduction

SingGuard is introduced as a policy-adaptive multimodal guardrail that accepts runtime natural-language policies and adapts reasoning depth to moderation needs. SingGuard-Bench evaluates this capability across broad multimodal, adversarial, cross-modal, and dynamic-rule settings, where SingGuard achieves state-of-the-art average F1 across six benchmark families and improves dynamic-policy accuracy from 0.6465 to 0.7415.

  • SingGuard: SingGuard treats the active policy as a runtime input, matching content against rules and predicting both safety labels and triggered rules.Policies can use concise category names or detailed natural-language rules with scope, exceptions, and domain-specific constraints.
  • Adaptive reasoning: SingGuard offers fast, hybrid, and slow inference modes spanning direct judgments, conditional reasoning, and explicit rule-by-rule deliberation.The modes target different latency, evidence, and auditability requirements.
  • SingGuard-Bench: SingGuard-Bench contains more than 80 fine-grained risk types across multimodal, attack-style, dynamic-rule, and cross-modal joint-risk evaluations.Joint-risk cases are individually benign by modality but unsafe in composition.
  • Results: SingGuard achieves state-of-the-art average performance across six benchmark families spanning 35 underlying datasets and evaluation splits.The result covers multimodal QA and response-moderation settings rather than a single isolated benchmark.
  • Results: 0.7415 average accuracy is achieved by SingGuard-slow under dynamic-policy evaluation, improving from 0.6465 for Qwen3-VL-8B.This result demonstrates improved accuracy under runtime rule conditioning.

2 Method

SingGuard is a policy-conditioned multimodal guardrail that evaluates content against runtime-supplied rules, producing safety labels, triggered rules, and optional policy-grounded reasoning. It combines a unified hierarchical taxonomy with policy-adaptive training and fast–slow inference to support varied moderation targets and changing policies.

  • Policy-conditioned formulation: The model predicts safe or unsafe, the triggered category or rule title, and an optional policy-grounded reasoning trace.Triggered categories must come from the categories or rule titles activated by the current policy, with Safe available for non-triggered content.
  • Training and inference: Training combines policy-conditioned cold-start SFT with fast–slow decoupled DAPO to separate initial judgments from later policy-grounded deliberation.The cold-start stage injects the judge–reason–review paradigm under a shared output grammar, while the decoupled approach reduces anchoring from the first fast token.
  • Policy-conditioned formulation: SingGuard takes explicit moderation instructions, an active policy, target content, and output requirements to generate a structured, policy-grounded moderation decision.Unlike fixed-label classifiers, its decision is grounded in the policy supplied at inference time.
  • Policy-conditioned formulation: The interface covers text-only queries, image inputs, image-text queries, assistant responses, and full query-response conversations under the same active policy.Assessment jointly considers user intent, visual evidence, assistant behavior, and cross-modal interaction when relevant.
  • Runtime policy: SingGuard uses a default taxonomy with 8 primary dimensions, 27 secondary categories, and more than 80 fine-grained risk types, while allowing runtime policy variants.The active policy may be the full taxonomy, a task-specific subset, narrowed or expanded rules, or newly introduced domain-specific rules.

3 SingGuard-Bench

SingGuard-Bench is a policy-conditioned multimodal safety benchmark designed for fine-grained, production-oriented guardrail evaluation. It combines broad risk coverage with adversarial cross-modal and runtime policy-following tests under a unified guardrail interface.

  • Benchmark scope: SingGuard-Bench contains 56,340 test examples spanning more than 80 fine-grained risk types, including image, multimodal, and dynamic-rule cases.The benchmark includes 40,663 image samples, 13,677 multimodal samples, and 2,000 dynamic-rule examples.
  • Unified interface: Each instance pairs target content with an active policy, safety label, and fine-grained risk category or rule title for unified evaluation.Dynamic-rule examples pair identical content with matching or non-matching rules, making the active rule set—not a static dataset label—determine correctness.
  • Evaluation axes: The benchmark evaluates adversarial robustness, harmful-content recall, benign-sensitive precision, and policy adaptation across complementary sample, taxonomy, attack, and dynamic-policy axes.Its coverage includes adversarial attacks, direct harms, benign-sensitive samples, keyword-level category coverage, hidden cross-modal intent, and matching or non-matching policy rules.
  • Adversarial evaluation: Cross-modal hidden-intent attacks split harmful evidence across image and text so that each modality appears benign alone while their combination reveals unsafe intent.The benchmark also includes ordinary harmful image–text combinations and conventional transformations such as Typography and Patch Shuffle.
  • Dynamic-rule evaluation: The dynamic-rule subset contains 2,000 examples evenly distributed across unsafe→unsafe, unsafe→safe, safe→unsafe, and safe→safe policy shifts.Each case contains 500 samples and tests policy following under both rule addition and removal.

4 Experiments

SingGuard is evaluated across six complementary safety axes using a standardized generative interface, with strong results across multimodal, image, text, multilingual, and dynamic-policy settings. Ablations further show that reinforcement learning, hybrid routing, and distillation improve the accuracy–efficiency tradeoff.

  • Evaluation scope: The evaluation spans multimodal query-response, image, text-query, text-response, multilingual, and dynamic-policy safety.These axes cover both content modalities and intent-versus-assistance moderation settings.
  • Multimodal safety: 0.9092 macro-average F1 is achieved by SingGuard-8B on multimodal safety, ahead of LLaVAShield at 0.8842 and GPT-5.1 at 0.8349.SingGuard-2B and SingGuard-4B reach 0.8924 and 0.8945, respectively.
  • Image safety: 0.9141 macro-average F1 is achieved by SingGuard-4B on image safety, exceeding GuardReasoner-VL at 0.8570.SingGuard-2B and SingGuard-8B obtain 0.9029 and 0.9099.
  • Text safety: 0.8740 macro-average F1 is achieved by SingGuard-8B on text-query safety, ahead of YuFeng-XGuard-Reason-8B at 0.8666 and Qwen3Guard-8B-strict at 0.8493.This setting tests malicious intent recall while avoiding over-refusal of ambiguous benign-sensitive requests.
  • Multilingual safety: 0.8872 and 0.8988 macro-average F1 are achieved by SingGuard-8B on multilingual query and response safety, respectively.SingGuard-8B leads both splits, while GraniteGuardian reaches 0.9734 on RTP-LX with near-saturated recall of 0.9999.
  • Dynamic policies and ablations: 0.7415 average accuracy is achieved by SingGuard-slow under dynamic policy shifts, improving safe2unsafe from Qwen3-VL-8B’s 0.3800 to 0.5700.Ablations show RL improves dynamic-policy performance, fast mode reduces latency, and on-policy distillation raises the 2B average from 0.8631 to 0.8840.

5 Related Work

Prior guardrails have progressed from fixed-taxonomy text moderation to richer reasoning and multimodal safety assessment, but most multimodal systems remain bounded by static policies. SingGuard extends this line with runtime rule conditioning, open policy edits, and inference paths adapted to policy difficulty and uncertainty.

  • Text guardrails: Llama Guard established predefined-taxonomy input-output safety classification, while later text guardrails broadened moderation beyond this formulation.WildGuard is identified as a subsequent system that broadens the formulation along multiple axes.
  • Text guardrails: Qwen3Guard uses instruction-following generation for fine-grained safe, controversial, and unsafe judgments, while GuardReasoner and YuFeng-XGuard expose richer reasoning outputs.These outputs include intermediate judgment traces, structured risk categories, confidence estimates, and tiered inference paths.
  • Safety benchmarks: Text safety benchmarks cover harmful prompts and responses, refusals, jailbreak robustness, exaggerated safety behavior, and multilingual moderation.The cited benchmark suite includes WildGuardMix, HarmBench, XSTest, StrongREJECT, BeaverTails, Aegis2.0, PolyGuardPrompts, and RTP-LX.
  • Multimodal guardrails: Multimodal guardrails inspect visual evidence through image safety classification, visual ratings and rationales, and multimodal multi-turn dialogue assessment.Representative systems include Llama Guard 3 Vision, ShieldGemma 2, LlavaGuard, and LLaVAShield.
  • Policy adaptation: Most multimodal guardrails still use static policy boundaries, whereas SingGuard conditions decisions on active rules, supports open rule edits at inference time, and routes among fast, hybrid, and slow paths.The motivation is that moderation rules can vary across products, regions, and business domains.

6 Conclusion

SingGuard is a policy-adaptive multimodal guardrail that conditions safety decisions on a runtime policy rather than a frozen taxonomy. Across six benchmark families, it achieves state-of-the-art average F1 and improves dynamic-policy following accuracy while addressing over-blocking.

  • Conclusion: SingGuard provides a unified runtime-policy interface for multimodal QA and assistant-response safety assessment.Its decisions are conditioned on the active policy, decoupling safety perception from memorized category priors.
  • Conclusion: The model combines a unified hierarchical taxonomy, dynamic rule-conditioned training, three inference regimes, and fast–slow decoupled reinforcement learning.The inference regimes are fast, hybrid, and slow.
  • Conclusion: 35 datasets across six benchmark families yield state-of-the-art average F1 for SingGuard in every family.The result spans the paper’s multimodal guardrail benchmark evaluation.
  • Conclusion: 0.6465 to 0.7415: dynamic-policy evaluation shows improved policy-following accuracy from Qwen3-VL-8B to SingGuard-slow.The conclusion emphasizes that deployed guardrails must identify risks outside the active policy and refrain from over-blocking.

7 Limitations

SingGuard’s policy-adaptive moderation remains dependent on the quality and clarity of the runtime policy. Ambiguous, incomplete, or conflicting rules and incomplete benchmark coverage may limit consistency and real-world generalization.

  • Policy Dependence: SingGuard’s decisions depend on the quality and clarity of the active policy supplied at inference time.This dependency can affect moderation reliability when runtime policies are underspecified.
  • Policy Dependence: Ambiguous, incomplete, or conflicting rules may cause inconsistent judgments across products, regions, or cultural contexts.The limitation arises because the active policy may not specify safety expectations consistently across deployment settings.
  • Benchmark Coverage: SingGuard-Bench cannot exhaust the long tail of real-world safety policies despite covering diverse multimodal QA, attack, and dynamic-rule settings.Its benchmark coverage is broad but remains incomplete relative to the range of policies encountered in practice.

8 Contributions

The section lists SingGuard contributors by contribution role, alphabetized within each group. Superscript marks identify equal contributors and corresponding authors.

  • Contributor groups: Yichen Bai, Liangbo He, Zongyi Li, Bingyan Liao, and Shenglin Yin are marked as equal contributors.Each name carries the ∗ notation.
  • Contributor groups: Yan Hong, Hongcheng Li, Siyuan Li, Chuanbiao Song, Kedong Xiu, Chao Xu, Tingting Xu, and Zijian Yu are listed in a contributor group.The names are presented in alphabetical order by last name.
  • Contributor groups: Shiwen Cui and Jun Lan are marked as corresponding authors alongside Changhua Meng, Weiqiang Wang, and Huijia Zhu.Cui and Lan carry the † notation.

A Prompt Templates · B Taxonomy Summary

SingGuard’s prompts make thinking mode and active policy runtime controls, supporting fast judgments, hybrid verification, and rule-by-rule reasoning. Its default taxonomy organizes risks into 8 primary dimensions, 27 secondary categories, and 80+ fine-grained risk types while allowing additional runtime rules.

  • A Prompt Templates: Inference templates accept optional thinking type and policy controls, replacing the default taxonomy with an active runtime rule set.The released models store the system instruction in the tokenizer chat template.
  • A.1 Default Policy-Grounded System Prompt: The default prompt evaluates multimodal conversations across Query, Image, and Response, including cross-modal combinations whose joint intent is unsafe.Responses are judged within the user’s input context, even when harmless in isolation.
  • A.2 Fast Judgment Prompt: Fast inference emits only a binary safety label and triggered category for low-latency deployment.The output format is safe/unsafe followed by the answer category.
  • A.3 Rule-by-Rule Reasoning Trace: Rule-by-rule reasoning summarizes content, checks each active rule, and aggregates results, allowing custom policies to replace the built-in taxonomy.In the example, a neutral rhinoceros image is judged safe because no active rule is violated.
  • A.4 Hybrid Reasoning Prompt: Hybrid inference first produces a binary judgment, then extends ambiguous or high-risk cases with policy-grounded verification while allowing confident cases to exit early.The early-exit output stops after the binary label when confidence is sufficient.
  • B Taxonomy Summary: The taxonomy contains 8 primary dimensions, 27 secondary categories, and 80+ fine-grained risk types, with runtime policies able to add open rules.The taxonomy summary and detailed taxonomy describe the default organization and its extensibility.
  • B Taxonomy Summary: The taxonomy covers agent safety risks such as system-prompt extraction, configuration probing, prompt injection, jailbreaking, and sensitive-attribute inference.These risks span prompt and internal-configuration leaks as well as model-behavior manipulation and unauthorized use.
  • B Taxonomy Summary: It also includes politically sensitive content, animal abuse, and benign no-risk content as structured categories with detailed subtypes.Examples include political rumors, subversive advocacy, unrest, historical distortion, attacks on political figures, and animal cruelty.

C Rule Isolation Mask for Parallel Multi-Rule Inference · C.1 Motivation

SingGuard isolates each category–rule block to improve moderation reliability, but serial evaluation incurs latency that grows with the number of active rules. The section motivates parallel inference methods that preserve isolated decision contexts while reducing redundant computation.

  • C Rule Isolation Mask for Parallel Multi-Rule Inference: Production moderation policies contain many active categories, each paired with its own natural-language rules.
  • C Rule Isolation Mask for Parallel Multi-Rule Inference: Independent category–rule testing is more reliable than classifying all categories in one joint prompt.
  • C Rule Isolation Mask for Parallel Multi-Rule Inference: Independent evaluation gives each target rule a clean decision context and reduces interference from unrelated policies.
  • C.1 Motivation: The central challenge is preserving independent-classification accuracy without incurring linear latency growth.
  • C.1 Motivation: Serial inference isolates each rule block, which tends to improve reliability, but latency increases with the number of active rules.
  • C.1 Motivation: Batch inference can parallelize independent sequences on GPUs using frameworks such as vLLM.
  • C.1 Motivation: Batching still redundantly encodes shared image and prompt tokens for every rule block.

C.2 Method

RI-Mask enables parallel multi-rule safety inference by sharing the multimodal content prefix while isolating each rule branch. Its content-first layout and custom attention mask preserve independent rule evaluation in one forward pass.

  • Rule Isolation Mask: RI-Mask places active rules after the shared system-prompt, image-token, and user-text prefix so content representations can be reused while rule branches remain isolated.This content-first, rule-postfix layout supports shared computation without mixing rule-specific processing.
  • Rule Isolation Mask: A packed sequence concatenates the shared prefix with N rule suffixes and output slots, using independently continued branch position IDs rather than packed-sequence offsets.Branch tokens continue from the shared prefix’s final position within each branch.
  • Rule Isolation Mask: The custom mask makes the prefix visible to every branch while restricting each rule suffix and output to the prefix and itself, preventing cross-rule interference.This avoids redundant KV-cache computation for the dominant shared prefix and preserves independent rule evaluation in one forward pass.

C.3 Implementation

RI-Mask is implemented with PyTorch FlexAttention and parallel packed rule branches. Prompts use a content-first, rule-postfix format with branch-local position IDs.

  • C.3 Implementation: RI-Mask uses PyTorch’s FlexAttention API for arbitrary block-sparse attention without custom CUDA kernels.This implementation supports the specified sparse attention patterns directly through FlexAttention.
  • C.3 Implementation: Prompts are rewritten before packing into a content-first, rule-postfix form.
  • C.3 Implementation: Position IDs are assigned branch-locally so packed rule branches start at |P| and continue within isolated branches.Although evaluated in parallel, each rule branch receives its own positional sequence after the shared prefix.

C.4 Discussion

RI-Mask targets SingGuard’s fast-mode deployment setting, where shared multimodal content dominates sequence length and each rule requires only a compact decision. By avoiding repeated content encoding while preserving rule isolation, it enables near-lossless parallel acceleration and generalizes to multi-criteria evaluation.

  • Deployment efficiency: RI-Mask is particularly effective for SingGuard because most online requests use fast-mode rule checking rather than slow deliberation.The deployment scenario emphasizes fast-mode processing over slow deliberation.
  • Deployment efficiency: Shared content prefixes dominate sequence length, while rule descriptions are short and branches emit compact hit/not-hit outputs.The prefix consists of image tokens and the system prompt.
  • Deployment efficiency: RI-Mask avoids repeatedly encoding shared content while preserving rule isolation, providing near-lossless parallel acceleration over independent single-rule inference.In a deployment-style offline-rule setting with 30 active rules, it accelerates SingGuard-2B multimodal inference by more than the truncated passage specifies.
  • Generalization: The technique generalizes to multi-label or multi-aspect evaluation in content moderation, compliance checking, and multidimensional quality assessment.These settings independently assess shared inputs against multiple criteria.
Loading 2606.22873v3…