PaperlayerView exact revision on arXiv

Source-linked AI summary

UniGuard: Multi-View Guardrail Learning for Cross-Interface Safety Generalization

Viet-Thanh Pham, Thuy-Trang Vu, Junae Kim, Rollin Omari, Dinh Phung, Gholamreza Haffari, Lizhen Qu

arXiv:2610.30742v1cs.CL

TL;DR

Guardrails trained on fixed interfaces can generalize poorly when the same safety case appears in conversations or agent traces, and their confidence can be unreliable under interface shift. UniGuard trains on aligned views of 1,000 safety cases using classification, contrastive learning, and asymmetric calibration. It improves over its base guardrail on unseen-interface benchmarks while remaining competitive with much larger guardrails.

  • Problem

    Guardrails often fail to generalize across interfaces even when the underlying harm is identical, and their confidence can be unreliable under interface shift.

    Source: §1 · p. 1

  • Method

    UniGuard trains on aligned views of safety cases with classification, supervised contrastive learning, and asymmetric confidence calibration objectives.

    Source: §1 · p. 2

  • Results

    +45.12 F1 and -35.41 Expected Calibration Error over its base guardrail on unseen-interface benchmarks, while remaining competitive with much larger proprietary guardrails.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Cross-view agreement serves as a shared supervision signal for cross-interface generalization and confidence calibration.

    Source: §1 · p. 2

    The evidence supports improved generalization across studied interfaces, not robustness to entirely new safety domains, tools, policies, or uncovered emergent multi-agent failures.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Guardrail models are increasingly used as safety layers for LLMs and agentic systems, but most are trained on fixed prompt or response interfaces and become unreliable when the same safety case appears as a conversation, agent trajectory, multi-agent trace, or policy-conditioned input. We propose UniGuard, a training framework that learns interface-invariant safety representations from multiple semantically aligned views of the same safety case. We also construct the UniGuard-FT dataset by transforming 1,000 public safety cases into prompt, response, multi-turn conversation, agent-trace, and multi-agent-trace views, and train UniGuard-0.6B with classification, supervised contrastive learning, and asymmetric confidence calibration losses. Across unseen-interface benchmarks ATBench, PolicyGuardBench, and DynaBench, UniGuard-0.6B improves over its pretrained guardrail by 45.12 F1 points on average and reduces expected calibration error by 35.41 points, while remaining competitive with much larger guardrails such as Claude-Sonnet-4.5 and GPT-5.2 - despite being a 0.6B model and trained only with synthetic data.

1 Introduction

UniGuard addresses guardrails’ fragmented generalization across interfaces and unreliable confidence under interface shift by learning from aligned views of the same safety case. Its reported results show improved unseen-interface performance and calibration over its base guardrail.

1 Introduction

  • A single safety case can surface as a prompt, a tool call within an agent trajectory, or an unauthorized disclosure in a multi-agent system, yet guardrails often fail across interfaces.The paper identifies fragmented generalization as a limitation of both general and specialized guardrails.Source: §1 · p. 1
  • Unseen-interface calibration matters because guardrails route by predicted probabilities, and overconfident errors can auto-accept harm while underconfident correct predictions burden review queues.The paper reports that all tested public guardrails exceed 30 ECE points on at least one unseen interface.Source: §1 · p. 1
  • UniGuard treats prompts, responses, conversations, agent traces, and multi-agent traces as aligned views, training with classification, supervised contrastive learning, and asymmetric confidence calibration.The calibration objective uses cross-view agreement as a reliability target and penalizes overconfidence more strongly than underconfidence.Source: §1 · p. 2
  • The framework uses cross-view agreement as a shared supervision signal for both cross-interface generalization and confidence calibration.Source: §1 · p. 2
  • UniGuard-FT renders 1,000 public safety cases into prompt, response, conversation, agent-trace, and multi-agent-trace views, with failure-mode annotations and a public release plan.Source: §1 · p. 2
  • +45.12 F1 and -35.41 Expected Calibration Error over the base guardrail on unseen-interface benchmarks, while remaining competitive with much larger proprietary guardrails.The model retains performance on six prompt and response benchmarks despite using 600M parameters and synthetic training data.Source: §1 · p. 2

2 UNIGUARD: Multi-View Guardrail Learning

UniGuard treats each safety case as multiple interface-specific views and trains a shared guardrail to preserve safety-label discrimination while aligning representations and calibrating confidence across views.

2.1 Multi-view Learning

  • A safety case is a latent example observed through serialized guardrail inputs that vary in interface specification, including target, interaction, agency, context, policy, and serialization.Unseen views differ from training support along one or more of these dimensions.Source: §2.1 · p. 2

2.2 Framework Overview

  • UniGuard processes serialized views of each safety case through classification, contrastive alignment, and confidence calibration objectives.Same-case, same-label views supply a shared signal for representation alignment and cross-view reliability estimation.Source: §2.2 · p. 3

2.3 Architecture

  • A shared pretrained guardrail produces final-layer token states that attention pooling converts into a view representation for classification and contrastive projection.The shared backbone handles all input formats; appended pooling and classification modules produce a compact safety representation.Source: §2.3 · pp. 3–4

2.4 Classification Objective

  • The classification loss preserves per-view SAFE or UNSAFE prediction behavior by training on each input view’s gold label.The loss uses the predicted probability assigned to that view’s gold label.Source: §2.4 · p. 4

2.5 Multi-view Contrastive Objective

  • Supervised contrastive learning pulls together same-case, same-label views and contrasts them with views from different cases or labels.Anchors without a positive view are excluded from the contrastive loss.Source: §2.5 · p. 4

2.6 Asymmetric Confidence Calibration

  • Asymmetric calibration compares an anchor’s confidence with reliability estimated from same-case, same-label views, weighting overconfidence more heavily by default.A single-view group is omitted; the reliability estimate can be detached, and the default higher overconfidence weight reflects the greater harm attributed to overconfident unfamiliar-format decisions.Source: §2.6 · pp. 4–5

2.7 Overall Training Objective

  • The overall objective combines classification, multi-view alignment, and asymmetric confidence calibration, with batches sampled to include multiple views of the same case.This sampling supplies positive pairs for contrastive learning and same-case views for reliability estimation.Source: §2.7 · p. 5

3 UNIGUARD-FT: Training Data Construction

UniGuard-FT is built from 1,000 stratified public safety cases rendered as aligned views across five guardrail interfaces, then used to fine-tune UniGuard-0.6B.

3 UNIGUARD-FT: Training Data Construction

  • The authors sample 1,000 cases from WildGuard, Aegis 2.0, BeaverTails, and ToxicChat after scoring clarity and difficulty and stratifying across those scores.The sampling is intended to cover both clear and difficult safety decisions.Source: §3 · p. 5
  • Each case receives semantically aligned prompt, response, conversation, agentic-trace, and multi-agent-trace views, with generated views changing interface structure while preserving the safety case.Agentic views follow realistic tool-use and multi-agent patterns and cover diverse safety failure modes; conversation views use conversation-specific failure modes.Source: §3 · p. 5
  • The resulting UNIGUARD-FT dataset is used to fine-tune UNIGUARD-0.6B from Qwen3Guard-Gen-0.6B.User-prompt and assistant-response views are taken directly from source examples when available.Source: §3 · p. 5

4 Experimental Setup

Across three unseen-interface benchmarks, UniGuard improves classification and calibration over its base model, while retaining prompt and response performance. The experiments also examine how its training objectives, representations, and confidence scores affect those results.

4.1 Baseline Guardrails; 4.2 Confidence Calibration Baselines

  • Baselines span prompt/response, agent-trace, and policy-aware guardrails; calibration comparisons include Temperature Scaling, Thermometer, and Contextual Calibration.Labels are mapped to SAFE and UNSAFE when needed for comparable binary evaluation; calibration baselines use their specified validation or content-free-input procedures.Source: §4.1 · p. 5, §4.2 · p. 6

4.3 Guardrail Benchmarks; 4.4 Evaluation Metrics

  • The evaluation tests schema-unseen agent trajectories and policy-unseen behavior, alongside retention on six prompt/response benchmarks, using F1 and ECE.F1 measures binary safety classification; ECE compares confidence with empirical accuracy across confidence bins.Source: §4.3 · p. 6, §4.4 · p. 6

4.6 Discussions

  • Fine-tuning changes average F1 by only -0.35 points and average ECE by -0.04 points across eight standard prompt/response benchmarks.Individual F1 differences remain within about one point.Source: §4.6 · p. 6
  • Removing contrastive learning drops average F1 by 20.50 points, while removing asymmetric calibration raises ECE by 18.51, 28.10, and 19.42 points across the three benchmarks.Removing both objectives drops average F1 by 12.97 points and raises average ECE by 24.90 points; the results associate contrastive learning with discriminative generalization and asymmetric calibration with confidence reliability.Source: §4.6 · p. 6

4.5 Main Results

  • 45.12 F1 points higher and 35.41 ECE points lower on average than Qwen3Guard-Gen-0.6B, with stable performance across all three unseen-interface settings.It closely matches GPT-5.2 on ATBench; AgentDoG-4B is strongest there but weaker on policy-aware settings, while PolicyGuard-4B performs well on PolicyGuardBench but fails on ATBench.Source: §4.5 · p. 6, Table 2 · p. 7

4.6 Discussions

  • The full model’s representation is more compact and label-consistent than the base and non-contrastive variants, with same-label views closer together.Cross-view retrieval improves over the base model from 0.030 to 0.431 on Same-Case@1 and from 0.110 to 0.589 on Cross-View@5.Source: §4.6 · pp. 7–8
  • Asymmetric calibration reaches average ECE 7.94, versus 18.46 for Temperature Scaling, 10.25 for Thermometer, and 15.34 for Contextual Calibration, with comparable average F1.Without calibration, selective F1 is achieved at coverage of 34.11%, 41.92%, and 40.67%; UniGuard maintains coverage of 73.26%, 78.35%, and 75.92% on ATBench, PolicyGuardBench, and DynaBench, respectively.Source: §4.6 · p. 8
  • Additional ablations find that explicit policy data mainly improves PolicyGuardBench, larger backbones yield small consistent gains, and agentic and multi-agent views are especially important.Consistency regularization and triplet loss remain competitive alternatives to supervised contrastive learning; the excerpt does not complete its comparison of prototypical objectives.Source: §4.6 · p. 8

5 Related Work

Related work spans guardrails for fixed prompt/response formats and specialized agentic or policy-conditioned settings. UniGuard connects these lines through aligned safety-case views and applies contrastive learning and calibration during training.

5 Related Work

6 Conclusions

UniGuard uses aligned views of safety cases to train a guardrail for cross-interface generalization and confidence calibration. Experiments report gains on unseen-format benchmarks while preserving original prompt/response performance.

6 Conclusions

  • UniGuard is a guardrail training framework for cross-interface generalization and confidence calibration using aligned prompt, response, conversation, agent-trace, and multi-agent-trace views.Source: §6 · p. 8
  • UniGuard-0.6B is trained with classification, contrastive learning, and asymmetric calibration objectives.Source: §6 · p. 8
  • Experiments show improved unseen-format performance while preserving prompt/response classification, and competitiveness with much larger guardrails despite 600M parameters and synthetic training data.Future work will extend UniGuard-FT with real-world traces, broader policies, and stronger domain-generalization evaluations.Source: §6 · p. 9

Limitations

UniGuard targets cross-interface generalization, not full domain or content generalization. Its benchmark results support claims only across the studied interfaces, not safety in arbitrary deployments.

Limitations

  • Strong ATBench, PolicyGuardBench, and DynaBench results are evidence of generalization across the studied interfaces, not a guarantee of safety in arbitrary future deployments.The work does not claim robustness to entirely new safety domains, tools, policies, or emergent multi-agent failure modes not covered by UNIGUARD-FT.Source: Limitations · p. 9
  • The evaluation focuses on cross-interface rather than full domain or content generalization.Source: Limitations · p. 9
  • Synthetic, limited-scale UNIGUARD-FT traces may simplify real agent systems, where tool use, memory, environment feedback, and interaction patterns vary widely.The authors propose expanding the dataset with real deployment traces, broader policies, and more diverse agent frameworks.Source: Limitations · p. 9

Ethical Statement

The ethical statement flags harmful training content, misuse, and errors as risks, and recommends safeguards for releases and deployment. The view ontology distinguishes safety cases from their serialized interfaces and types of unseen views.

Ethical Statement

  • Training data contains harmful or policy-violating content, and its construction uses an uncensored model to generate safety-relevant views.The authors say releases should include documentation, access controls when needed, and warnings that the data is intended for safety research rather than harmful behavior.Source: Ethical Statement · p. 9
  • Misuse and false-negative or false-positive errors remain risks, so UNIGUARD should be one component of a broader safety pipeline, not a complete solution.The authors name human oversight, logging, appeal mechanisms, and system-level constraints as safeguards, especially in high-stakes settings.Source: Ethical Statement · p. 9
  • Synthetic training views may miss real-system complexity and reflect biases from source datasets, the generator, and prompted failure modes.The authors caution that benchmark performance is not a guarantee of safety in open-ended deployments.Source: Ethical Statement · p. 9

A View Ontology and Unseen View Types

  • The ontology separates a latent safety case from the interface used to serialize and present it to a guardrail.Unseen views can differ by schema or policy conditioning, even without introducing a wholly new view family.Source: A View Ontology and Unseen View Types · pp. 11–12

B.1 Data Statistics

The data-statistics section describes counts and token lengths across view types and failure modes. The examples section presents generated unsafe views for conversation, agentic, and multi-agent settings.

B.1 Data Statistics

  • Table 5 reports sample counts by view type and token-length statistics for UNIGUARD-FT.Source: §B.1 · p. 12
  • Tables 6–8 report failure-mode counts for agentic traces, multi-agent trajectories, and generated multi-turn conversations.Source: §B.1 · p. 12

B.2 Training Examples

  • The examples include an agentic trace unsafe because an agent selects a malicious tool, and a multi-agent trace unsafe because a reviewer ignores a privacy flag, causing data leakage.The section also presents an unsafe multi-turn conversation view.Source: §B.2 · p. 12

C Training Setups

UNIGUARD-0.6B is fully fine-tuned from Qwen3Guard-Gen-0.6B with batches containing multiple views of the same safety case. The calibration-baseline section describes temperature, auxiliary, and contextual calibration methods.

C Training Setups

  • Training fully fine-tunes Qwen3Guard-Gen-0.6B, using batches with multiple views of each safety case and 256-dimensional normalized projected embeddings.The setup uses AdamW, a cosine learning-rate schedule with warmup, gradient clipping, and a maximum sequence length of 16K tokens.Source: C Training Setups · p. 12

D.1 Confidence Calibration Baselines

  • Temperature scaling tunes a scalar on held-out development data and rescales logits without changing the predicted class when T > 0.The temperature is tuned on the UNIGUARD-FT validation subset.Source: §D.1 · p. 12
  • Thermometer trains an auxiliary calibrator on UNIGUARD-FT to predict a temperature parameter for each LLM output across tasks.Source: §D.1 · p. 12
  • Contextual calibration estimates label bias from content-free inputs and discounts that prior confidence when calibrating real predictions.The method can detect a default SAFE or UNSAFE preference on meaningless inputs.Source: §D.1 · pp. 12–13

D.2 Benchmark Descriptions

The benchmarks test guardrails across long-horizon agent trajectories, policy-conditioned judgments, and standard prompt- and response-level safety classification.

D.2 Benchmark Descriptions

  • ATBench tests agent safety in long-horizon tool use by requiring inspection of tools, intermediate calls, observations, and final behavior, not just the initial instruction [Li et al., 2026].It is used to assess performance on unseen agentic formats.Source: §D.2 · p. 13
  • PolicyGuardBench tests whether an agent trajectory violates an externally specified policy, providing an unseen-policy evaluation because training data excludes policies outside an ablation.Source: §D.2 · p. 13
  • DynaBench tests compliance with dynamically specified policies, where the applicable rule can vary across examples and differ from standard harmfulness categories [Hoover et al., 2026].Source: §D.2 · pp. 13–14
  • Prompt- and response-level benchmarks assess whether UniGuard-0.6B retains Qwen3Guard-Gen-0.6B’s original classification ability across datasets including ToxicChat [Lin et al., 2023], OpenAI Moderation [Markov et al., 2023b], and BeaverTails [Ji et al., 2023].Other listed datasets are WildGuard Test, Aegis 2.0 [Ghosh et al., 2025], and SafeRLHF [Ji et al., 2025].Source: §D.2 · p. 14

E.1 Cross-view Retrieval Analysis

Across four analyses, UniGuard’s contrastive objective improves cross-view retrieval, asymmetric calibration supports higher acceptance coverage under a fixed threshold protocol, and policy additions or larger backbones yield narrower gains.

E.2 Calibration-Aware Deferral

  • 73.26% on ATBench, 78.35% on PolicyGuardBench, and 75.92% on DynaBench: asymmetric calibration preserves higher coverage than the uncalibrated variant under the same threshold-selection protocol.Without calibration, coverage falls to 34.11%, 41.92%, and 40.67%, respectively; the calibrated model’s selective F1 is lower, so the comparison should be read jointly with coverage.Source: §E.2 · pp. 15–16, Table 11 · p. 15

E.1 Cross-view Retrieval Analysis

  • 0.395 Cross-View@1, 0.589 Cross-View@5, and 0.689 Cross-View@10: UniGuard-0.6B exceeds the non-contrastive variant by 12.4, 14.8, and 14.7 absolute points, respectively.It also reaches 0.431 Same-Case@1, versus 0.030 for the base model and 0.181 without contrastive learning, supporting improved organization of same-case views across interfaces.Source: §E.1 · pp. 14–15, Table 12 · p. 16

E.3 Adding Safety Policies

  • 70.26 versus 64.78 on PolicyGuardBench: adding its training-subset policies improves that benchmark, but yields only minor gains on ATBench and DynaBench.The authors contrast this with aligning semantically equivalent safety cases across heterogeneous views rather than relying only on explicit policy text.Source: §E.3 · p. 16

E.4 Increasing UNIGUARD Size

  • Scaling from UniGuard-0.6B to 4B brings slight gains on ATBench and PolicyGuardBench and comparable DynaBench performance; 8B does not consistently outperform 4B.The authors report that the larger variants improve over their corresponding base guardrails, while the small gap from 0.6B suggests training-signal structure matters beyond model size alone.Source: §E.4 · p. 16

E.5 Importance of Training Data Views

This section examines the role of training-data views in cross-interface generalization and describes how agentic and multi-agent traces are constructed to preserve cases’ failure modes and difficulty while exposing safety-relevant behavior.

E.5 Importance of Training Data Views

  • 66.40 to 62.35 on ATBench and 64.78 to 60.45 on PolicyGuardBench when agent-trace training views are removed; removing multi-agent views also hurts ATBench and DynaBench.Removing prompt, response, or conversation views causes only small drops. Agent traces include structured actions, observations, tool outputs, and intermediate decisions, encouraging representations tied to behavioral patterns rather than surface text.Source: §E.5 · pp. 16–17

E.6 Alternatives to Contrastive Loss

  • Consistency regularization is close but slightly worse across benchmarks; triplet loss is strong on ATBench but less robust on PolicyGuardBench and DynaBench, while prototypical variants underperform contrastive learning.Supervised contrastive learning supplies a denser signal by comparing each example with multiple positives and negatives; class-wise prototypes are too coarse, and case-wise prototypes still reduce each case to a centroid.Source: §E.6 · p. 17

F.1 Prompts Description

  • UniGuard-FT uses one prompt to score source samples and others to generate conversation, agentic-trace, and multi-agent-trace views, using uncensored GLM-4.7-Flash 1 to avoid refusals.Source: §F.1 · p. 17

F.2 Failure Modes Description

  • Failure-mode guidance supports generation of agentic, multi-agent, and multi-turn views; generated conversations preserve the assigned mode and label and are judged by assistant behavior across turns, not the user request alone.Source: §F.2 · pp. 18, 20
  • Generated traces must preserve the source failure mode and difficulty, and classify agent, tool, delegation, or final-output behavior rather than the inbound request alone.They should expose a consequential decision, such as whether the agent validates tool output, obtains authorization, or withholds a risky action.Source: §F.2 · pp. 23–24
  • The specification calls for 6–12 distinct ordinary tools and 6–12 trace events, with plausible outputs and internally consistent timestamps and freshness claims.A consequential tool action—not generic web search or a classifier that announces the answer—should determine the label.Source: §F.2 · pp. 23–24
  • The requested formats use framework-specific traces: LangChain-like nodes, CrewAI-style agent delegation, or AutoGen typed multi-agent events with review or disagreement.The examples specify that unsafe outcomes follow problematic branches or over-trust prior events, while safe outcomes show validation, review, or refusal of the risky action.Source: §F.2 · p. 24

Related papers

Loading 2610.30742v1…