PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaGuard: An Adaptive Guard Model with User-defined Policies

Yunhao Feng, Yifan Ding, Yuxiang Xie, Zheng Li, Mingrui Lao, Zeyuan Wang, Yanming Guo

arXiv:2609.34241v1cs.AI

TL;DR

Fixed safety taxonomies do not fully capture requirements that vary across applications, making policy-conditioned assessment of agent behavior important. AdaGuard pairs policy and behavioral counterfactual data with SafePO training; its 4B model reports binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench.

  • Problem

    Fixed safety taxonomies do not capture all application-specific rules, while trajectory assessment requires interpreting the supplied policy and the agent’s recorded interaction together.

    Source: §1 · pp. 1–2

  • Method

    AdaptiveSafety pairs policy and trajectory counterfactuals with explanations and complete violation lists, and SafePO combines structured verdict rewards with value-guided token weighting.

    Source: §1 · pp. 2–3

  • Results

    The 4B AdaGuard model achieves binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench.

    Source: §1 · p. 3

  • Takeaways & Limitations

    AdaGuard achieves strong detection and rule-identification results on AdaptiveSafety and supports both tasks on DynaBench.

    Source: §6 · p. 9

    SafePO’s reward measures verdict correctness and output structure, but does not independently verify the factual correctness of the analysis.

    Source: §3.2 · p. 6

Abstract

from arXiv · show

Guard models support the safe deployment of language model agents, but fixed risk taxonomies limit their ability to accommodate requirements that vary across applications and tasks. Under user-defined policies, detecting violations requires interpreting both the applicable rules and the agent's behavior, since identical actions can receive different judgments under different policies. To support learning this capability, we introduce AdaptiveSafety, a dataset of 10,939 training examples and 1,000 test examples covering policies with 1--100 rules. The dataset combines trajectories from multiple sources with policy and behavioral counterfactuals, pairing each example with an explanation and the complete set of violated rules. These counterfactuals expose changes that alter compliance, while structural augmentations provide supervision for consistency under rule reordering and identifier remapping. Building on this supervision, we propose SafePO, a reinforcement learning algorithm for refining violation identification while balancing explanatory reasoning and final verdicts. SafePO uses structured rewards to assess prediction correctness, retains group-relative advantages at the response level, and employs a separately trained value model to modulate token weights within explanation and verdict regions. Separate normalization controls their relative contribution to training despite differences in length. Through supervised initialization followed by SafePO, we develop AdaGuard, a family of 0.6B, 4B, and 8B guard models that assess agent trajectories under policies supplied at inference time. Our 4B model achieves binary accuracies of 89.30\% on AdaptiveSafety and 71.82\% on DynaBench. The project repository is available at https://github.com/Yunhao-Feng/AdaGuard

1 INTRODUCTION

Because permitted agent actions vary across deployment policies, fixed safety taxonomies do not fully address trajectory assessment. AdaGuard combines policy-conditioned data and SafePO training to assess agent behavior under supplied rules.

1 INTRODUCTION

  • Fixed safety criteria cannot capture permissions that vary by application, and tool-mediated behavior assessment must jointly interpret the policy and interaction trajectory.The same recorded operation can receive different judgments under different policies.Source: §1 · pp. 1–2
  • AdaptiveSafety contains 10,939 training and 1,000 test examples, pairing trajectories with explanations and complete violation lists across policy and behavioral counterfactuals.Policies range from 1 to 100 rules.Source: §1 · p. 2
  • SafePO combines structured verdict rewards with value-guided token weighting while preserving fixed weight budgets for analysis and verdict regions.Its reward distinguishes ordering errors from incorrect rule predictions and scores partial matches; token weighting adjusts emphasis within each region.Source: §1 · pp. 2–3
  • The 4B AdaGuard model achieves binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench.The model family includes 0.6B, 4B, and 8B parameter variants.Source: §1 · p. 3

2 PRELIMINARIES

The guard assesses agent behavior against a supplied policy, using the recorded interaction as evidence. The section formalizes policy and interaction inputs, structured analysis and verdict outputs, and learning notation.

2.1 POLICIES AND INTERACTION RECORDS

  • A policy is an ordered list of rules with unique identifiers, while an interaction record contains chronologically ordered events such as requests, actions, and tool observations.The trusted policy is placed in the system instruction and the interaction record in a delimited, untrusted input region.Source: §2.1 · p. 3
  • A violation judgment depends on the policy’s conditions and the interaction evidence, with the target set limited to policy rules supported as violated within scope.For user-only records, the request itself is assessed; a malicious request or injected instruction alone does not establish an agent violation.Source: §2.2 · p. 3, §2.1 · p. 3

2.2 ANALYSIS AND STRUCTURED VERDICTS

  • The guard generates one overall analysis and an ordered sequence of violated-rule identifiers, using NR for an empty sequence.The predicted identifiers must be unique and follow policy order; membership and serialization order are evaluated distinctly.Source: §2.2 · pp. 3–4

2.3 LEARNING NOTATION

  • Learning notation distinguishes the response-level group-relative advantage from a separate value model that predicts reward from response prefixes and modulates token weights.Prescribed analysis and verdict weight masses control their relative contribution independently of region lengths; inference requires only the guard model.Source: §2.3 · p. 4

3 METHOD

AdaGuard combines policy-conditioned supervision with SafePO to produce an overall analysis and a verdict identifying violated rules. Its method uses counterfactual data, structured rewards, and value-guided weighting across analysis and verdict tokens.

3 METHOD

  • AdaGuard maps a policy and interaction record to an analysis and verdict, then uses SafePO to refine structured assessments.SafePO separates verdict scoring from the distribution of learning signals across analysis and verdict tokens.Source: §3 · p. 4

3.1 ADAPTIVESAFETY AND SUPERVISED INITIALIZATION

  • AdaptiveSafety varies rule order, identifiers, policy length, permissions, and recorded behavior, with 10,939 training and 1,000 test examples spanning policies of 1–100 rules.Structural changes preserve verdict meaning; policy counterfactuals hold the interaction fixed, while behavioral counterfactuals hold the policy fixed.Source: §3.1 · pp. 4–5

3.2 STRUCTURED REWARDS AND GROUP-RELATIVE OPTIMIZATION

  • SafePO scores verdict completeness, rule-set overlap, ordering, and response validity, then normalizes rewards among responses sampled for the same input.The reward does not independently verify the factual correctness of the analysis.Source: §3.2 · pp. 5–6

3.3 VALUE-GUIDED REGION BALANCING

  • An independent value model predicts final task reward from response prefixes and redistributes token weights while preserving fixed analysis and verdict budgets.The objective allocates one-third of the total weight to analysis and two-thirds to verdict; this is not a guaranteed gradient-norm ratio.Source: §3.3 · p. 6

4 EXPERIMENTS

AdaGuard performs strongly on AdaptiveSafety and supports rule attribution on DynaBench, though DynaGuard-8B retains stronger aggregate binary performance there. Results vary by model scale, policy-length group, and training stage.

AdaptiveSafety

  • 89.50% accuracy and 88.67% F1 make AdaGuard-8B the leading local model on AdaptiveSafety, with 77.10% complete-set accuracy and 74.05% rule micro-F1.The 4B model reaches 89.30% accuracy and 88.53% F1; AdaGuard-8B's accuracy and F1 exceed the strongest non-AdaGuard baseline by 22.20 and 21.64 percentage points.Source: §4 · p. 7, Table 2 · p. 8

Evaluation beyond AdaptiveSafety

  • 76.80% accuracy and 76.05% F1 put AdaGuard-8B above Qwen3Guard-Gen and YuFeng-XGuard on DynaBench, while DynaGuard-8B scores 80.11% and 77.31%.AdaGuard-8B reaches 70.72% exact match for rule attribution; DynaGuard-8B has lower recall, 68.91% versus 74.91%.Source: §4 · p. 8

Policy length and model size

  • With 51–100 rules, AdaGuard-8B reaches 88.40% accuracy and 77.20% exact match, but exact match is not monotonic across policy-length groups.The groups differ in content and label composition, so the comparisons are not controlled policy changes.Source: §4 · p. 8, Figure 4 · p. 9

Effect of the training stage

  • At 8B, the SafePO checkpoint improves accuracy, binary F1, and rule exact match on both datasets, but changes at smaller sizes are mixed.On DynaBench, 8B exact match rises from 67.40% to 70.72%; the stage comparison is descriptive because the historical SFT evaluator used a different input-budget calculation.Source: §4 · p. 8, Figure 5 · p. 9

5 RELATED WORK

Prior work supports flexible policy specification and decision interfaces, while AdaGuard focuses on identifying the complete set of rules violated by agent behavior under a supplied policy.

Related work

  • YuFeng-XGuard and DynaGuard address dynamic enforcement or user-defined policies, with a primary focus on conversational compliance and content safety (Lin et al., 2026; Hoover et al., 2026).Source: §5 · p. 9
  • Jev and Laya use typed decision interfaces that let developers express policy checks as questions with probabilistic outputs (Jev Model Guide, 2026; Convai Innovations, 2026).Source: §5 · p. 9
  • AdaGuard targets complete rule identification for agent behavior, using policy and behavioral counterfactuals in AdaptiveSafety and structured rewards with value-guided token weighting in SafePO.Source: §5 · p. 9

6 CONCLUSION

The conclusion presents AdaptiveSafety, SafePO, and AdaGuard for assessing agent behavior under user-defined policies. API-based comparisons extend the evaluation, with results interpreted in light of differing interfaces and inference budgets.

6 CONCLUSION

  • The conclusion says AdaGuard supports detection and rule identification on AdaptiveSafety and both tasks on DynaBench, while noting a remaining DynaBench gap and unverified explanation quality.It calls for controlled ablations and broader deployment studies, and recommends human review for consequential decisions.Source: §6 · p. 9

A.1 API-BASED COMPARISONS

  • 7.10 and 5.30 percentage points are AdaGuard-8B’s AdaptiveSafety accuracy and binary F1 margins over Jev; on DynaBench, Jev leads accuracy and GPT-5.2 leads F1.The comparisons use the same examples but different model interfaces and inference budgets.Source: Table A1 · p. 12, Table A2 · p. 12, §A.1 · p. 12

A.2 MODEL INTERFACES AND INFERENCE SETTINGS

  • The evaluation files call the 1,000-example AdaptiveSafety test set CUA_exea_policy; the policy-free CUA-Exec evaluation is excluded.Historical SFT records call this split validation, but checkpoint selection uses development data drawn separately from the training pool.Source: §A.2 · p. 13
  • Qwen3Guard-Gen and YuFeng-XGuard-Reason produce binary judgments rather than policy-local rule sets, while AdaGuard, GPT-5.2, and Gemini use an ordered label sequence.DynaGuard retains its PASS/FAIL interface; the evaluation does not infer an overall risk probability from per-rule probabilities.Source: §A.2 · p. 13
  • Laya truncation affects at least one question in every AdaptiveSafety example for both checkpoints and all 543 DynaBench Laya examples; these results are not full-context capability estimates.The passage reports truncation in 454 of 543 DynaBench examples for Laya-Typed-Decisions as well.Source: §A.2 · p. 13
  • Server-side truncation is unknown for the closed-source generative models, and Jev’s internal context handling is also unknown.Both closed-source models use a 2,048-token output budget; Jev resolves to typesafe/jev-1.13-20260917.Source: §A.2 · p. 13

A.3 COVERAGE AND METRIC DEFINITIONS

The appendix specifies how invalid responses affect binary and rule metrics, reports protocol-dependent failures, and defines the coverage and macro metrics shown in Table A3.

A.3 COVERAGE AND METRIC DEFINITIONS

  • Invalid responses remain in the evaluation: they count as false positives on compliant examples and false negatives on violating examples, while contributing empty predicted sets to rule micro metrics.Two valid empty sets receive exact-match credit; rule identifiers are compared within each example’s own policy.Source: §A.3 · p. 13
  • GPT-5.2 has two invalid AdaptiveSafety outputs, Gemini has 103, and the local YuFeng run has 139 on AdaptiveSafety and 36 on DynaBench.These protocol-dependent failures are included in the main scores, so coverage matters when comparing models.Source: §A.3 · p. 13
  • Table A3 reports invalid counts, detected truncation, and macro metrics; M-P, M-R, and M-F1 denote binary macro averages.Err. counts invalid or failed responses; Trunc. is the fraction with detected input truncation, and a dash means unknown.Source: Table A3 · p. 14

A.4 SUPERVISED AND REINFORCEMENT-LEARNED CHECKPOINTS

Saved SFT and current SafePO checkpoints show gains for some 8B metrics but declines for other sizes and metrics. Because the evaluations differ, the comparison is descriptive rather than a controlled estimate of SafePO’s effect.

A.4 SUPERVISED AND REINFORCEMENT-LEARNED CHECKPOINTS

  • For 8B, AdaptiveSafety accuracy rises from 88.40% to 89.50% and exact match from 76.20% to 77.10%; DynaBench accuracy rises from 74.77% to 76.80% and exact match from 67.40% to 70.72%.Table A4 labels these historical SFT and current SafePO checkpoint comparisons as descriptive, not controlled component ablations.Source: §A.4 · p. 14, Table A4 · p. 14
  • Changes are not uniformly positive: 4B AdaptiveSafety accuracy and exact match decline, as do DynaBench binary F1 for 4B and DynaBench F1 for 0.6B.For 4B on DynaBench, exact match increases from 63.35% to 63.72% despite the binary F1 decrease from 73.15% to 71.82%.Source: §A.4 · p. 14
  • Historical SFT evaluations used different preprocessing and were not rerun under the current evaluator, so the checkpoint comparison is descriptive, not a controlled estimate of SafePO’s effect.No component ablations isolate the reward terms, value modulation, or region budgets.Source: §A.4 · p. 14, Table A4 · p. 14

A.5 ASSESSMENT SCOPE

AdaptiveSafety reports results separately for user-only requests and recorded trajectories, while the interpretation section cautions that evaluations are single runs and explanations are not independently verified.

A.5 ASSESSMENT SCOPE

  • AdaGuard-4B reaches 91.45% accuracy on user-only requests and 89.01% on trajectories; AdaGuard-8B reaches 90.60% and 89.35%, respectively.The combined result covers both assessment scopes, and 4B leads request-level exact match while 8B leads trajectory-level exact match.Source: §A.5 · pp. 14–15, Table A5 · p. 15

A.6 INTERPRETATION AND REMAINING EVIDENCE

  • All evaluations are single runs, with no significance claims or estimates of training-seed variability; generated explanations are not independently checked for factual correctness.The passage also notes differences in policy-length composition, model interfaces, and context limits.Source: §A.6 · p. 15
  • Controlled component ablations and matched-protocol SFT comparisons remain necessary to attribute gains to individual SafePO mechanisms.The reported analysis measures verdict accuracy and rule identification.Source: §A.6 · p. 15

A.7 REPRODUCIBILITY RESOURCES

The reproducibility section points to the repository and evaluation details; the value-learning appendix specifies how value predictions modulate token weights and when that modulation falls back to fixed weighting.

A.7 REPRODUCIBILITY RESOURCES

  • The repository and evaluation interfaces and inference settings are identified as reproducibility resources.The paper points to Appendix A for evaluation interfaces and inference settings.Source: §A.7 · p. 15

B VALUE LEARNING AND MODULATION DETAILS

C SUPERVISED TRAINING OBJECTIVE

The supervised objective uses full-parameter training, with token weights distinguishing the complete label span from other response tokens and prompt or padding positions.

C SUPERVISED TRAINING OBJECTIVE

Loading 2609.34241v1…