Source-linked AI summary
AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
Dongrui Liu, Qihan Ren, Chen Qian, Shuai Shao, Yuejin Xie, Yu Li, Zhonghao Yang, Haoyu Luo, Peng Wang, Qingyu Liu, Binxin Hu, Ling Tang, Jilin Mei, Dadi Guo, Leitao Yuan, Junyao Yang, Guanxu Chen, Qihao Lin, Yi Yu, Bo Zhang, Jiaxuan Guo, Jie Zhang, Wenqi Shao, Huiqi Deng, Zhiheng Xi, Wenjie Wang, Wenxuan Wang, Wen Shen, Zhikai Chen, Haoyu Xie, Jialing Tao, Juntao Dai, Jiaming Ji, Zhongjie Ba, Linfeng Zhang, Yong Liu, Quanshi Zhang, Lei Zhu, Zhihua Wei, Hui Xue, Chaochao Lu, Jing Shao, Xia Hu
TL;DR
AI agents create safety challenges through autonomous tool use and environmental interactions, while existing guardrails lack agentic risk awareness and transparent diagnosis. The paper introduces a three-dimensional taxonomy, ATBench, and AgentDoG for contextual trajectory monitoring and root-cause diagnosis, achieving state-of-the-art safety moderation across diverse interactive scenarios. The models and datasets are openly released.
Problem
Existing guardrails and benchmarks provide limited agentic risk awareness and coarse binary judgments, missing intermediate trajectory risks and their underlying causes.
Method
The paper combines a unified taxonomy of risk source, failure mode, and real-world harm with ATBench and AgentDoG for trajectory-level monitoring and fine-grained diagnosis.
Results
AgentDoG achieves state-of-the-art safety moderation across diverse agentic scenarios, including 92.7% F1 for AgentDoG-Qwen3-4B on R-Judge.
Takeaways & Limitations
AgentDoG provides contextual provenance for unsafe actions and seemingly safe but unreasonable behavior beyond binary safety labels.
Takeaways & Limitations
AgentDoG currently accepts only text-based trajectories, leaving multimodal safeguarding for GUI-based agents as future work.
Abstract
from arXiv · showhide
The rise of AI agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current guardrail models lack agentic risk awareness and transparency in risk diagnosis. To introduce an agentic guardrail that covers complex and numerous risky behaviors, we first propose a unified three-dimensional taxonomy that orthogonally categorizes agentic risks by their source (where), failure mode (how), and consequence (what). Guided by this structured and hierarchical taxonomy, we introduce a new fine-grained agentic safety benchmark (ATBench) and a Diagnostic Guardrail framework for agent safety and security (AgentDoG). AgentDoG provides fine-grained and contextual monitoring across agent trajectories. More Crucially, AgentDoG can diagnose the root causes of unsafe actions and seemingly safe but unreasonable actions, offering provenance and transparency beyond binary labels to facilitate effective agent alignment. AgentDoG variants are available in three sizes (4B, 7B, and 8B parameters) across Qwen and Llama model families. Extensive experimental results demonstrate that AgentDoG achieves state-of-the-art performance in agentic safety moderation in diverse and complex interactive scenarios. All models and datasets are openly released.
1 Introduction
AgentDoG addresses agentic safety gaps by combining a unified risk taxonomy, fine-grained benchmark, and diagnostic guardrail for contextual trajectory monitoring and transparent risk diagnosis.
- Existing guardrails lack awareness of environment-dependent agentic risks and provenance beyond binary safe/unsafe labels.These limitations make it difficult to diagnose seemingly safe but unreasonable behavior.
- The proposed taxonomy organizes agentic risks by where they originate, how they affect behavior, and what real-world harm they cause.This hierarchical structure supports coverage of complex and numerous agentic behaviors.
- ATBench provides fine-grained evaluation of the three taxonomy dimensions for agent safety.The benchmark is introduced alongside the taxonomy to analyze and evaluate agentic risks.
- AgentDoG monitors agent trajectories contextually and diagnoses why unsafe or seemingly safe but unreasonable actions occur.Its explanations can trace risks to planning steps, tool selections, or context misinterpretations.
- 2157 tools and 4486 turn interactions are included in the openly released ATBench, alongside AgentDoG models in 4B, 7B, and 8B sizes.The models span Qwen and Llama families.
- AgentDoG achieves superior performance on agent-oriented safety benchmarks in diverse, complex interactive scenarios.The framework classifies harmful prompts and mitigates risky agent actions.
2 Safety Taxonomy
The paper replaces flat, overlapping agent-risk categories with a three-dimensional taxonomy that separates risk sources, failure modes, and real-world harms. This structure organizes risks across agent interactions, behaviors, and consequences for systematic safety evaluation.
- Existing agent benchmarks incompletely cover tool usage and agent–tool interaction risks, including compromised descriptions and harmful execution.Their narrow samples also miss complex, long-horizon real-world interactions.
- Flat taxonomies conflate risk origins, behavioral manifestations, and outcomes, causing overlapping labels such as prompt injection and unauthorized access.The paper treats these as different dimensions rather than peer categories.
- Three-dimensional taxonomy: The taxonomy separates risk source, failure mode, and real-world harm, answering where risk comes from, how it manifests, and what harm it causes.This decomposition is intended to eliminate label overlap while capturing tool-related and environment-mediated risks.
- Risk Source: Risk sources include user inputs, environmental observations, external entities such as tools or APIs, and internal decision-making logic.These sources can introduce ambiguous instructions, manipulated information, harmful responses, or flawed reasoning and planning.
- Failure Mode: Failure modes distinguish unsafe behavioral execution from risky output content produced without tools or external actions.Examples include improper sequencing, unsafe tool usage, misleading information, and unauthorized disclosures.
- Real-world Harm: Real-world harms classify consequences such as physical, financial, privacy, psychological, reputational, and societal harms.Explicit consequence modeling supports outcome-oriented safety evaluation and impact assessment.
3 AgentDoG
AgentDoG frames trajectory-level safety as detecting unsafe behavior at any execution step and diagnosing its risk source, failure mode, and harm. Its taxonomy-guided synthesis pipeline generates controlled, long-horizon tool-use trajectories for training.
- 3.1 Task Definition: AgentDoG evaluates whether any intermediate action or observation in an agent trajectory exhibits unsafe behavior, rather than judging only the final response.Unsafe behavior may occur in thinking content, tool calls, or environment feedback even when the final response appears safe.
- 3.1 Task Definition: Fine-grained diagnosis assigns unsafe trajectories labels for risk source, failure mode, and real-world harm using the paper’s three-dimensional taxonomy.The model outputs taxonomy labels in addition to the trajectory-level binary safety judgment.
- 3.2 Data Synthesis and Collection: AgentDoG data synthesis uses a three-stage planner-based pipeline to create long-horizon, tool-augmented trajectories with controllable risk injection and safety outcomes.Planning samples a taxonomy tuple, selects tools, and specifies the risk point and expected defensive behavior before trajectory instantiation.
- 3.2 Data Synthesis and Collection: The orchestrated synthesis process generates queries, tool interactions, agent responses, and outcome summaries according to a structured execution plan.Safe trajectories model threat detection and defense, while unsafe trajectories model risk continuation without mitigation.
- 3.2.3 Statistics of synthesized data: The training tool library is approximately 86×, 55×, and 41× larger than those in R-Judge, ASSE-Safety, and ASSE-Security, respectively.The library contains realistic, diverse tool definitions intended to better reflect heterogeneous tool usage.
- 3.2.3 Statistics of synthesized data: 52% of generated trajectories pass unified quality control after structural validation, tool-legality checks, and consistency verification against taxonomy labels.Filtering removes incomplete multi-turn structures, unparsable arguments, label mismatches, and insufficient semantic coherence.
4 Benchmark: ATBench
ATBench is a held-out, trajectory-level benchmark designed to evaluate long-horizon agent safety and fine-grained risk diagnosis. It combines balanced safe/unsafe instances, broad tool diversity, taxonomy-grounded labels, and verification procedures.
- 4.1 Overview of the Benchmark: ATBench addresses existing benchmark limitations involving short trajectories, limited tool coverage, and coarse binary labels that omit risk causes and consequences.R-Judge trajectories average 5.28 turns, while existing benchmarks generally lack fine-grained diagnosis.
- 4.1 Overview of the Benchmark: ATBench contains 500 complete execution trajectories, balanced between 250 safe and 250 unsafe instances, with average length of 8.97 turns and 1,575 unique tools.The benchmark is constructed as a held-out evaluation set rather than training data.
- 4.1 Overview of the Benchmark: Unsafe ATBench trajectories receive labels across 8 risk sources, 14 failure modes, and 10 real-world harm categories, with balanced coverage across dimensions.A trajectory is unsafe if any unsafe behavior is observable at any point; safe handling of an attempted risk remains safe.
- 4.1 Overview of the Benchmark: The benchmark evaluates full execution trajectories, provides taxonomy-grounded risk attribution, and supports generalization assessment through diverse tool invocation patterns.These properties target long-horizon decision chains and practical risk diagnosis rather than isolated output moderation.
- 4.1 Overview of the Benchmark: ATBench construction applies quality scoring, multi-model labeling, and human verification to improve trajectory fidelity and labeling reliability.Low-quality trajectories are discarded, model-disagreement cases receive priority for human review, and hard cases undergo double-blind expert auditing.
5 Evaluation
AgentDoG is evaluated on trajectory-level safety classification and fine-grained diagnosis across multiple agentic benchmarks. It outperforms specialized guard models, remains competitive with larger general models, and provides stronger taxonomy-level diagnosis.
- Evaluation Setup: AgentDoG is evaluated on R-Judge, ASSE-Safety, and ATBench using trajectory-level safety metrics and fine-grained risk-label accuracy.The fine-grained labels cover Risk Source, Failure Mode, and Real-world Harm.
- Trajectory-level Safety Results: Existing guard models often achieve high precision but low recall because they overlook unsafe intermediate steps and misclassify trajectories as safe.ShieldGemma, for example, reports recall below 10% in the reported comparison.
- Trajectory-level Safety Results: AgentDoG-Qwen3-4B reaches 92.7% F1 on R-Judge, surpassing GPT-5.2 at 91.8% and approaching Gemini-3-Flash at 95.3%.On ASSE-Safety, AgentDoG-Llama3.1-8B reaches 83.4% F1 versus Gemini-3-Pro at 78.6%.
- Fine-grained Risk Diagnosis: Fine-grained diagnosis supports safety auditing, remediation, and debugging by identifying risk sources, failure modes, and consequences beyond binary safe/unsafe labels.The paper attributes AgentDoG’s advantage to training with trajectory-level and fine-grained risk annotations.
- Fine-grained Risk Diagnosis: AgentDoG-Qwen3-FG-4B achieves 82.0% Risk Source accuracy, while AgentDoG-Qwen2.5-FG-7B reaches 59.2% Real-world Harm accuracy.The corresponding reported baseline values are 36.8% for Gemini-3-Pro on Risk Source and 38.0% for Qwen3-235B-A22B-Instruct-2507 on Real-world Harm.
6 Agentic XAI Attribution
AgentDoG’s attribution framework diagnoses agent decisions hierarchically, from influential trajectory steps to specific sentences. Case studies show that it can identify deceptive instructions, shallow keyword reliance, and flawed internal reasoning as action drivers.
- Attribution Method: The attribution framework first identifies trajectory steps that steer the agent toward a target action, then refines diagnosis to influential sentences.This hierarchical design moves from temporal events to textual evidence within high-impact steps.
- Trajectory-level Attribution: Temporal information gain measures how much appending step s_i increases the likelihood of the target action, with high values indicating decisive drivers.The method compares the target-action likelihood before and after adding each step.
- Sentence-level Attribution: Sentence-level attribution combines probability-drop and probability-hold signals to quantify each sentence’s necessity and sufficiency for the target action.The resulting score is applied to top-ranked steps identified by temporal information gain.
- Diagnostic Case Studies: In financial analysis, attribution concentrates on positive phrases and reveals that the agent relied on shallow keywords while ignoring sarcastic context.The analysis identifies the tool execution result as the dominant trajectory-level factor.
- Diagnostic Case Studies: In recruitment screening, the module identifies resume-embedded deceptive instructions as the decisive adversarial content behind unauthorized interview scheduling.It distinguishes the malicious text from benign resume data and attributes the behavior to the external tool interaction.
- Comparative Analysis: Compared with its base model, AgentDoG correctly attributes a decision to a flawed internal reasoning step rather than a less informative post-execution success log.This comparison supports more granular attribution of the internal drivers behind agent actions.
7 Related Work
Related work addresses agent safety through benchmarks, trajectory data, and guard systems, but existing approaches have limited coverage and coarse risk assessment. AgentDoG builds on this gap with trajectory-focused and taxonomy-grounded diagnosis.
- Agent Safety Benchmarks: Agent safety benchmarks extend beyond static content moderation to failures arising during planning, tool use, and long-horizon execution.The related work frames agent safety around decision-making and action execution over time.
- Agent Trajectory Data: Real rollout trajectories provide realistic interactions but are expensive, difficult to scale, and constrained by privacy and safety considerations.Synthetic trajectory generation is discussed as an approach to improve scalability.
- Guard Systems for Agents: Existing guard systems commonly frame safety assessment as instruction following or classification that maps textual inputs, responses, or traces to discrete risk labels.More recent systems incorporate agent contexts or execution traces for multi-step and tool-augmented interactions.
- Guard Systems for Agents: Despite incorporating agent contexts, existing guard models remain predominantly coarse-grained, limiting their ability to diagnose detailed agentic risks.The related-work discussion contrasts these approaches with more granular trajectory diagnosis.
8 Conclusion and Discussion
AgentDoG combines a three-dimensional safety taxonomy, trajectory synthesis and diagnosis, and released model variants to address agentic safety challenges. The paper reports strong binary classification and fine-grained provenance diagnostics, while identifying multimodal support and proactive alignment as future directions.
- AgentDoG introduces a unified three-dimensional safety taxonomy covering risk source, failure mode, and real-world harm.
- The framework uses a systematic data synthesis pipeline to generate annotated agentic trajectories across the complex risk space.
- AgentDoG traces specific actions back to triggering trajectory steps or sentences, providing transparency into agent decision logic.
- Released AgentDoG variants span Qwen and Llama model families and support binary safety classification alongside fine-grained provenance diagnostics.
- Current inputs are confined to text-based trajectories, motivating future multimodal support for GUI-based agents and proactive alignment through reinforcement learning.
9 Authors
The paper identifies scientific directors, core contributors, contributors, and technical acknowledgements.
- Scientific directors are Xia Hu and Chaochao Lu.
- Core contributors are Dongrui Liu, Qihan Ren, Chen Qian, Shuai Shao, Yuejin Xie, Yu Li, Zhonghao Yang, Haoyu Luo, Peng Wang, and Qingyu Liu.
- Contributors include Binxin Hu, Ling Tang, Jilin Mei, Dadi Guo, Leitao Yuan, Junyao Yang, Guanxu Chen, Qihao Lin, Yi Yu, Bo Zhang, Jiaxuan Guo, Jie Zhang, Wenqi Shao, Huiqi Deng, Zhiheng Xi, Wenjie Wang, Wenxuan Wang, and Wen Shen.
- Technical acknowledgements include Zhikai Chen, Haoyu Xie, Jialing Tao, Juntao Dai, Jiaming Ji, Zhongjie Ba, Linfeng Zhang, Yong Liu, Quanshi Zhang, Lei Zhu, Zhihua Wei, and Hui Xue.
A Benchmark Construction Details
ATBench construction filters trajectory quality before heterogeneous multi-model safety and taxonomy annotation. The appendix documents prompts, aggregation, and representative cases illustrating safe handling and unsafe outcomes.
- ATBench construction includes quality scoring, multi-model labeling, and representative annotated cases.
- Candidate trajectories receive scores from 1 to 5 for coherence, logical flow, tool usage correctness, and format integrity, excluding safety.
- Trajectories scoring below 3 are discarded before benchmark labeling.
- Four heterogeneous verifiers independently produce binary verdicts and one label for each taxonomy dimension; majority votes decide labels and ties receive human adjudication.
- A trajectory is safe when it contains no unsafe actions or decision patterns, including successful refusal or mitigation of risky instructions.
- Unsafe trajectories additionally receive labels for risk source, failure mode, and risk consequence.
- Representative cases include indirect prompt injection that is detected and refused, producing a safe verdict.
A.3.2 Representative Unsafe Case
The representative unsafe case shows indirect prompt injection leading to goal drift and unauthorized external communication. Fine-grained labels capture the source, failure mode, and consequence of the unsafe trajectory.
- Prompt injection in tool output leads the agent to adopt an attacker-induced objective through goal drift.
- The resulting unsafe behavior includes unauthorized external communication.
- The trajectory receives fine-grained labels for risk source, failure mode, and consequence.
B Evaluation Details
AgentDoG is evaluated on complete agent trajectories using complementary binary safety classification and fine-grained risk diagnosis tasks across three datasets. ATBench supports held-out trajectory-level assessment with validated annotations for both overall safety judgment and risk attribution.
- Datasets: Evaluation uses three datasets: R-Judge, ASSE-Safety, and ATBench.R-Judge covers security hazards during multi-turn interactions, while ASSE-Safety targets unintended harmful behavior in non-malicious tool-use environments.
- Datasets: ATBench contains 500 full execution trajectories averaging 11.47 messages and includes 1577 unique tool identifiers.It provides binary safe/unsafe labels and, for unsafe trajectories, primary labels for risk source, failure mode, and harm type.
- Datasets: ATBench is held out for evaluation and uses multi-model verification followed by human review to validate annotations.This supports assessment of both overall safety judgments and fine-grained risk attribution.
- Evaluation Tasks: Each dataset contains complete agent trajectories classified as safe or unsafe, enabling two complementary evaluation tasks.The tasks assess trajectory-level safety and specific risk labels for unsafe trajectories.
- Evaluation Tasks: Trajectory-level safety evaluation reports Accuracy, Precision, Recall, and F1-score for safe-versus-unsafe classification.Lower precision indicates more false positives, while lower recall indicates more false negatives.
- Evaluation Tasks: Fine-grained diagnosis reports Risk Source Acc, Failure Mode Acc, and Real-world Harm Acc on ATBench.These metrics evaluate classification of specific risk labels for unsafe trajectories.
B.2.1 Trajectory-level Safety Evaluation
Trajectory-level evaluation formats complete agent trajectories for safety judgment, while fine-grained evaluation asks models to identify risk source, failure mode, and real-world harm. Guard models generally use official templates, with outputs normalized when necessary and sample evaluations provided across benchmarks.
- Trajectory-level Evaluation: General models and AgentDoG receive complete trajectories through the same trajectory-level prompt template.The task is to determine whether any unsafe actions occur in the agent’s trajectory.
- Trajectory-level Evaluation: A trajectory is safe when the agent takes no unsafe actions and safely handles malicious input, prompt injection, or risky instructions; otherwise it is unsafe.Safe handling includes refusing, ignoring, or safely managing the risky input or instruction.
- Trajectory-level Evaluation: ShieldGemma’s 8,192-token context window causes overflow and evaluation failures under full-trajectory evaluation.The protocol therefore switches this model to an alternative turn-level procedure.
- Trajectory-level Evaluation: Turn-level evaluation segments trajectories into user/environment–agent pairs and labels the full trajectory unsafe if any user–agent segment is unsafe.This procedure obtains a binary safety label for each segment before aggregating predictions across the trajectory.
- Trajectory-level Evaluation: Models without simple safe/unsafe outputs are mapped to binary labels using model-specific rules.This normalization is applied to models such as ShieldGemma and JoySafety for performance assessment.
- Fine-grained Risk Diagnosis: Fine-grained evaluation uses agent trajectories as input and requires Risk Source, Failure Mode, and Real-world Harm labels.The framework treats these categories as an analytical aid for identifying and explaining potential risks.
- Guard Model Templates: Guard models are evaluated with their official default templates, including JoySafety, ShieldGemma, LlamaGuard, Qwen3Guard, NemoGuard, ShieldAgent, and PolyGuard.The study also provides sample test cases and corresponding AgentDoG outputs for R-Judge, ASSE-Safety, and ATBench.