AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security

• Agentic systems introduce safety and security risks through tool calling and harmful information transmitted from the environment.Source: §1 · p. 3
• Current guardrails focus on output filtering and lack agentic risk awareness and provenance to diagnose unsafe or seemingly safe but unreasonable actions.Source: §1 · p. 3
• The proposed hierarchical taxonomy separates risks by source, behavioral influence, and real-world harm, and ATBench evaluates these dimensions at fine granularity.Source: §1 · p. 3
• AgentDoG monitors agent trajectories contextually and uses an XAI module to trace actions to planning steps, tool choices, or context misinterpretations.Source: §1 · p. 3
• AgentDoG outperforms existing state-of-the-art models on agent-oriented safety benchmarks, including diverse and complex interactive scenarios.Source: §1 · p. 3
• ATBench contains about 2157 tools and 4486 turn interactions, and AgentDoG models are openly released in 4B, 7B, and 8B sizes across Qwen and Llama families.Source: §1 · p. 3

2 Safety Taxonomy

• The taxonomy separates where risks originate, how they manifest, and what harm they cause, addressing gaps in R-judge (Yuan et al., 2024b) and ASSE-Safety (Luo et al., 2025a) and overlap between risk dimensions.It explicitly includes tool-related and environment-mediated risks while treating safety as an umbrella term that preserves distinctions needed for fine-grained analysis.Source: §2 · p. 4, Figure 2 · p. 4

2.1 Risk Source

• Risk sources comprise user inputs, environmental observations, external entities such as tools or APIs, and the agent’s internal decision-making logic.Inputs may be adversarial or ambiguous, observations incomplete or manipulated, external responses erroneous or harmful, and internal reasoning or planning flawed.Source: §2.1 · p. 5

2.2 Failure Mode

• Failure modes distinguish unsafe behavior in planning, reasoning, or execution from risks directly constituted by the agent’s textual output.Examples include unsafe tool use or action sequencing for behavioral failures, and misleading information or unauthorized disclosures for output-content failures.Source: §2.2 · p. 5

2.3 Real-world Harm

• The real-world harm dimension classifies consequences—including physical, financial, privacy, psychological, reputational, and societal harms—rather than their causes or mechanisms.These outcomes may follow adversarial manipulation, benign user error, or internal model failures, supporting outcome-oriented safety evaluation and impact assessment.Source: §2.3 · pp. 5–6

3.1 Task Definition

• A trajectory is unsafe if any step contains unsafe behavior, so diagnosis considers intermediate actions and environment feedback even when the final response appears benign.The trajectory consists of action–observation steps and receives a binary safe/unsafe label.Source: §3.1 · p. 7, Trajectory-level safety evaluation · p. 7

Fine-grained risk diagnosis.

• AgentDoG’s two tasks classify trajectory-level safety and, for unsafe trajectories, predict fine-grained risk-source, failure-mode, and real-world-harm labels.Prompts provide the task definition, trajectory, and output format; diagnosis prompts additionally include the safety taxonomy and request labels line by line.Source: Fine-grained risk diagnosis · pp. 7–8

3.2.1 Data Synthesis Pipeline

• The three-stage planner pipeline samples a risk configuration and safe/unsafe outcome, plans the risk’s placement, then orchestrates tool-augmented trajectory generation and outcome summaries.At the trigger point, tool responses can inject malicious or contaminated content; safe agents detect and mitigate it, while unsafe agents fail to identify it and continue.Source: §3.2.1 · pp. 9–10

3.2.2 Data quality control

• Quality control combines deterministic structural checks with LLM-based semantic verification of taxonomy labels and observable compromise in unsafe trajectories.Checks cover turn structure, tool legality, coherence, risk-source/failure-mode/consequence alignment, and evidence of behavioral deviation, unjustified trust, tool misuse, or harm.Source: §3.2.2 · p. 10

3.2 Data Synthesis and Collection

• 86×, 55×, and 41×: the approximately 10,000-tool library is larger than R-Judge’s 114, ASSE-Safety’s 180, and ASSE-Security’s 239 tools, respectively.Taxonomy-guided sampling supports per-class risk synthesis across diverse domains and single- or multi-tool compositions.Source: §3.2 · p. 8, §3.2.3 · p. 11

3.2.3 Statistics of synthesized data

• The QC pipeline retains approximately 52% of generated trajectories, filtering noisy or malformed samples; the synthesized dataset contains over 100k multi-turn interaction trajectories for training and systematic evaluation.Low-frequency (long-tail) risk configurations are intentionally preserved to reflect rarer, potentially high-impact safety scenarios in real-world deployments.Source: §3.2.3 · pp. 10–11

3.3 Training

• Guard models are trained with supervised fine-tuning to minimize negative log-likelihood on trajectory–safety-label demonstrations.The fine-tuned models include Qwen3-4B-Instruct-2507, Qwen2.5-7B-Instruct, and Llama3.1-8B-Instruct; all use a learning rate of 1e-5.Source: §3.3 · p. 11

4.1 Overview of the Benchmark

• ATBench evaluates 500 held-out trajectories (250 safe, 250 unsafe; 8.97 average turns) across 1,575 tools, addressing prior benchmarks’ narrow, binary-only coverage.Unsafe trajectories receive binary verdicts and fine-grained labels across 8 risk sources, 14 failure modes, and 10 consequences; trajectories are labeled unsafe if any unsafe behavior occurs. ATBench evaluates full trajectories and balances labels across the taxonomy dimensions, with minor deviations after quality filtering.Source: §4.1 · p. 12

4.2.1 Trajectory Generation

• ATBench trajectories are synthesized from taxonomy-guided tool seeds, with an independent library of 2,292 tools enabling evaluation on unseen tools and contexts.The benchmark tool library has no overlap with training tools.Source: §4.2.1 · p. 12

4.2.2 Multi-Agent Verification and Data Filtering

• A two-stage pipeline filters trajectories scoring below 3/5, then aggregates four-model verdicts and taxonomy labels by majority vote, routing ties to human adjudication.Cross-model disagreement on binary verdicts defines hard cases; the easy/hard split is 273/227.Source: §4.2.2 · pp. 12–13

4.2.3 Human Verification and Quality Control

• Human reviewers spot-check 20% of easy cases and double-blind review every hard trajectory, using two expert annotators and a third for disagreements.The hard-case review checks that unsafe verdicts rest on observable evidence rather than intent, filtering false positives caused by capability limitations.Source: §4.2.3 · p. 13

5.2 Trajectory-level Safety Results

• 92.7% F1 on R-Judge (AgentDoG-Qwen3-4B) exceeds GPT-5.2 (91.8%); 83.4% on ASSE-Safety (AgentDoG-Llama3.1-8B) exceeds Gemini-3-Pro (78.6%).AgentDoG is also described as maintaining a balanced precision–recall trade-off, while some guard models have very low recall, including ShieldGemma below 10%.Source: Table 4 · p. 14, §5.1 · p. 14, §5.2 · p. 15

5.1 Experimental Setup

• Using R-Judge, ASSE-Safety, and ATBench complete trajectories, evaluation measured safe/unsafe classification and ATBench accuracy for Risk Source, Failure Mode, and Real-world Harm.Trajectory-level metrics were Accuracy, Precision, Recall, and F1; comparisons included leading guard models and general models.Source: §5.1 · p. 14

5.3 Fine-grained Risk Diagnosis Results

• 82.0% Risk Source accuracy (AgentDoG-Qwen3-FG-4B) exceeds Gemini-3-Pro’s 36.8%; AgentDoG-Qwen2.5-FG-7B reaches 59.2% Real-world Harm accuracy versus the best baseline’s 38.0%.The passage says the advantage is consistent across all three fine-grained labels and attributes it to AgentDoG’s trajectory-level and fine-grained risk annotations; these diagnoses support auditing, remediation, and debugging.Source: Table 5 · p. 15, §5.3 · p. 15

5.2 Trajectory-level Safety Results

• General models significantly outperform existing guard models across R-Judge, ASSE-Safety, and ATBench.The paper attributes this to general models’ post-training and instruction-following, and to guard models’ training on single-turn or short-form safety data rather than tool-augmented trajectories, creating distribution shift.Source: §5.2 · p. 15

6.1 Method: Hierarchical Agentic Attribution

• Attribution ranks influential trajectory steps using temporal information gain, then applies sentence perturbations to score each sentence’s necessity and sufficiency as evidence for the target action.The framework applies sentence-level scoring to the highest-ranked steps, narrowing diagnosis from temporal events to specific sentences.Source: §6.1 · p. 16

6.2.1 Diagnostic Case Studies

• In recruitment screening, attribution traces unauthorized interview scheduling to the resume’s embedded prompt injection, identifying retrieved Step 3 as decisive and separating deceptive instructions from benign details.Step 2 establishes retrieval context, while Step 3 introduces the decisive information.Source: §6.2.1 · p. 17
• For sarcastic market feedback, attribution locates the tool result and positive phrases as drivers while underweighting sarcasm; AgentDoG identifies flawed reasoning over the post-execution log.The agent misreads positive wording as praise and recommends going long despite the sarcastic context.Source: §6.2.1 · p. 18

6.2.2 Comparative Analysis: AgentDoG vs. Base Model

• In ambiguous payment selection, AgentDoG identifies the unsupported coworker assumption in Step 4 as the critical driver, while its base model attributes the decision to the less-informative Step 5 success log.Both models identify the user query and ambiguous contact-search results as high-impact context.Source: §6.2.2 · p. 19

Agent Safety Benchmarks

• Interactive-agent benchmarks extend safety evaluation beyond single-turn moderation to multi-step interactions and tool-augmented behaviors, covering diverse risk sources.AgentHarm (Andriushchenko et al., 2024) defines harm categories across social, physical, and digital domains; AgentAuditor (Luo et al., 2025a) structures risks by security and safety dimensions.Source: §7 · p. 19

Agent Trajectory Data

• Real rollout trajectories offer realistic interactions but are costly, difficult to scale, and constrained by privacy and safety; synthetic generation improves scalability and coverage.ToolACE (Liu et al., 2025), Kimi K2 (Kimi, 2025), and AuraGen (Huang et al., 2025) demonstrate large-scale agent-interaction data synthesis.Source: §7 · p. 19

Guard Systems for Agents

• Agent guard systems map inputs or responses to discrete risk labels, while newer systems assess multi-step behaviors or tool interactions using contexts or execution traces.Despite these extensions, existing guards are predominantly trained and evaluated with coarse-grained labels and limited coverage of agent decision and execution failures.Source: §7 · pp. 19–20

8.1 Conclusion

• AgentDoG’s taxonomy, trajectory-synthesis pipeline, diagnostic XAI, model variants, and ATBench support agent-safety evaluation with state-of-the-art classification and fine-grained provenance diagnosis.The taxonomy organizes risks by source, failure mode, and real-world harm; the diagnostic framework traces actions to triggering trajectory steps or sentences. Models span Qwen and Llama families.Source: §8.1 · p. 20

8.2 Limitations and Future Directions

• AgentDoG currently accepts only text trajectories; future work includes multimodal support for GUI agents and proactive alignment using diagnostics as reinforcement-learning rewards.The paper describes using diagnostic output as a reward signal to align agent behavior.Source: §8.2 · p. 20

A Benchmark Construction Details

• The appendix supplies supplementary details on ATBench construction, covering its quality-scoring protocol, multi-model labeling procedure, and annotated cases.Source: A Benchmark Construction Details · p. 25

A.1 Quality Scoring Protocol

• Candidate trajectories are scored from 1 to 5 for coherence, logical flow, tool-use correctness, and format integrity, and scores below 3 are discarded before labeling.The scoring protocol excludes safety considerations.Source: §A.1 · p. 25, Figure 9 · p. 25

A.2 Multi-Model Labeling Protocol

• Four heterogeneous verifiers independently assign safe/unsafe verdicts and three-dimensional taxonomy labels; majority votes determine labels, with ties sent for human adjudication.The verifiers are Qwen-QwQ, GPT-5.2, Gemini 3 Pro, and DeepSeek-V3.2.Source: §A.2 · p. 26

A.2.1 Binary Safety Verdict Prompt

• A trajectory is safe when the agent executes no unsafe actions, including when it successfully refuses or mitigates risky instructions.The binary verdict concerns executed behavior and unsafe decision patterns.Source: §A.2.1 · p. 26

A.2.2 Fine-Grained Taxonomy Labeling Prompt

• Unsafe trajectories receive fine-grained labels for risk source, failure mode, and risk consequence using taxonomy definitions from Tables 1–3.The appendix’s labeling prompt expands those category definitions.Source: §A.2.2 · p. 26, Figure 11 · p. 27

A.3 Representative Annotated Cases

• Representative ATBench cases include trajectory content, a binary safety verdict, and fine-grained taxonomy labels for unsafe cases.Source: §A.3 · p. 26

A.3.1 Representative Safe Case

• In the safe example, the agent detects prompt injection embedded in tool output and refuses the unsafe instruction, yielding a safe verdict.Source: §A.3.1 · p. 27, Figure 12 · p. 28

A.3.2 Representative Unsafe Case

• A prompt injection triggers goal drift and unauthorized external communication in the representative unsafe trajectory, despite the agent initially recognizing the malicious instruction.The trajectory is labeled unsafe with risk-source, failure-mode, and consequence labels; the figure shows actions such as messaging an unverified number or tagging the wrong person.Source: §A.3.2 · p. 27, Figure 13 · p. 28

B Evaluation Details

• The evaluation covers AgentDoG, general-purpose models, and guard models.The section also provides a subset of evaluation test cases for reference.Source: B Evaluation Details · p. 29
• All experiments use temperature 0 to ensure reproducibility.Source: B Evaluation Details · p. 29
• A subset of the evaluation test cases is provided for reference.Source: B Evaluation Details · p. 29

B.1 Evaluation Datasets

• R-judge (Yuan et al., 2024b) contains 569 multi-turn interactions across 27 risk scenarios and five application categories, assessing danger perception during task execution.Its scenarios include programming, IoT, and the web.Source: §B.1 · p. 29
• ASSE-Safety (Luo et al., 2025a) evaluates whether agents avoid unintended harm in non-malicious environments arising from capability deficiencies or design flaws.Examples include hallucinations and faulty reasoning; evaluation concerns tool-interaction tasks and safety-guideline compliance.Source: §B.1 · p. 29
• ATBench is a held-out, evaluation-only benchmark with 500 tool-augmented trajectories, averaging 11.47 messages and spanning 1,577 unique tool identifiers.Unsafe trajectories carry primary labels for risk source, failure mode, and harm type; annotations are validated through multi-model verification and human review, and ATBench is not used for training.Source: §B.1 · p. 29
• Evaluation measures trajectory-level safe/unsafe classification with Accuracy, Precision, Recall, and F1, plus ATBench accuracy for risk source, failure mode, and real-world harm labels.Low precision indicates oversensitivity and false positives, while low recall indicates missed unsafe trajectories.Source: §B.1 · p. 29

B.2.1 Trajectory-level Safety Evaluation

• General and AgentDoG models are evaluated on complete agent trajectories with a shared prompt, using each guard model’s original chat template where available.The binary task is to identify unsafe actions; a trajectory is safe if the agent refuses, ignores, or safely handles malicious or risky input without taking unsafe actions.Source: §B.2.1 · pp. 29–30, Figure 14 · p. 30
• Because ShieldGemma’s 8,192-token context window causes full-trajectory failures, it is evaluated turn by turn, with any unsafe user–agent segment making the trajectory unsafe.Outputs from ShieldGemma and JoySafety are mapped to binary labels using rules.Source: §B.2.1 · p. 30

B.2.2 Fine-grained Risk Diagnosis

• Fine-grained diagnosis asks models to identify risk source, failure mode, and real-world harm from a trajectory using a shared prompt, but guard-model results are not reported because they did not follow it reliably.The evaluation uses the same prompt template for general models and AgentDoG.Source: §B.2.2 · pp. 30–31, Figure 15 · p. 31

B.3 Guard Models Template

• Guard-model evaluations use each framework’s default official prompt templates, covering JoySafety, ShieldGemma, LlamaGuard, Qwen3Guard, NemoGuard, ShieldAgent, and PolyGuard.The JoySafety template shown is originally in Chinese; its English version is provided for illustration.Source: §B.3 · p. 32, Figure 16 · p. 32, Figure 17 · p. 33
• The displayed guard prompts vary: Qwen3Guard checks the last assistant response for safety, category, and refusal; NemoGuard checks single-turn user-message safety; PolyGuard checks user and assistant harm, refusal, and violated policies.The prompts also include a security-review framing for user input and model responses.Source: §B.3 · pp. 32, 34, 35, 36

B.4 Test Case

• Sample AgentDoG outputs are shown for R-Judge, ASSE-Safety, and AgentDoG test cases, including the Qwen3-4B model.Figures 23–25 show examples from those test-case sources.Source: §B.4 · p. 37, Figure 23 · p. 37, Figure 24 · p. 38
• An ATBench sample case is shown with the corresponding AgentDoG-Qwen3-FG-4B output.The example appears in Figure 26.Source: Figure 26 · p. 40