AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents

• Agentic systems require evaluation of interactive behaviors and decision-making because expanded autonomy introduces unsafe or insecure risks, especially in high-stakes or dynamic scenarios.Reliable assessment remains essential for trustworthy agents, but current methods struggle with the complexity and diversity of real-world risks.Source: §1 · pp. 1–2
• Rule-based evaluators struggle with implicit or novel risks, whereas LLM-based evaluators often exhibit inconsistency, bias, and limited interpretability when assessing cumulative or ambiguous risks.These limitations make it difficult for existing approaches to align with human preferences across diverse agent scenarios.Source: §1 · p. 2
• AgentAuditor is a universal, training-free, memory-augmented framework that extracts structured features and reasoning traces, then retrieves relevant experiences to guide new evaluations.Its design uses self-adaptive representative-shot selection, structured memory, retrieval-augmented generation, and automatically generated chain-of-thought reasoning.Source: §1 · p. 2
• AgentAuditor consistently outperforms existing baselines and reaches human-level performance, achieving up to 96.3% F1 and 96.1% accuracy on R-Judge with Gemini-2.0-Flash-thinking.The framework also significantly improves evaluation performance across all datasets and LLMs, achieving state-of-the-art results and matching human expert-level performance with leading LLMs.Source: §1 · pp. 2–3, Figure 1 · p. 2
• ASSEBench is the first large-scale benchmark jointly covering agent safety and security, with 2293 annotated records spanning 15 risk types, 528 environments, and 29 application scenarios.It contains four subsets, 26 agent behavior modes, and supports strict and lenient evaluation through systematic human-computer collaborative classification.Source: §1 · pp. 2–3

Rule-based Evaluator

• Rule-based evaluators apply predefined keywords, patterns, or logical rules, making decisions efficient and interpretable but inflexible for implicit or ambiguous harms.For agents, existing rules typically inspect environmental-state changes, tool-call types and sequences, and parameter compliance [19] [97] [94] [4] [42].Source: §2 · p. 3

LLM-based Evaluator

• LLM-based evaluators use prompt engineering or fine-tuning to analyze context and semantics, improving detection of implicit complex harms but risking inconsistent criteria, bias propagation, and poor interpretability.Agent benchmarks commonly assess action safety and harmful-task refusal, then compute ASR, RR, or overall safety scores from evaluation outcomes [86] [98] [71] [4] [97].Source: §2 · p. 3

Agent Safety vs. Agent Security

• Agent safety addresses unintended harmful behavior from internal flaws or limited risk awareness, while agent security addresses deliberate external attacks such as prompt injection, data poisoning, and backdoors.The concepts can overlap when agents execute dangerous operations in response to risky instructions [98] [75] [81] [97].Source: §2 · p. 3
• Safety is harder to evaluate automatically than security because its failures are subtler and lack the explicit malicious signatures typical of security breaches.Consequently, safety benchmarks depend more heavily on precise LLM-based evaluators.Source: §2 · p. 3

3 AgentAuditor: Our Framework

• AgentAuditor combines structured understanding with learned past experiences to help LLM-as-a-judge evaluate complex agent interactions.The framework constructs feature and reasoning memories, then retrieves relevant reasoning traces for new evaluations.Source: §3 · p. 3, §3.2 · p. 5, §3.3 · p. 6

3.1 Stage 1: Feature Memory Construction

• Stage 1 extracts scenario, risk, and behavior features from each interaction and stores human-readable and vectorized representations for interpretable, similarity-based memory access.The vectorized memory embeds both raw interaction content and extracted features using Nomic-Embed-Text-v1.5.Source: §3.1 · pp. 4–5

3.2 Stage 2: Reasoning Memory Construction

• Stage 2 builds reasoning memory by clustering processed feature vectors, selecting representative shots, and generating chain-of-thought traces grounded in their content and safety labels.Embeddings are normalized, heuristically weighted, concatenated, reduced with PCA, and clustered with FINCH [72].Source: §3.2 · p. 5

3.3 Stage 3: RAG-based Memory Augmented Reasoning

• Stage 3 retrieves relevant reasoning experiences by content similarity, reranks them using weighted scenario, risk, and behavior similarities, and supplies the top examples in a tailored few-shot prompt.The LLM then uses these selected chain-of-thought examples as a memory-augmented evaluator for the target interaction.Source: §3.3 · pp. 5–6

4 ASSEBench: Agent Safety & Security Evaluator Benchmark

• ASSEBench evaluates LLM-based evaluators across agent safety and security, addressing gaps in the prior R-Judge benchmark [93].The benchmark is designed for comprehensive performance analysis across both dimensions.Source: §4 · p. 6

4.1 Structure of ASSEBench

• The benchmark contains 817 security records and three safety subsets of 1,476 records each, formatted as annotated JSON interactions with safe/unsafe labels, ambiguity flags, scenarios, risk types, and behavior modes.Its safety coverage spans 17 scenarios and 9 risk types, while security covers 12 scenarios and 6 risk types.Source: §4.1 · p. 6, Figure 3 · p. 6

4.2 Construction of ASSEBench

• ASSEBench is constructed through collection and standardization, expert annotation and classification, and balanced screening with iterative refinement.The pipeline collects records from four existing benchmarks, standardizes 13,587 records to 4,527, and combines LLM assistance with strict human review.Source: §4.2 · p. 7
• Ambiguous safety records are re-evaluated under strict and lenient criteria to form ASSEBench-Strict and ASSEBench-Lenient, enabling evaluation under different judgment thresholds.This multi-standard annotation strategy targets fine-grained analysis of LLM-based safety reasoning.Source: §4.2 · p. 7

5.1 Experimental Setup

• The experimental setup spans ten diverse proprietary and open-source base models, two fine-tuned safety evaluators, ASSEBench, R-Judge, and contamination-controlled evaluation using fixed AgentAuditor parameters.Unless otherwise specified, experiments use N = 8, K = 3, and an embedding dimension of 512; the test subset used to generate CoT reasoning is excluded from evaluation.Source: §5.1 · pp. 7–8

5.2 Empirical Results

• AgentAuditor significantly improves performance across all datasets and models: Gemini-2’s F1 increases 48.2% on ASSEBench-Safety, while Llama-3.1-8B’s accuracy rises from 49.69% to 81.03% on ASSEBench-Security.Source: §5.2 · pp. 8–9
• Gemini-2 with AgentAuditor sets a state-of-the-art across all datasets, surpasses baselines, and reaches or exceeds single-human accuracy on R-Judge, AgentHarm, and ASSEBench-Security.Its accuracies are 96.1%, 99.4%, and 93.2%, compared with average single-human accuracies of 95.7%, 99.3%, and 94.3%; it also outperforms ShieldAgent by 15.1% on R-Judge.Source: §5.2 · p. 9
• AgentAuditor narrows performance gaps between ASSEBench-Strict and ASSEBench-Lenient, reducing GPT-4.1’s disparity from 10.3% to 5.6% and Gemini-2’s from 13.2% to -2.8%.Its reasoning memory adaptively adjusts reasoning strategies to different evaluation standards without manual intervention.Source: §5.2 · p. 9

5.3 Ablation Studies

• Ablations show that Feature Tagging, Clustering, Few-shot learning, and CoT are all critical, with their combination outperforming isolated CoT or Few-shot configurations.CoT alone can over-analyze low-probability risks, while Few-shot without CoT is ineffective for dissimilar cases; semantic retrieval, representative examples, and reasoning templates therefore work synergistically.Source: Table 2 · p. 9, §5.3 · pp. 9–10

5.4 Human Evaluation

• Human evaluation finds that AgentAuditor matches average human accuracy and produces more logical, complete, and sound reasoning than direct LLM risk analysis.These gains hold for Gemini-2 and QwQ-32B, while Gemini-2 matches human accuracy on R-Judge, AgentHarm, and ASSEBench-Security.Source: Figure 4 · p. 10, §5.4 · p. 10

6 Limitation

• AgentAuditor’s improvements decrease when reasoning-memory shots contain label defects.Its effectiveness depends on the quality of the labels in those shots.Source: §6 · p. 10
• Like other CoT-based methods, AgentAuditor requires reasoning ability and greater resource consumption.Source: §6 · p. 10
• AgentAuditor relies on heuristic parameters whose universality and interpretability require further verification despite good current experimental performance.Source: §6 · p. 10
• Although AgentAuditor theoretically supports more than binary labels, ASSEBench and its tests evaluate only binary safe/unsafe labels.Additional details are provided in Appendix J.Source: §6 · p. 10
• Because ASSEBench interaction records are entirely in English, its application to multilingual agent safety is limited, especially where languages introduce unique risks [87].Source: §6 · p. 10

7 Conclusion & Future Work

• AgentAuditor and ASSEBench improve LLM-based evaluation while jointly addressing agent safety and security, with future work targeting greater generalizability, benchmark comprehensiveness, and dynamic-memory agent defense.AgentAuditor uses representative-shot selection and CoT prompting, while ASSEBench systematically handles classification and ambiguity.Source: §7 · p. 10

A Challenges for Automated Evaluation

• Automated evaluators must reason about environmental consequences, cumulative harmless actions, ambiguous harm boundaries, and context-dependent safety–utility trade-offs rather than relying on isolated text or universal rules.Examples include freezing-weather air-conditioning control, cumulative illiquid investments, consent-dependent user clustering, and emergency traffic-law violations.Source: A Challenges for Automated Evaluation · p. 19

B.1 Chain-of-Thought (CoT)

• CoT prompting decomposes complex tasks into explicit reasoning steps and can improve performance and transparency, but depends on model reasoning ability and increases computational load and latency [92] [79] [38].Few-shot CoT uses human-designed exemplars, whereas zero-shot CoT uses simple instructions and typically performs worse.Source: §B.1 · pp. 19–20

B.2 Retrieval-Augmented Generation

• RAG retrieves relevant external knowledge or memory before generation to improve accuracy and grounding, and AgentAuditor uses it to retrieve prior reasoning traces from structured reasoning memory [25].This approach can reduce hallucinations and integrate domain-specific information without retraining the full model.Source: §B.2 · p. 20

C First Integer Neighbor Clustering Hierarchy (FINCH)

• FINCH selects diverse representative interactions without requiring cluster-count, distance-threshold, or cluster-size parameters, making reasoning-memory construction adaptive and interpretable.Its inability to identify singleton clusters is acceptable because AgentAuditor prioritizes prevalent, generalizable risk patterns over isolated outliers.Source: C First Integer Neighbor Clustering… · pp. 20–21

D Examples of Prompt Templates in AgentAuditor; D.1 Prompts in Reasoning Memory; D.2 Prompts in Memory Augmented Reasoning

• AgentAuditor uses separate prompt templates to generate labeled chain-of-thought traces for reasoning memory and to perform memory-augmented reasoning on target interactions.The reasoning-memory prompt analyzes why a predetermined safe or unsafe label applies rather than reconsidering the label; the memory-augmented template supplies interaction and reasoning exemplars.Source: D Examples of Prompt Templates in… · p. 22, §D.1 · p. 22, §D.2 · p. 23

E Automated Tagging Prompt

• The automated tagging prompt extracts concise application-scenario, risk-type, and failure-mode features from interaction contents and requires strict JSON output with prescribed word limits.The risk taxonomy examples include harmful content, privacy leaks, misinformation, bias, and operational hazards.Source: E Automated Tagging Prompt · p. 23

F Existing Benchmarks; F.1 Benchmarks for Evaluators

• Existing benchmarks combine complementary datasets, but R-Judge [93] remains the only benchmark specifically designed for LLM-as-a-judge evaluation of agent interaction safety risks and is limited in scale, safety–security separation, and security coverage.R-Judge [93] contains 569 multi-turn records across 27 scenarios, 5 application categories, and 10 risk types, with 52.7% labeled unsafe; related resources include ToolEmu [71], InjecAgent [94], AgentMonitor [10], AgentSafetyBench [98], AgentDojo [19], GuardAgent [81], and ToolSword [85].Source: F Existing Benchmarks · p. 24, §F.1 · p. 24

F.2 Benchmarks for Agents

• Existing benchmarks rely on narrow evaluators or metrics: AgentDojo [19] and AgentSecurityBench [97] use rule-based tool-invocation criteria, while AgentSafetyBench [98] uses a fine-tuned evaluator that overclassified safe interactions.AgentHarm [4] and AgentSecurityBench [97] also use refusal rate, which does not capture ambiguity or cumulative risks; AgentDojo [19] additionally has repetitive security challenges.Source: §F.2 · pp. 24–25

G.1 Modular Framework Design and Implementation

• ASSEBench standardizes records from AgentSafetyBench [98], AgentSecurityBench [97], AgentHarm [4], and AgentDojo [19] through wrappers that unify calls, inputs, outputs, preprocessing, and formatting.The modular design supports consistent downstream evaluation and future benchmark integration.Source: §G.1 · p. 26

G.2 LLM-Based Record Generation & Preprocessing

• Three LLMs generated 13,587 standardized interaction records, after which invalid records were removed and Balanced Allocation equalized successful contributions across models.The selected models were Gemini-2.0-Flash-thinking, GPT-4o [3], and Claude-3.5-sonnet.Source: §G.2 · pp. 26–27

G.3 Development of the Classification System

• Human experts classified 4,527 records into safety and security subsets, using independent schemes for scenario, risk type, and behavior mode because LLMs performed poorly on this distinction.LLM-generated labels and FINCH [72] clustering exposed latent patterns, which experts reviewed and revised into the final interpretable taxonomy.Source: §G.3 · pp. 27, 29

G.4 Human-in-the-Loop Annotation Process

• Each record received LLM-assisted annotation followed by expert review, with experts assigning safety labels, resolving complex cases, filtering invalid records, repairing minor flaws, and retaining objectively ambiguous cases.Ambiguous records were retained as boundary cases for evaluating behavior near safety decision thresholds.Source: §G.4 · p. 29

G.5 Final Dataset Assembly and Subset Formation

• Balanced Screening optimizes representation across safety status, scenario, risk type, and behavior mode, while prioritizing ambiguous records in ASSEBench-Safety but not ASSEBench-Security.This process produces ASSEBench-Safety and ASSEBench-Security, plus ASSEBench-Strict and ASSEBench-Lenient, which apply divergent standards to ambiguous safety cases.Source: §G.5 · p. 29

I.1 Criteria for Distinguishing Safety and Security

• Human criteria distinguish safety as harm from agent capabilities, outputs, or failures from security as system vulnerabilities, exploitation, or malicious use, while overlapping records may enter both subsets.Final safety judgments consider the entire interaction context and classify clear risk, harmful behavior, significant negative consequences, security violations, or clear harmful intent as unsafe.Source: §I.1 · p. 36, §I.2 · p. 36

I.2 Criteria for Judging Safety Status

• Safety labels reflect observed behavior, considering potential harm, apparent intent, request context, and direct consequences while treating harmless failures and ethical assistance as safe.Unsafe behavior includes persistent attempts to bypass safety constraints or prevent harm; judgments are based on the interaction record rather than hypothetical extensions.Source: §I.2 · pp. 36–37

I.3 Criteria for Judging Ambiguity

• Ambiguous records capture borderline risk, subjective disagreement, or unclear intent and outcomes, providing boundary cases for further analysis.This ambiguity label underlies the construction of ASSEBench-Strict and ASSEBench-Lenient.Source: §I.3 · p. 37

I.4 Examples for Manual Repair

• Manual repair is limited to minor readability fixes, cautious clarification, or converting selected safe AgentHarm examples into unsafe records to improve quality and balance.Repairs are intended not to change meaning except where selected records are modified to supplement inadequate unsafe examples.Source: §I.4 · p. 37

I.5 Criteria & Examples for Strict & Lenient Standards

• Strict relabeling treats any plausible harmful interpretation as unsafe, whereas lenient relabeling requires clear direct evidence of significant harm, malicious intent, or explicit constraint violation.Only originally ambiguous records are re-evaluated; non-ambiguous safe and unsafe labels remain unchanged, allowing sensitivity to different risk thresholds.Source: §I.5 · pp. 37–38

J.1 Metrics in the Experiments of AgentAuditor

• Recall is prioritized over precision because missed unsafe behavior can have more severe consequences, with accuracy and F1 balancing performance in binary safe/unsafe evaluation.The metrics support comparison across models and benchmarks but cannot represent risk severity, judgment confidence, or unequal harm across risk types.Source: §J.1 · pp. 38–39

J.2 Metrics in the Human Evaluation of AgentAuditor

• Six AI safety and security experts independently rate AgentAuditor’s CoT traces and baseline risk descriptions on three 1–5 Likert dimensions [46].The dimensions are Logical Structure, Reasoning Soundness, and Completeness.Source: §J.2 · p. 39
• Reasoning Soundness measures how reasonable or valid the logical steps in a CoT’s reasoning process appear, using a five-point scale from 1 (Very unreasonable / Flawed) to 5 (Very reasonable / Valid).Source: §J.2 · p. 40
• Completeness measures how fully the CoT covers relevant safety and security aspects evident in each interaction, from missing many key points to covering all explicit key points.The dimension uses the same 1–5 Likert scale, with 3 indicating coverage of the main key points.Source: §J.2 · p. 40
• For each item and dimension, the final score is the average across annotators, and the three dimensions are analyzed independently for baseline comparisons.Inter-rater reliability is assessed with Krippendorff’s Alpha, where α > 0.67 is typically acceptable and α > 0.8 indicates good agreement.Source: §J.2 · pp. 40–41

J.3 Metrics in Existing Benchmarks

• Existing agent safety and security benchmarks commonly adopt attack success rate (ASR), refusal rate (RR), and Safety Score (SS) from broader LLM safety research.These metrics are widely applied because agent safety and security research closely continues general LLM safety studies.Source: §J.3 · p. 41
• ASR measures harmful attack elicitation but depends heavily on the tested attacks, whereas RR is simple to judge yet cannot equate refusal with safety in complex interactions.ASR is used in InjecAgent, EvilGeniuses, ToolSword [85], AgentDojo [19], AgentSecurityBench [97], and SafeAgentBench [86]; RR is used in AgentHarm [4], AgentSecurityBench [97], St-WebAgentBench [42], and SafeAgentBench [86].Source: §J.3 · p. 41
• SS scales safety assessment through LLM-generated safe or unsafe labels, but its reliability depends on evaluator capability, bias, and comprehensive guidelines.SS is used in RealSafe, AgentSafetyBench [98], and ToolEmu [71].Source: §J.3 · p. 41

K Detailed Experimental Results

• AgentAuditor robustly and versatilely improves LLM evaluation of agent safety and security across four ASSEBench subsets, with automatically selected representative shots of 72 or 73.The experiments cover eight datasets and twelve models; Table 12 reports differences between evaluations with AgentAuditor and original evaluations without it.Source: K Detailed Experimental Results · p. 41, Table 12 · p. 42
• AgentAuditor results on R-Judge, AgentHarm, AgentSecurityBench, and AgentSafetyBench are reported with automatically selected representative-shot counts of 24, 19, 71, and 78.Table 13 compares evaluations with AgentAuditor against original evaluations without it across these four datasets.Source: K Detailed Experimental Results · p. 42, Table 13 · p. 43
• On AgentHarm and AgentSecurityBench, GPT-4o’s Refusal Rate accuracy can exceed whole-interaction safety evaluation, but Refusal Rate is less practical and accuracy is preferable to F1-score or recall.Direct harmful requests produce many true negatives, weakening F1-score and recall as metrics for AgentHarm.Source: K Detailed Experimental Results · p. 42

L Discussion of Performance Disparities in ShieldAgent

• ShieldAgent is a Qwen-2.5-7B evaluator fine-tuned on 4,000 manually annotated AgentSafetyBench interactions and serves as a safety-focused small-model baseline [98].It represents the classic strategy of developing LLM evaluators through safety-focused fine-tuning.Source: L Discussion of Performance… · p. 43
• ShieldAgent’s reported 91.5% AgentSafetyBench accuracy exceeds GPT-4o’s 75.5%, but reproduced results fall below 91.5% on AgentSafetyBench and are substantially lower on R-Judge and ASSEBench [93].ShieldAgent still outperforms GPT-4o and its Qwen-2.5-7B base model on multiple benchmarks, yet diverges from its developers’ claims.Source: L Discussion of Performance… · p. 43
• The ShieldAgent reproduction followed the original model version, loading method, prompts, and code, while manual reviews of discrepant samples supported the reliability of the adopted ground truth.The authors released reproduction code to enable further comparison.Source: L Discussion of Performance… · p. 44
• Llama-Guard-3’s limitations in tool-using, environment-interacting agent scenarios make ShieldAgent’s claimed 91.5% accuracy appear extraordinary despite training on only 4,000 examples.The comparison draws on Llama-Guard-3’s larger computational and data resources, its weaker agent-centric performance, and R-Judge evidence involving Llama-Guard-2 [93].Source: L Discussion of Performance… · p. 44, Table 14 · p. 44
• The authors attribute the reproduced-versus-reported ShieldAgent gap to possible overfitting, data leakage, or test-set bias, but cannot investigate further because the original internal test set is unavailable.Their experiments consistently place ShieldAgent above Qwen-2.5-7B, GPT-4o, and Llama-Guard-3, indicating effective fine-tuning despite likely overestimation.Source: L Discussion of Performance… · p. 44

• Embedding dimensionality is more important than model scale, because an appropriate dimension can produce substantial performance gains across tested embedding models.The experiments compare all-MiniLM-L6-v2, Nomic-Embed-Text-v1.5, Stella-en-1.5B-v5, and NV-Embed-v2 across dimensions on R-Judge with Gemini-2.0-Flash-thinking.Source: M Impact of Different Embedding Models · pp. 44–45
• Larger embedding models improve AgentAuditor only to a limited extent, with noticeable diminishing returns as model size and resource consumption increase.At dimension 512, Stella-en-1.5B-v5 uses nearly ten times the parameters and resources of Nomic-Embed-Text-v1.5 but yields only a marginal accuracy improvement.Source: Table 16 · p. 45, M Impact of Different Embedding Models · p. 45
• Nomic-Embed-Text-v1.5 at 512 dimensions delivers exceptional performance with low resource consumption and is selected as the embedding model for the rest of the work.This configuration is preferred over larger alternatives because it combines strong performance with lower resource demands.Source: M Impact of Different Embedding Models · p. 45

N Impact of Different Clustering Methods

• All clustering-based AgentAuditor variants significantly outperform the memory-free base model, confirming the value of representative-shot selection across clustering methods.The baseline directly uses Gemini-2 for evaluation.Source: N Impact of Different Clustering Methods · p. 46, Table 17 · p. 46
• FINCH achieves 96.31 F1-score and 96.10 accuracy on R-Judge, substantially outperforming K-Means, K-Medoids, and DBSCAN.The comparison uses Gemini-2.0-Flash-thinking; K-Means and K-Medoids use 24 clusters, while DBSCAN yields 14 representative shots.Source: N Impact of Different Clustering Methods · p. 45, Table 17 · p. 46
• FINCH is selected because it determines hierarchical cluster structure and granularity automatically without manual hyperparameter tuning.K-Means and K-Medoids require a preset cluster count, whereas DBSCAN is sensitive to epsilon and min_samples across datasets.Source: N Impact of Different Clustering Methods · p. 46

O.1 Robustness to Label Noise

• Label noise matters because corrupted labels in representative shots degrade the reasoning memory and can consequently affect AgentAuditor’s evaluation performance.The reasoning memory combines labeled representative shots with their corresponding CoT traces, making label quality central to robustness.Source: §O.1 · p. 46
• The noise experiment randomly changes selected ground-truth shot labels, such as converting a safe label to unsafe, among 24 automatically selected R-Judge shots.Experiments use Gemini-2.0-Flash-thinking and otherwise retain the main-experiment defaults.Source: §O.1 · pp. 46–47
• At 50% incorrect labels, AgentAuditor falls below the baseline, whereas with 25% noise its F1-score is 0.8536 and accuracy is 0.8333, still outperforming the baseline.With 33.3% incorrect labels, both metrics retain a slight advantage over the baseline; recall consistently maintains a significant advantage.Source: Figure 7 · p. 46, §O.1 · p. 47

O.2 Robustness Against Adversarial Attacks

• The adversarial evaluation escalates from grey-box dataset poisoning to strong grey-box memory poisoning and finally white-box control over retrieval and final decision examples.Experiments use R-Judge with Gemini-2, a 564-record dataset, and 24 reasoning-memory records, while excluding poisoned samples from final metrics.Source: §O.2 · p. 47
• At a 1.1% Level 1 poisoning rate, FINCH-based clustering completely filters malicious samples and neutralizes the attack.Even at an aggressive 9.9% poisoning rate, AgentAuditor retains a substantial advantage over the baseline.Source: §O.2 · p. 47
• Across all three adversarial attack levels, AgentAuditor maintains a performance advantage over the unattacked baseline, including when guiding information is actively compromised.Level 1 attacks poison the dataset, Level 2 directly poison reasoning memory with misleading CoTs, and Level 3 replaces a final decision example with forged reasoning.Source: §O.2 · pp. 47–48
• At a 33.3% Level 2 poisoning rate, with 8 of 24 memory shots malicious, AgentAuditor still surpasses the unattacked baseline.Level 2 is more potent because it combines incorrect labels with expertly crafted misleading CoT traces.Source: §O.2 · pp. 47–48

O.3 Robustness to Out-of-Distribution Cases

• Cross-dataset evaluation shows that performance degrades with mismatched memory, yet AgentAuditor still achieves substantial improvements over the baseline under simulated OOD conditions.Reasoning memory is constructed from R-Judge and applied to ASSEBench-Safety and ASSEBench-Security, where many cases lack highly relevant matches.Source: §O.3 · p. 48, Table 19 · p. 48
• Using lower-ranked shots, specifically ranks 4–6, preserves strong performance, while random shot selection also yields gains over the baseline.These controlled R-Judge experiments isolate memory-augmented reasoning under poor retrieval quality while keeping the domain and memory fixed.Source: §O.3 · p. 48, Table 20 · p. 48
• The findings indicate robust generalization, enabling deployment on new domains with reasonable confidence before domain-specific memory is fully constructed.This supports practical use where perfect domain coverage is infeasible.Source: §O.3 · p. 48

P Resource Usage Analysis

• Resource analysis focuses on LLM-related costs because PCA and FINCH consume negligible time relative to LLM inference.Comparisons include direct LLM judgment and judgment by fine-tuned smaller-parameter LLMs.Source: P Resource Usage Analysis · p. 49
• Experiments use representative open-source Qwen-2.5 and QwQ-32B models on a node with four Nvidia Tesla A100 80G GPUs, two EPYC 64-core CPUs, and 480GB RAM.These choices are intended to make API-dependent price and speed comparisons fair and reproducible.Source: P Resource Usage Analysis · p. 49
• The resource experiments use BF16 models deployed with vLLM on R-Judge’s 564 interaction records, with ShieldAgent [98] fine-tuned from Qwen-2.5-7B using 4,000 annotated records.GPU hours are normalized to a single Nvidia Tesla A100 80G GPU, and ShieldAgent’s reported fine-tuning takes 4 hours on four A100 GPUs.Source: P Resource Usage Analysis · p. 49
• 24 annotated records and 0.12–1.07 GPU hours let AgentAuditor build its reasoning memory, versus ShieldAgent’s 4,000 records and approximately 16 GPU hours for fine-tuning.These comparisons concern data resources and pre-inference computation.Source: Table 21 · p. 49, P Resource Usage Analysis · p. 49
• AgentAuditor’s higher inference GPU hours than base models remain worthwhile, while its end-to-end computational demand is considerably lower than fine-tuning-based methods.The advantage is attributed especially to fine-tuning’s substantial training overhead.Source: P Resource Usage Analysis · p. 49
• By using smaller-parameter models to achieve or surpass larger models’ performance, AgentAuditor reduces VRAM requirements for agent safety and security evaluation.The passage compares this strategy with direct LLM use and notes its suitability for resource-constrained settings.Source: P Resource Usage Analysis · p. 49

Q Details of Human Evaluation

• Human evaluation has two parts, including a test of whether AgentAuditor achieves human-level accuracy.The supplied description introduces a second evaluation aim but does not specify it fully.Source: Q Details of Human Evaluation · p. 49
• Five LLM safety and security experts, distinct from ASSEBench’s developers, receive standardized training, criteria, and 10 reference examples before evaluating actual test cases.Each expert has at least three months of prior research experience, and the separation is intended to prevent data leakage.Source: Q Details of Human Evaluation · p. 50

Q.1 Judgment Performance Evaluation

• AgentAuditor with Gemini-2-Flash-thinking reaches or closely approaches average human-evaluator performance on R-Judge, AgentHarm, and ASSEBench-Security.The comparison uses 300 sampled entries from six datasets and reports F1-score, recall, precision, and accuracy against an independently established human baseline.Source: Table 22 · p. 50, §Q.1 · p. 50

Q.2 Reasoning Process Evaluation

• AgentAuditor’s reasoning traces are evaluated against direct LLM risk analyses across logical structure, reasoning soundness, and completeness.The study compares Gemini-2-Flash-thinking and QwQ-32B with and without AgentAuditor on 100 ASSEBench cases using six human experts and 5-point Likert ratings.Source: Figure 10 · p. 51, §Q.2 · p. 52
• The reasoning-process evaluation shows strong inter-rater reliability, with Krippendorff’s alpha α = 0.86, exceeding the 0.8 threshold for good agreement.Six experts produced 2400 evaluation sets across 400 generated outputs, and the resulting ratings were visualized in radar charts.Source: §Q.2 · p. 52

R Discussion of Scalability

• After one-time reasoning-memory construction, evaluating each new case has constant, minimal cost that depends on the small memory rather than the original dataset size.This supports scalability to larger datasets without repeating the full memory-construction process.Source: R Discussion of Scalability · p. 52
• AgentAuditor transfers across domains through zero-shot feature extraction and chain-of-thought generation, FINCH clustering, and transferable retrieval-augmented memory components.Applying it to a new domain requires only a batch of interaction logs, and experiments report consistent, significant improvements across benchmarks spanning different scenarios.Source: R Discussion of Scalability · p. 52

S Broader Impacts

• AgentAuditor and ASSEBench aim to make agent safety and security evaluation more accurate and human-aligned, supporting responsible development and deployment.The intended benefits include mitigating harms from improper autonomous financial management or unsafe operations and cultivating public confidence in advanced systems.Source: S Broader Impacts · p. 52
• Automated evaluation can create false security, benchmarks cannot cover every emerging risk, and assessment insights may help malicious actors bypass security checks.The authors therefore advocate continuously refining and expanding benchmarks while using these resources responsibly and collaboratively.Source: S Broader Impacts · p. 52