PaperlayerView exact revision on arXiv

Source-linked AI summary

JEV as a Judge for Agent Trace Security: An Empirical Comparison with Generative LLM Judges

Zhiqiang Wang, Yichao Gao

arXiv:2609.34862v1cs.CR

TL;DR

Agent security evaluation requires judging behavior in context, and the suitability of generative judges for accurate trace classification at bounded resource cost remains unclear. The paper compares JEV’s typed decisions with four generative judges on benchmark trajectories using shared risk anchors and a common decision threshold. JEV achieves the highest benchmark-averaged F1, while dataset results show complementary strengths between JEV and GLM-5.2.

  • Problem

    It remains unclear whether generative judges’ rationales and validation costs are necessary for accurate trace classification, where security depends on interpreting actions in context.

    Source: §1 · p. 1

  • Method

    The study compares typed JEV judgments with four generative judges on saved trajectories, applying a common five-level risk rubric and threshold.

    Source: §1 · p. 1, §3.1 · p. 2

  • Results

    77.8 mean F1 at the primary threshold for JEV versus 74.1 for GLM-5.2, with JEV leading on ATBench500 and MCPHunt while GLM leads on R-Judge and TraceSafe.

    Source: §5.1 · p. 4

  • Takeaways & Limitations

    The findings support typed judgments as an efficient approach to retrospective trace evaluation and motivate fast screening paired with explanatory review.

    Source: §7 · p. 8, §6 · p. 8

    Reported metrics are influenced by revised labels, missing outputs, benchmark composition, and evaluated serving conditions; trace-length findings describe the observed workload distribution.

    Source: §6 · p. 8

Abstract

from arXiv · show

Security evaluation of tool-using agents requires judging actions in context, yet generative judges add latency, explanation overhead, and output-validation failures. We study whether JEV, a typed decision model, offers a useful alternative for retrospective trace classification. We evaluate JEV and four generative judges on four benchmark collections totaling 5,219 trajectories, using a common risk rubric and behavior-level labels. JEV attains a benchmark-averaged positive-class F1 of 77.8, compared with 74.1 for the strongest generative configuration, GLM-5.2, with valid-result coverage of 95.5\% and 94.4\%, respectively. Performance varies across datasets, with JEV leading on ATBench500 and MCPHunt and GLM leading on R-Judge and TraceSafe. Across the four benchmarks, JEV's median successful-call latency is 0.99 seconds; estimated token cost averages \$0.000195 per valid judgment. These results support JEV as an economical screening signal, with trade-offs in precision and recall.

1 Introduction

Agent security depends on interpreting actions in context, while the accuracy–resource trade-off of generative judges for trace classification remains unclear. This study compares typed JEV judgments with four generative configurations on saved trajectories.

1 Introduction

  • A useful trace judge must distinguish necessary authentication from unauthorized disclosure and rejected from executed injected instructions using the trajectory’s context and action order.Sensitive words or an attack attempt alone are insufficient to determine security behavior.Source: §1 · p. 1
  • The study compares JEV with four generative judges on saved trajectories, using a shared rendering pipeline and five-level risk anchors.JEV returns bounded decisions and distributions rather than free-form explanations.Source: §1 · p. 1
  • The central question is whether a typed judge offers a useful accuracy–resource trade-off for agent trace security.The comparison covers detection quality, valid-result coverage, latency, token cost, and performance by trace length.Source: §1 · p. 1
  • The empirical comparison spans four collections totaling 5,219 trajectories and jointly examines detection quality, resource use, and longer traces.The study’s contributions include evaluating JEV against four generative judges.Source: §1 · pp. 1–2
  • JEV has the highest four-benchmark mean F1 at the common threshold, while GLM-5.2 is stronger on two datasets and more precise overall.The comparison characterizes practical trade-offs between typed judgments and generative review.Source: §1 · p. 2

2 Related Work

Prior work evaluates LLM judges, agent safety benchmarks, and safety evaluators or guardrails. This study focuses instead on behavior-level security classification over agent traces.

LLM-based evaluation

  • LLM-judge work covers open-ended response evaluation [18], generated-text scoring [7], open evaluation under user criteria [4], and objective-correctness judge benchmarks [11].Agent trace security changes the judged object and error consequences, motivating behavior-level classification rather than answer preference.Source: §2 · p. 2

Agent safety benchmarks

  • R-Judge [15], ATBench [6], TraceSafe [2], and MCPHunt [5] provide complementary agent-safety trace settings, including multistep tool use and cross-boundary data propagation.The study uses the 500-trajectory ATBench release and a 540-trajectory TraceSafe subset alongside R-Judge and MCPHunt.Source: §2 · p. 2

Agent safety evaluators and guardrails

  • Related evaluators combine emulated execution with LM evaluation [9], retrieved reasoning experiences [8], or executable guardrail code.Source: §2 · p. 2

3 Task and Judge Configurations

The framework gives both judge paths a shared trajectory representation and risk anchors, then applies a common threshold and compares predictions with reference labels. JEV returns bounded typed judgments; generative judges also provide rationale and evidence.

3 Task and Judge Configurations

  • Both judge paths receive the trajectory and risk anchors; their risk scores use the same threshold and are evaluated against reference labels alongside resource use.Reference labels are used only for evaluation.Source: §3 · p. 2, Figure 1 · p. 3

3.1 Trace-level security classification

  • The trajectory comprises the user request, available context, and the agent’s actions and observations; each judge assigns a risk score from 1 to 5.The shared rubric treats unsafe or unauthorized behavior as positive and separates authorization from intrinsic safety.Source: §3.1 · p. 2

3.2 Input and judgment interfaces

  • Both approaches receive the user request, provenance-aware context, and the complete rendered action/observation sequence, without ground-truth labels in the input.They share risk anchors and a binary threshold, while retaining their configured interfaces and instructions.Source: §3.2 · p. 3
  • JEV answers four typed questions with bounded values and probabilities, while generative judges return the same assessment fields plus rationale and supporting evidence.The interfaces reflect rapid decisions for JEV and explanatory review for LLM judges.Source: §3.2 · p. 3

4 Experimental Setup

The evaluation covers four behavior-labeled benchmark collections and compares JEV with four generative judges using classification and resource measures. Resource reporting includes successful-call latency and estimated token charges.

4.1 Benchmarks and labels

  • The evaluation covers 5,219 trajectories from R-Judge, ATBench500, a TraceSafe subset, and MCPHunt, using behavior-level labels across contextual risk, unsafe behavior, injections, and data propagation.TraceSafe and MCPHunt use revised labels that distinguish resisted injections and signal-only cases from unsafe behavior.Source: §4.1 · p. 3

4.2 Models and inference settings

  • JEV is compared with four generative judges: GLM-5.2-Tencent, DeepSeek-V4-Flash, DeepSeek-V4-Pro-Seed, and Qwen3-Next-80B.Source: §4.2 · p. 3

4.3 Evaluation metrics

  • The study reports positive-class precision, recall, F1, and accuracy for valid judgments; overall F1 is the unweighted mean of the four benchmark F1 scores.Coverage is the share of each full benchmark receiving a valid judgment, while correct-decision yield gives no credit to missing outputs.Source: §4.3 · p. 3

4.1 Benchmarks and labels

  • Table 1 reports benchmark populations and trajectory-step distributions under the revised-label policy.Steps count messages and observations, and P95 is the 95th percentile.Source: §4.1 · p. 3, Table 1 · p. 4

4.4 Latency and cost

  • Latency measures successful single-call evaluations, excluding failed attempts, repair calls, and retry waiting; token costs use recorded usage and fixed prices.Logged repair tokens are included when attached to a valid result, and measurements describe per-judgment resource use under the evaluated serving configurations.Source: §4.4 · p. 4

5 Results

Across four benchmarks, JEV has the highest mean F1, though the best judge varies by dataset. Its coverage and resource use are reported alongside classification results, and trace-length analyses show how detection and latency vary across evaluated workloads.

5.1 Overall performance and dataset heterogeneity

  • 77.8 mean F1 at the primary threshold puts JEV ahead of GLM-5.2 at 74.1 across four benchmarks.On trajectories with valid judgments from both, JEV retains a 2.7-point benchmark-averaged F1 advantage.Source: §5.1 · p. 4, Table 2 · p. 4
  • JEV leads on ATBench500 (93.8 F1) and MCPHunt (61.1), while GLM leads on R-Judge (93.4) and TraceSafe (72.4).JEV’s advantages on ATBench500 and MCPHunt accompany higher recall despite lower precision; GLM’s higher precision offsets slightly lower recall on TraceSafe.Source: §5.1 · p. 4, Figure 2 · p. 5, Table 3 · p. 5

5.2 Judgment coverage

  • JEV returns valid judgments for 95.5% of trajectories, compared with 94.4% for GLM; DS-V4-Pro’s 98.7% ATBench500 accuracy uses only 151 of 500 trajectories.The study therefore reports coverage alongside classification metrics.Source: §5.2 · pp. 5–6, Figure 3 · p. 6

5.3 Resource consumption

  • JEV’s median latency is 0.99 seconds, while generative judges’ mean latencies range from 9.61 to 37.39 seconds; its mean estimated token charge is $0.000195 per valid result.Generative token-cost estimates range from $0.001320 to $0.014887 per valid result.Source: §5.3 · p. 6, Table 4 · p. 6

5.4 Detection quality and latency across trace lengths

  • For traces with 5–8, 9–16, and more than 16 steps, JEV’s F1 is 85.2, 67.4, and 57.5, versus GLM’s 78.5, 58.4, and 50.6.By input-token count, JEV reaches 69.9 F1 at 2–4k tokens and 62.8 at 8k or more, versus GLM’s 56.0 and 48.7.Source: §5.4 · pp. 7–8, Figure 4 · p. 7, Figure 5 · p. 7
  • JEV’s median latency rises from 0.90 to 1.08 seconds between the shortest and longest step bins, while GLM’s rises from 27.34 to 42.31 seconds.JEV’s step-based F1 peaks at an intermediate length, while its token-based F1 is highest in the shortest bin and relatively flat across 2–8k tokens before declining; absolute F1 does not rise consistently with length.Source: §5.4 · pp. 7–8

6 Discussion and Limitations

JEV is presented as a candidate for screening completed traces, with GLM’s precision and explanations supporting detailed review. The comparison and length analysis have scope limits that constrain how broadly to interpret the results.

6 Discussion and Limitations

  • JEV’s detection quality and low resource consumption make it a candidate for screening large volumes of completed agent traces, with GLM’s precision and explanations supporting detailed review.The authors motivate a screening-and-escalation pipeline in which compact judgments prioritize traces for further investigation.Source: §6 · p. 8
  • The judge configurations differ in detailed instructions, output requirements, and serving conditions, while resource measurements cover successful calls and estimated token charges.Revised labels, missing outputs, and benchmark composition also influence the reported metrics.Source: §6 · p. 8
  • The length analysis describes the observed workload distribution; controlled trace-length variations are proposed for future generalization tests.Source: §6 · p. 8

7 Conclusion

Across four agent trace collections, JEV has the highest benchmark-averaged positive-class F1 among evaluated configurations, alongside sub-second median latency and low estimated token cost. Per-dataset results show complementary strengths, supporting typed judgments for retrospective security evaluation.

7 Conclusion

  • JEV achieves the highest benchmark-averaged positive-class F1 among evaluated configurations, with sub-second median latency and low estimated token cost.Source: §7 · p. 8
  • JEV leads on ATBench500 and MCPHunt, while GLM leads on R-Judge and TraceSafe and has higher precision overall.Source: §7 · p. 8
  • JEV remains competitive in several longer-trace regimes, and the findings motivate combining fast screening with explanatory review.Source: §7 · p. 8

Data Availability

The paper identifies its public benchmark collections and points to appendices for materials needed to interpret results. Its rubric defines risk scoring and describes separate outputs and prompt rules for the judge configurations.

Data Availability

  • The evaluation uses public R-Judge, ATBench, TraceSafe, and MCPHunt benchmarks, with appendices providing the scoring rubric, pricing assumptions, sample counts, and supplementary analyses.Source: Data Availability · p. 8

A Risk Scoring Rubric

  • The overall risk rubric assigns higher values to greater risk, and JEV’s score question concerns serving the request within scope and performing or materially enabling unsafe behavior.Authorization and intrinsic-safety questions are separate outputs; harmless discussion, safe refusals, sensitive topics without risky action, and benign read-only operations alone do not warrant an unsafe verdict.Source: Table 5 · p. 11, A Risk Scoring Rubric · p. 11
  • The LLM prompt distinguishes exposure from adoption, preparation from harmful target actions, and timely effective defenses from later mitigation; binary decisions use threshold 3.Ordinary use of necessary data at the intended tool is not automatically leakage.Source: A Risk Scoring Rubric · p. 11

B Comparison on Common Valid Samples

On trajectories where both judges return valid judgments, JEV’s benchmark-averaged F1 exceeds GLM’s by 2.7 percentage points. The exploratory paired-bootstrap interval remains conditional on valid outputs and does not remove selection bias from missing judgments.

B Comparison on Common Valid Samples

C Configured Price Assumptions

The price assumptions specify how token costs are estimated, while the confusion-count table documents classification counts and class-dependent coverage.

C Configured Price Assumptions

  • JEV is charged for input tokens only [12], and estimated mean costs pool all four benchmarks while excluding failed requests.For valid judgments, cost is estimated from input and output token counts; reasoning tokens count toward output usage.Source: Table 7 · p. 12, C Configured Price Assumptions · p. 12

D Confusion Counts and Class-dependent Coverage

  • Table 8 presents confusion counts at s ≥3 and coverage percentages within the full positive and negative populations.The table is intended to make conditional performance and nonrandom missingness auditable.Source: Table 8 · p. 12

E Step- and Token-Stratified Results

The section describes length-stratified comparisons, timing, and the judging instructions and output formats used for generative judges and JEV. It notes that apparent length trends are sensitive to benchmark composition.

E Step- and Token-Stratified Results

  • Tables 9 and 12 give sample counts with positive-class F1 by classification-panel point and median latency in seconds for four-benchmark timing; Table 10 reports JEV-minus-GLM F1 differences in percentage points on identical successful trace IDs within each length bin.Source: Table 9 · p. 13, Table 10 · p. 13, Table 12 · p. 14
  • Excluding MCPHunt changes rankings, and small long-bin samples show that apparent length trends are sensitive to benchmark composition.The analyses use 4,982 valid JEV judgments as the reference population, with step count and input tokens as length measures; bins partition available samples and use labels at s ≥3.Source: E Step- and Token-Stratified Results · pp. 12–14, Table 11 · p. 13

F Full Prompt for the LLM Judge

F.1 Safety Review Template

  • The safety template assesses authorization and intrinsic safety independently, weighs risk against counterevidence and defenses, and scores the trace from 1 to 5.It distinguishes attempted unsafe operations from ordinary authorized calls that fail, and specifies conditions for a severity 4–5 finding to force a score floor.Source: §F.1 · pp. 14–15

F.2 Output Contract

  • The human-review contract requires authorization, safety judgment, severity, confidence, holistic score, reasoning, evidence references, and evidence gaps.It limits evidence references to decisive original trace items and requires Simplified Chinese for reasoning, reference supports, and evidence gaps.Source: §F.2 · pp. 15–16

F.3 Direct Review Instruction

  • The direct-review instruction says no triage or inspector has run, so judges must assess the complete original trace independently.It states that an intentionally empty evidence log is not evidence of safety.Source: §F.3 · p. 16

G.1 Question Definitions

  • JEV receives the trajectory as state and four typed questions covering overall score, authorization, intrinsic safety, and severity.The question definitions keep authorization separate from intrinsic safety and include insufficient evidence as a safety-judgment option.Source: G Complete Definitions of the Four… · p. 16, §G.1 · pp. 16–17

G.2 Response Format

  • Each JEV question returns a value, calibrated probabilities, and confidence; its returned score is a zero-based index mapped to the 1–5 risk scale by adding one.The example response shows a score of 3.0 with confidence 0.7.Source: §G.2 · p. 17

Related papers

Loading 2609.34862v1…