PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Diagnosing Verifier Brittleness in Agentic Evaluation

Radhika Gaonkar

arXiv:2610.11678v1cs.CL

TL;DR

Verifier scores can change without a corresponding change in agent capability, so a score gap needs diagnosis. TRACE combines paired mutations, trajectory inspection, and counterfactual rescoring to test whether behavior or evaluation changed. In follow-up studies, meaning-preserving changes are equivalent within ±0.10 for seven of eight agent–change pairs, while deliberately misleading names lower reward for all four agents; judges remain stable across presentations yet disagree on many records.

  • Problem

    Verifier scores are used to evaluate and train LLM agents, but a score change may reflect the evaluation rather than a change in capability.

    Source: §1 · p. 1

  • Method

    TRACE applies targeted evaluation mutations, compares paired runs, inspects trajectories for behavioral changes, and counterfactually rescores unchanged trajectories to test the evaluator.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    Seven of eight agent–change pairs are equivalent within ±0.10 for meaning-preserving changes, misleading names lower reward for all four agents, and two judges disagree on 57% of the same records.

    Source: §5 · pp. 7–8

  • Takeaways & Limitations

    A score change is an observation rather than an explanation; interpreting it requires distinguishing agent behavior from the measurement and comparing it against an identical rerun and a change known to matter.

    Source: §8 · p. 9

    The public studies cover a limited set of tasks, presentation changes, and four agents; native reward is a reference rather than ground truth, and judge audits assess consistency and agreement, not accuracy.

    Source: §7 · p. 9

Abstract

from arXiv · show

Verifier scores now serve as both benchmark metrics and training rewards for large language model (LLM) agents, and a change in score is routinely read as a change in capability. It may instead reflect a change in the evaluation. We introduce TRACE, a protocol that turns a score change from a verdict into a testable diagnosis: it applies a targeted change to one part of an evaluation, compares paired runs, checks whether the agent's behavior changed, and rescores unchanged trajectories to test whether the scoring rule is responsible. In a controlled suite of 25 synthetic tasks, renaming tools lowers a scripted agent's score by 0.250 even though it performs exactly the same operations; restoring the original names at scoring time closes the entire gap, while the same mutation exposes a genuine behavioral failure in a second agent. On public $\tau^2$-bench tasks with four LLM agents, an initial 30-task study finds mixed reward changes whose one clear effect does not replicate. In a larger follow-up on 88 new tasks with repeated runs per condition, renaming tools or reformatting tool outputs leaves reward unchanged to within $\pm$0.10 for seven of eight agent-change pairs, whereas tool names that deliberately mislead lower every agent's reward by 0.20-0.44, showing that the setup can detect real effects. Identical reruns flip 15-36% of task outcomes, so single-run comparisons cannot separate presentation effects from run-to-run variation. Two frontier LLM judges give consistent verdicts when a fixed trajectory is presented differently, yet disagree with each other on 57% of the same records, largely because one grades procedure rather than outcome. TRACE thus separates what a score change says about the agent from what it says about the measurement.

1 Introduction

Verifier scores are often read as measures of agent capability, but evaluation changes can move scores even when behavior does not. TRACE tests what caused a score change by comparing runs, inspecting trajectories, and rescoring unchanged records.

1 Introduction

  • A verifier score change can reflect the evaluation rather than a capability change, with a tool-renaming mutation lowering an unchanged agent’s score by 0.250.The verifier matches tool names instead of checking the operations performed, so the drop would register a false regression on a leaderboard or penalize correct behavior as a training reward.Source: §1 · p. 1
  • Interactive benchmarks depend on tool interfaces, observation formats, simulated users, and LLM judges, while audits have identified task and reward-design flaws.The paper frames its question through measurement validity: what conclusions a verifier score supports.Source: §1 · p. 1
  • TRACE diagnoses score changes by pairing targeted evaluation mutations with trajectory inspection and counterfactual rescoring of unchanged records.Its central question is whether the agent, environment, or verifier is responsible for a score gap.Source: §1 · p. 2

2 TRACE: From Score Differences to Explanations

TRACE treats a score difference as a diagnostic question: paired runs measure score changes, trajectory comparisons check behavior, and rescoring unchanged trajectories tests the evaluator. Its diagnostics are interpreted relative to each probe’s intended change.

2 TRACE: From Score Differences to Explanations

  • TRACE runs paired tests, compares scores, inspects trajectories, then counterfactually rescores the same trajectory under a corrected scoring rule.The stages distinguish whole-system sensitivity from evaluator sensitivity; identical reruns estimate chance variation, while a positive control checks detection of a real effect.Source: §2 · p. 2

2.1 What the Probes Change

  • The six probe families use fixed templates and target different changes; tool renaming preserves behavior, whereas rubric changes alter what earns credit.Results characterize those particular transformations, not the full space of changes in each family; incentive changes apply only to tasks with proxy channels.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 Score-Level Diagnostics

  • The verifier robustness gap (VRG) is the original score minus the mutated score, so a positive gap indicates a score decrease under mutation.Each trajectory is scored under its own evaluation condition when a mutation changes the scoring rule.Source: §2 · p. 2, §2.2 · p. 3
  • Mutated success rate (MSR) measures the fraction of mutated runs that pass, while mutation transfer rate (MTR) restricts that fraction to pairs whose original run passed.Both are verifier pass rates, not independent checks of task completion; MTR is undefined when no original run passes.Source: §2.2 · p. 3
  • Proxy–true divergence (PTD) subtracts simulator-defined quality from visible proxy reward, with positive values indicating that the proxy exceeds quality.PTD is reported only for the 12 tasks with proxy channels because task scales differ and proxy reward may exceed one.Source: §2.2 · p. 3

3 Study 1: Diagnosing Known Failures

In a controlled suite of 25 synthetic tasks, TRACE distinguishes score-rule defects from behavioral failures and shows why score gaps alone can mislead. The study also contrasts proxy-based and composite scoring objectives.

3.1 The Same Execution Can Receive a Lower Score

  • Careful’s tool-renaming gap is 0.250 and falls to 0.000 after mapping aliases back before rescoring, identifying the name-matching rule as the cause.Figure 2 shows the gap by policy and probe and the result of rescoring after the mapping correction.Source: Figure 2 · p. 4, §3.1 · p. 4

3 Study 1: Diagnosing Known Failures

  • ResearchOpsEnv contains 25 synthetic tasks, three scripted policies with prescribed behavior, and a composite verifier distinct from simulator quality.The archive contains 1,575 records across 525 task–policy–condition cells, each repeated under three seed labels; task-cluster bootstrapping describes variation within this designed suite.Source: §3 · p. 4

3.1 The Same Execution Can Receive a Lower Score

  • The same correction leaves overfit’s gap at 0.130 because renaming triggers a different scripted branch and an unsuccessful report.The probe therefore exposes a behavioral failure for overfit alongside the scoring defect for careful.Source: §3.1 · p. 4

3.2 Small Gaps Can Hide Consistent Failure

  • Shortcut’s mean gap is only 0.006 despite an MSR of 0.000, because it scores poorly in both conditions.For comparison, MSR is 0.860 for careful and 0.053 for overfit; a gap-only report would misrank shortcut as most robust.Source: Figure 2 · p. 4, §3.2 · p. 5
  • Overfit’s MTR of 0.667 rests on only 36 original-passing pairs, so it does not indicate broad success.Careful’s MSR has a 95% task-cluster interval of [0.840, 0.887].Source: §3.2 · p. 5
  • Shortcut’s mean PTD is 0.932 on the 12 proxy-relevant tasks, with a wide task-cluster interval of [0.176, 1.781].In one reward-hack task, shortcut sets two visible-reward parameters to 2.0 each, earning visible reward 4.0 while simulator quality stays at 0.0.Source: §3.2 · p. 5

3.3 Scoring Objectives Favor Different Behaviors

  • A visible-proxy objective selects proxy exploit with held-out pass rate 0.103 and PTD 0.996, while tool-volume and composite objectives select careful.Both latter objectives yield held-out pass rate 0.718 and PTD −0.180; the composite verifier still has the tool-name defect, separating useful policy selection from consistent scoring.Source: §3.3 · p. 5

4 Study 2: LLM Agents on τ2-bench

On public τ2-bench, the study compares fresh agent interactions and fixed-trajectory judge views under interface changes. Fresh-run effects are mixed, while Study 3 finds meaning-preserving changes near zero and misleading tool names lower every agent’s reward.

4 Study 2: LLM Agents on τ2-bench

  • Four LLM agents ran original, renamed-tool, and value-preserving reformatted-observation conditions against fixed simulated users and benchmark evaluators.The task split, configuration, and analyses were frozen after development review.Source: §4 · p. 5

4.1 Fresh Interactions: Whole-System Sensitivity

  • Seven of eight intervals include zero; Laguna under tool renaming improves by +.267 [.067, .433], while 65 of 236 complete pairs change reward.The sole interval excluding zero is weak evidence of a systematic effect and does not replicate on new tasks.Source: Figure 3 · p. 6, §4.1 · p. 6
  • The single uncorrected Laguna effect does not replicate; trajectory review links changes to different write actions, arguments, or incomplete required writes.The fresh-run changes indicate system sensitivity, not by themselves a verifier defect.Source: §4.1 · p. 6

4.2 Fixed Trajectories: Evaluator Sensitivity

  • On 118 fixed trajectories, repetition and tool renaming each yielded 0/118 verdict flips, while reformatting yielded 1/118; the lone flip cannot establish a formatting effect.The judge saw the task criteria, domain policy, tools, and trajectory, but not the native reward or agent identity.Source: §4.2 · p. 6
  • In Study 3, meaning-preserving changes stay near zero, while misleading tool names lower every agent’s reward.The shaded band marks the ±0.10 equivalence margin, judged on 90% intervals.Source: Figure 4 · p. 7

5 Study 3: Replication at Scale with Repeat and Positive Controls

Study 3 tests meaning-preserving presentation changes and deliberately misleading tool names with repeated runs, a repeat control, and a positive control. Meaning-preserving changes are equivalent for nearly all agent–change pairs, while the misleading-name control is detected for every agent; rerun noise and judge disagreement qualify these findings.

5 Study 3: Replication at Scale with Repeat and Positive Controls

  • The follow-up uses four agents on 158 airline and retail tasks, with three runs per condition and a pre-fixed ±0.10 equivalence margin.The main test set comprises 88 previously unevaluated tasks; a repeat control compares changed-run flips with identical-run flips, and a positive control permutes domain tool names while keeping descriptions and schemas true.Source: §5 · p. 7
  • Seven of eight agent–change pairs are equivalent within ±0.10, while the positive control lowers reward by 0.196 to 0.437 for all four agents.Six pairs remain equivalent after Bonferroni correction; the positive control shows the pipeline can detect changes known to matter.Source: §5 · p. 7
  • Laguna’s tool-renaming gain of +.267 falls to −.050 on an intermediate 40-task wave and −.011 on the original 30 tasks, where it is equivalent within ±0.10.An exploratory 108-task telecom replication gives the same picture.Source: §5 · p. 7
  • Identical reruns flip outcomes on 14.7–16.3% of runs for DeepSeek, GLM, and Qwen, and 35.6% for Laguna.A single-run comparison cannot separate a presentation effect from ordinary variation; Laguna’s high rerun noise is consistent with its non-replicating gain.Source: §5 · p. 8
  • The two judges disagree on 57% of the same records, although presentation changes do not shift verdicts beyond call-to-call variation.Against native reward, Claude Opus 5.5 agrees on 95% of verdicts (κ = 0.89), while GPT-6.1 Sol agrees on 38% (κ = 0.09); its disagreements all mark benchmark successes, often objecting to procedure.Source: §5 · p. 8

6 Related Work

TRACE builds on work about reward mismatch, evaluator feedback, agent benchmarks, and behavioral testing. It adds a paired procedure for attributing score changes to behavior or scoring rather than merely detecting them.

6 Related Work

7 Discussion: What the Evidence Establishes

The evidence supports diagnosing specific evaluation changes, not estimating their prevalence in deployment. Interpreting null effects requires repeat controls and a positive control, and the judge audits do not establish accuracy against ground truth.

7 Discussion: What the Evidence Establishes

  • Study 3 requires comparing paired differences with identical reruns and using a positive control, because equivalence is informative only if the pipeline detects a change known to matter.The authors also recommend reporting pass rates alongside gaps and confirming suspected defects by rescoring the same trajectories.Source: §7 · p. 9
  • The evidence spans 25 designed ResearchOps tasks and public studies of airline, retail, and exploratory telecom tasks, four agents, and two presentation changes; it does not estimate deployment failure rates or learned training exploits.Equivalence is relative to ±0.10, missing runs are not random, native reward is a reference rather than ground truth, and judge audits measure consistency and agreement rather than accuracy; probe overfitting across mutation families remains untested.Source: §7 · p. 9

8 Conclusion

TRACE treats a score change as an observation requiring diagnosis: paired mutations, trajectory inspection, and counterfactual rescoring distinguish behavioral change from scoring defects. The appendix details the composite verifier and additional process metrics used in the controlled study.

8 Conclusion

  • A score difference becomes evidence only against an identical rerun and a change known to matter.The conclusion reports that counterfactual rescoring fully repairs the controlled tool-name defect, distinct from behavioral failures; on τ2-bench, nearly all meaning-preserving changes are equivalent within ±0.10 while misleading names are detected.Source: §8 · p. 9
  • All 1,575 controlled-study records reproduce exactly, and frozen τ2 artifacts include configurations, hashes, task splits, raw traces, and analyses that recompute without model calls.The authors report that TRACE code and synthetic ResearchOps tasks are available in an archived repository.Source: §8 · p. 9

A Scoring Details

  • The archived composite verifier weights hidden-target terms, canonical tool names, constraint checks, efficiency, and hack-risk penalties; its score is not clipped to [0, 1].The tool-name defect lies in component C; replacing it with an alias-aware check changes the score by 0.25(Calias − C) with the trajectory and other components fixed.Source: A Scoring Details · p. 14
  • The auxiliary hack-detection rate is the mean of D, while the process-overfitting score compares call-count and tool-use heuristics, not reasoning quality.PTD uses maximum proxy reward and maximum simulator quality across runs, which may come from different runs; its 12 applicable tasks are included, while other tasks are excluded rather than assigned zero.Source: A Scoring Details · p. 14

B Policy-Selection Objectives

The study compares three selection objectives on held-out tasks and finds that visible-proxy selection favors exploitation, while tool-volume and composite selection choose careful behavior. The composite objective shows no advantage over tool-volume selection.

B Policy-Selection Objectives

  • Each objective selects the highest-scoring of five programs on the first 12 original tasks, then evaluates the selected program on 13 held-out task IDs.Task types and mutation templates are shared across the split.Source: B Policy-Selection Objectives · pp. 14–15
  • 0.103 versus 0.718 held-out pass rates, and 0.996 versus −0.180 proxy PTD, favor careful over proxy exploit under tool-volume and composite selection.The visible-proxy objective selects proxy exploit; tool-volume and composite both select careful, so the composite objective shows no advantage over that comparator.Source: B Policy-Selection Objectives · p. 15, Table 5 · p. 15
  • Because careful behavior and exploitation are already among the five candidates, this search does not show how a learning agent would acquire either behavior.Source: B Policy-Selection Objectives · p. 15

C Public Study Details

The public study fixes its benchmark components and defines tool-alias and formatting mutations that preserve execution semantics. The proposed held-out-probe design aims to reduce memorization risk, but is not demonstrated as a solution to probe overfitting.

C Public Study Details

  • The public study fixes the benchmark commit, models, simulators, evaluator, limits, mutations, and retry policy; aliases preserve tool schemas and execution, while formatting changes JSON presentation without changing values or array order.Source: C Public Study Details · p. 15
  • Two original runs are missing from the 360-job inventory, and neither reward is imputed; a separate Qwen alias run that reached harness timeout remains a scored zero.The missing runs were one DeepSeek airline run truncated at the output cap and one externally interrupted Qwen airline run.Source: C Public Study Details · p. 15

D Toward Held-Out Probes

  • The proposed probe design separates development probes from held-out instances and families, but the authors state it is not a demonstrated solution to probe overfitting.The proposal warns that tuning on fixed probes could encourage memorizing aliases or reward traps.Source: D Toward Held-Out Probes · p. 15

E Study 3 Details

Study 3 uses frozen plans and repeated runs to evaluate presentation mutations and a misleading-name control, alongside a one-run replication wave and a repeat-controlled judge audit. The intermediate and exploratory results indicate that presentation effects must be considered against run-to-run variation.

E Study 3 Details

  • Study 3 runs used sealed plans, frozen analysis rules, independent review, and audits that recomputed every reported number without mismatches.Changes were averaged over same-seed repetitions within tasks and then over tasks; missingness was not random, mostly reflecting DeepSeek output-limit truncations.Source: E Study 3 Details · p. 16
  • The larger Study 3 comparison covers tool renaming and observation reformatting on 88 new tasks with three runs per condition, plus a misleading-name control on 158 tasks.Table 6 defines Δ as changed minus original reward and “equiv.” as a 90% interval within ±0.10.Source: Table 6 · p. 16
  • Every interval in the 40-task, one-run replication included zero, and the smallest exact McNemar p was 0.29; the authors note these differences mix perturbation effects with agent and user-simulator variation.478 of 480 runs completed; the two excluded DeepSeek runs hit the output cap.Source: E Study 3 Details · p. 16, Table 7 · p. 16
  • The judge audit withheld agent identity and native reward, used a pass/fail/insufficient prompt, and sent byte-identical original and repeat payloads as separate requests.One Claude Opus 5.5 repeat was missing after a provider content filter.Source: E Study 3 Details · pp. 16–17
  • The repeat-controlled judge audit covered 118 fixed trajectories with three calls per view; its native-reward comparison is descriptive because native reward is not ground truth.One judge grades procedure rather than outcome, and procedural-objection counts use a keyword heuristic.Source: Table 8 · p. 17, E Study 3 Details · p. 17
  • On 108 exploratory telecom tasks, seven of eight pairs were equivalent within ±0.10, and no change exceeded rerun noise; one trajectory’s credential-like string requires redaction before release.Laguna under reformatting was inconclusive.Source: E Study 3 Details · p. 17, Table 9 · p. 17
Loading 2610.11678v1…