Source-linked AI summary
A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging
Peiying Zhu, Sidi Chang
TL;DR
A verifier can make an optimizer comparison vacuous when its probes or predicates disclose the target instead of preserving genuine ambiguity. This paper replaces solver-first evaluation with a support-gated verifier that checks evidence before returning a signal; in the heldout, false admission met its frozen safety statement and affected clean traffic was the better power coordinate.
Problem
Method
The paper uses a support-gated verification contract that checks clean reference exposure and matched runtime evidence before an independently calibrated signal rule returns a detection.
Source: §3.1 · p. 2, §3.2 · p. 3, §10 · p. 6Results
0 stable false admissions among 20 represented components had a one-sided exact 95% upper bound of 0.1391, and affected clean traffic was the better primary power coordinate within admitted formal units.
Source: §7.2 · p. 5, §7.3 · p. 5Takeaways & Limitations
Reliable agent development requires verifying that the evidence generator is eligible and non-revealing before trusting an algorithm that optimizes its output.
Source: §10 · p. 6The study uses one simulator, one closed-loop policy family, three demand regimes, and 24 disjoint components, and does not claim its support thresholds transfer to language-agent benchmarks.
Source: §9 · p. 6
Abstract
from arXiv · showhide
Agent developers increasingly compare prompts, tools, policies, and diagnosis algorithms through simulator-grounded verifiers. A verifier can nevertheless make a solver comparison vacuous: if its probes or predicates encode the target identity, an exact optimizer may appear effective without resolving any genuine ambiguity. We report such a failure in an aggregate-trace debugger for a closed-loop decision agent. Exact minimum hitting set (MHS) and a propagation-aware greedy method returned identical supports in 12/12 development cases and the same planted-fault recovery in 9/12. A subsequent audit found that exact-anchor predicates produced the planted pair in 9/9 cases. After removing those anchors, overall planted-pair recovery was 8/9; hard-probe singleton pairs nevertheless matched the planted pair in 9/9, and no case retained a nonempty residual conflict family after propagation (0/9). The optimizer was correct, but the verifier had already disclosed the answer. We replace solver-first evaluation with a support-gated verification contract. A clean reference map must first show repeated component exposure; a matched reference/current gate must then establish comparable runtime evidence; only afterward may an independently calibrated signal rule return a detection. In a preregistered heldout comprising 1,440 cases and 21,600 partition rows, 55/72 regime-component units passed the reference gate, 54/55 passed the runtime gate, and stable false admission was 0/20 represented components with a one-sided exact 95% upper bound of 0.1391. Within admitted units, affected clean traffic predicted detection better than nominal fault-cell fraction. The main lesson is structural: verify evidence eligibility and non-revelation before optimizing the component selector. Otherwise a stronger solver can merely certify a stronger verifier artifact.
1 Introduction
In a controlled closed-loop debugger, the audit found that the verifier had already disclosed planted faults, making the solver comparison vacuous. The paper replaces solver-first evaluation with a contract that checks evidence eligibility and non-revelation before optimization.
Setting
- The study uses aggregate traces from a closed-loop decision agent to localize planted faults among policy components, with exact MHS compared against weighted greedy selection.The original development claim was that exact MHS would improve localization over greedy selection.Source: §1 · p. 1
Audit and contribution
- The exact-MHS advantage claim was withdrawn after an audit showed that exact-component predicates and one-component hard probes created planted-component singleton conflicts.After propagation, neither exact nor greedy selection had a substantive choice; the failure was in the verifier, not MHS mathematics.Source: §1 · p. 1
Verification principle
- A verifier must establish that evidence is eligible and non-revealing before its score can support an optimizer comparison.Eligibility requires sufficient policy exposure and comparable observations; non-revelation requires interventions that do not encode the hidden answer.Source: §1 · p. 1
Replacement verifier
- The replacement contract uses a clean reference-map gate, a matched runtime two-stream gate, and independent calibration of stable false admission.The heldout admitted 55/72 units at reference, rejected one more represented component at runtime, and found no stable false admissions among 20 represented components.Source: §1 · p. 2
Scope
- The heldout intentionally neither compares exact MHS with greedy selection nor establishes localization accuracy.The experiment instead tests whether the verifier can identify when its evidence is usable.Source: §1 · p. 2
2 From environment-grounded evaluation to verifier-grounded claims
Environment-grounded evaluation improves on static answer matching, but grounding alone does not ensure that a verifier supports the claim being scored. This section connects verifier validity to diagnosis, intervention leakage, and evidence-based abstention.
Verifier-grounded claims
- A deterministic or reproducible verifier can still fail to support its intended claim if its operationalization omits requirements, exposes privileged state, rewards shortcuts, or alters what it measures.The relevant question is whether the operationalization supports the intended claim, not only whether it is reproducible.Source: §2 · p. 2
Diagnosis framing
- In model-based diagnosis, conflicts are derived from a system model and observations before candidate diagnoses are computed as hitting sets.Exact search improves optimization resources without changing this conditional structure.Source: §2 · p. 2
Intervention leakage
- A single-component counterfactual hard probe can leak the target when paired with a predicate scoped to that same component.The analogous benchmark failure is writing a hidden label into an observable field and rewarding the agent for reading it.Source: §2 · p. 2
Abstention and eligibility
- The verifier abstains before diagnosis when the policy has not produced enough matched evidence; this is an evidence-eligibility test, not a confidence threshold on a completed prediction.Reference mismatch can trigger review or retraining rather than silently extending the original claim.Source: §2 · p. 2
3 Closed-loop debugging setting
The study evaluates a closed-loop policy through aggregate traces, directional fault interventions, and a downstream conflict-to-hitting-set pipeline. Its exposure measure reflects clean visits to selected cells, while the optimizer's guarantee remains conditional on the compiled conflicts.
3.1 Agent and observable components
- The formal study fits a closed-loop target policy in a simulator, partitions it into 24 components, and evaluates three demand regimes using typed aggregate traces rather than planted-component identity.The regimes yield 72 regime-component units; the signal calculation uses 15 independent partitions per unit and refits the policy per regime.Source: §3.1 · p. 2
3.2 Fault interventions and exposure
- Faults shift selected direction-changeable cells by one action bucket, and directional affected traffic weights selected cells by their clean visit counts.The primary bidirectional exposure averages outward and inward exposure; zero clean occupancy gives zero exposure. The two frozen mask families are uniform and flow-weighted.Source: §3.2 · pp. 2–3
3.3 Conflict compilation and downstream optimization
- The diagnostic compiles aggregate evidence and scoped predicates into conflicts, then exact MHS or weighted greedy chooses a low-cost set intersecting every conflict.The exact guarantee applies to the compiled conflict family; it does not establish preserved physical ambiguity, planted-component recovery, or an advantage over greedy on verifier-produced instances.Source: §3.3 · p. 3
4 Failure audit: the verifier had already solved the task
The failure audit found that apparently structured conflicts still forced identical solver outputs because the verifier's probe and predicate encoded the planted component. It therefore calls for auditing both non-revelation and evidence eligibility before treating optimization as a meaningful comparison.
Development evidence
- Despite 55.5% non-singleton conflicts and overlapping conflicts in all 12 scenarios, exact MHS and weighted greedy returned identical supports in every scenario.These were audited development results, not formal heldout endpoints.Source: §4 · p. 3
Leakage mechanism
- A predicate and hard probe scoped to the same named component produced singleton conflicts naming the planted component, so removing an explicit anchor did not remove the forced choice.After propagation, both solvers inherited the same choice.Source: §4 · p. 3
Audit requirements
- A meaningful audit checks whether upstream transformations encode target identity, whether propagation leaves residual decisions, whether traces support the component, and whether reference and current traces are comparable.The first two checks concern non-revelation; the latter two concern evidence eligibility, and both classes are needed.Source: §4 · p. 3
5 A support-gated verification contract
The verification contract gates diagnosis on clean component exposure and matched runtime evidence, then applies an independently calibrated signal rule. It keeps execution, safety, and structural claims separate rather than collapsing them into one verdict.
5.1 Reference-map gate
- At least 14 of 15 clean partitions must support a regime-component unit for it to enter the reference map; otherwise it returns REFERENCE_ABSTAIN.The gate screens all 72 units without using planted-fault identity or inferring a diagnosis.Source: §5.1 · p. 3
5.2 Runtime two-stream gate
- At least 14 of 15 partitions must have reference and current support of 12 or more in the same partition; otherwise the unit returns RUNTIME_ABSTAIN.Bidirectional observations require both directions to pass; matching partitions ensures the aggregate difference uses comparable evidence.Source: §5.2 · p. 3
5.3 Stable signal and independent null calibration
- A direction signals at z ≥1.50 in at least 14 of 15 matched partitions, and both directions must be stable for unit-level detection.The cutoff was selected before formal execution from a fixed grid and was not retuned on the formal null; false admission is calibrated on a separate clean-null schedule with disjoint seeds, using physical components as the sampling unit and a one-sided exact 95% upper-bound threshold of 0.20.Source: §5.3 · pp. 3–4
5.4 Result state and claim separation
- The protocol separates execution, operational safety, and structural hypotheses, with no combined scientific PASS/FAIL label.This prevents a favorable endpoint from being treated as permission to ignore an unfavorable one or to change the verifier because an unrelated endpoint missed its threshold.Source: §5.4 · p. 4
6 Preregistered heldout
The preregistered heldout uses a frozen, independently validated protocol and separates coverage and safety, traffic-indexed power, and structural tests. Its purpose is to test the replacement verifier, not compare solvers.
6 Preregistered heldout
- The formal schedule contains 1,440 cases, 21,600 partition rows, and 3,456,000 simulator episodes; episodes are Monte Carlo trajectories, not independent statistical units.Coverage uses 72 regime-component units, while safety and bootstrap inference use 20 represented physical-component clusters.Source: §6 · p. 4
- Development, pilot, and formal seeds are disjoint, and the frozen protocol, code, configuration, environment, schema, and validator are locked to a git commit.Reference-gate failures remain scheduled so schedule validation does not depend on observed eligibility.Source: §6 · p. 4
- The formal analysis tests coverage and safety, traffic-indexed power across two mask families, and clustered structural comparisons; it does not compare MHS with greedy selection.The confirmatory experiment tests the replacement verifier's admission, calibration, and signal behavior.Source: §6 · p. 4
7 Results
The heldout found affected clean traffic more informative about detection than nominal selected-cell fraction, while the verifier admitted supported units and met its frozen false-admission criterion. Detection counts were nondecreasing across traffic targets in both mask families.
7 Results
- Independent validation passed all 56 formal checks, so the protocol was labeled EXECUTED while scientific endpoints retained separate labels.Source: §7 · p. 4
7.1 The verifier refuses unsupported units
- The clean reference gate admitted 55 of 72 regime-component units (76.4%), and 54 of 55 reference-admitted units remained runtime-admitted.The 55 units represented 20 of 24 physical components; runtime rejected one represented component, while reference-map invalidation was 0/20. Abstentions indicate insufficient evidence from the deployed policy, not missing data to impute.Source: §7.1 · p. 4
7.2 Stable false admission met the frozen safety statement
- 0 stable false admissions occurred among 20 represented components; the one-sided exact 95% upper bound was 0.1391, below the frozen 0.20 threshold.The zero count does not establish a true rate of zero; the upper bound is the claim.Source: §7.2 · p. 5
7.3 Agent traffic, not edit size, governs verifiability
- Bidirectionally stable detection counts were nondecreasing across five traffic targets: uniform 0, 17, 37, 52, 54; flow-weighted 0, 13, 35, 49, 54.A counterfactual edit can affect cells the agent never visits and remain nearly invisible, or affect few high-traffic cells and become easy to detect.Source: §7.3 · p. 5
- 0.1264 nats per unit-arm row: replacing affected clean traffic with nominal selected-cell fraction increased negative log likelihood.The two-sided 95% component-cluster interval was [0.0593, 0.1918]. Adding mask family and traffic-by-family interaction improved log loss by 0.0015 nats per unit-arm row, meeting the frozen 0.01-nat practical-sufficiency rule; this does not establish coefficient-scale equivalence.Source: §7.3 · p. 5
8 Implications for reliable agent development
The discussion argues that reliable agent evaluation must audit how evidence is compiled and what interventions reveal, while treating abstention and distinct claims explicitly. It also describes a claim ledger and a reproducibility artifact.
8.1 Verify the compiler, not only the scorer
- Upstream transformations can leak a target or erase ambiguity even when the final scorer is formally correct, so compiler audits should test residual ambiguity after forced facts and propagation.If every residual candidate family is empty or singleton, a solver benchmark cannot support optimizer-quality claims.Source: §8.1 · p. 5
8.2 Separate intervention from measurement
- Passive observation, active information gathering, and counterfactual intervention are distinct; interventions may change the system and encode their target in the response.Benchmarks using privileged tools, exposed database state, or injected faults should document each probe's role.Source: §8.2 · p. 5
8.3 Treat abstention as a verifier output
- When an agent never enters the state region needed to test a requirement, a support gate makes “not testable here” a first-class outcome.Reporting failure confounds lack of capability with lack of evidence, while reporting success is worse.Source: §8.3 · p. 5
8.4 Keep claims modular
- Execution integrity, false admission, coverage, power, and solver superiority are distinct claims; a claim ledger keeps their regressions local.The reproduction artifact includes the frozen protocol, code, simulator source, and saved data, and independently reconstructs 56 validation checks and associated analyses without generating trajectories.Source: §8.4 · p. 5
9 Limitations
The study’s claims are limited to a specific simulator and policy setting, and its heldout does not establish planted-fault localization or behavior on genuinely ambiguous conflict families. Several calibration results also have finite-sample caveats, while the refresh test does not establish repair after real drift.
9 Limitations
- The study covers one simulator, one closed-loop policy family, three demand regimes, and 24 disjoint components; its support thresholds are not claimed to transfer to language-agent benchmarks.The contribution is a verification structure and a falsified optimizer claim, not a universal threshold.Source: §9 · p. 6
- The formal heldout confirms admission, false-admission calibration, and traffic-indexed signal behavior, but not planted-fault localization or genuinely ambiguous residual conflict families.Exact MHS remains a valid risk-cost optimizer when such families exist; this study shows only that the audited verifier failed to produce them.Source: §9 · p. 6
- With approximately 20 represented component clusters, the family-sufficiency bootstrap may undercover favorably, and its 0.0066 upper bound is only 0.0034 nats below threshold.The passage cautions that a percentile cluster bootstrap at this cluster count may undercover in a direction favorable to the result.Source: §9 · p. 6
- The refresh endpoint is only a state-machine smoke test using the same frozen reference buffer, so it cannot establish repair of a stale map after real drift.Source: §9 · p. 6
10 Conclusion
The audited verifier made the optimizer comparison vacuous by forcing planted-component singletons, leaving exact MHS without genuine ambiguity to resolve. The paper replaces the solver-first claim with support-gated verification and calls for a new optimizer comparison only after residual ambiguity is demonstrated.
10 Conclusion
- Exact-scope predicates and one-component interventions produced forced planted-component singletons, so exact MHS never faced the ambiguity it was meant to resolve.Source: §10 · p. 6
- The paper replaces solver-first evaluation with a support-gated contract that checks whether the agent generated comparable evidence before returning a signal.Source: §10 · p. 6
- A future optimizer comparison should begin only after a redesigned verifier demonstrates nonempty residual ambiguity.Reliable agent development requires verifying the evidence generator before trusting the algorithm that optimizes its output.Source: §10 · p. 6