Source-linked AI summary
Who Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents
Fabio Rovai
TL;DR
Causal action verifiers rely on committed graphs that they do not check, leaving the impact of graph errors and the cost of defending against them open questions. This paper red-teams CIVeX by changing only that graph and tests bounded-experiment attestation. Attestation removes false executions in measured settings, but recovering actions wrongly rejected by a bad graph requires a separate audit.
Problem
CIVeX reported zero false executions on a confounded tool-use benchmark, but the committed graph it trusts is unchecked and the speed, form, and defence cost of its failures were unknown.
Source: §1 · pp. 1–2Method
Results
At the published strength, execution attestation lifts utility from +0.35 to +1.12 and takes false executions to zero, but 97.1% of beneficial actions remain lost.
Source: §6 · p. 5Takeaways & Limitations
Auditing executions removes generated false executions but does not recover beneficial actions rejected by the same graph error; recovering them requires auditing rejections at a cost that scales with rejection volume.
Source: §9 · p. 8Attestation assumes a bounded randomized sample exists and is cheap, an assumption that fails for irreversible actions.
Source: §8 · p. 7
Abstract
from arXiv · showhide
Causal action verifiers gate an agent's state-changing tool calls by checking whether each proposed intervention is identifiable against a committed action-state graph, and they issue a certificate that carries the identification argument and a one-sided lower confidence bound. One such verifier, CIVeX, reports zero false executions on a confounded tool-use benchmark. We red-team it by corrupting only the committed graph. Omitting a single bidirected edge takes it from zero false executions to 15.3% at the benchmark's published confounding strength, with 91% of its executions harmful and utility falling from +2.27 to +0.35. Reversing one arrowhead, so that a mediator is committed as a confounder, gives 48.9% false executions and no correct ones. Every one of these actions carries an internally valid certificate. An attestation step that tests each observationally certified execution against a bounded randomised sample detected both attacks, with 2 false alarms in 555 executions on a truthful graph; refusing what fails the test, or cannot be tested, gave zero false executions in every setting we measured. It does not restore beneficial execution: at the published strength 97.1% of beneficial actions are still never executed, because the same misspecification rejects them before attestation runs. Those rejections carry certificates too, and auditing them works, but its cost scales with the number of rejections rather than the number of executions. Recovering safety costs 127 experiments per 1,050 actions; recovering the lost value costs 614 more, at which point the audited verifier makes the honest graph's decisions on every instance and spends exactly its experiment budget. An audit that inspects only executions protects against wrongful action. Wrongful inaction has to be paid for separately.
1 Introduction
Causal action verifiers assess proposed state-changing calls against a committed action–state graph and issue auditable certificates when actions are identifiable. This paper tests how graph errors affect CIVeX and measures a defence and its costs.
1 Introduction
- Causal action verifiers check identifiability against a committed action–state graph and issue certificates containing the identification argument, estimate, lower confidence bound, and risk limit.They address the gap between validating an action’s form and validating its expected effect.Source: §1 · p. 1
- CIVeX reported zero false executions on a confounded tool-use benchmark, but relied on a committed graph that it did not check.The original work identified the graph as part of the trusted computing base and listed graph injection, adjustment-set poisoning, and version drift as open concerns.Source: §1 · p. 1
- The paper measures how quickly graph misspecification breaks verification, what failures look like, and what a defence costs.It distinguishes a test of declared ignorance from attacks that supply a wrong graph, and studies both harmful executions and paralysis.Source: §1 · p. 2
2 Setup
The experiments change only CIVeX’s committed graph while using its triage procedure, certificate checker, and adversarial benchmark generator. They measure graph attacks under a defined threat model and benchmark setup.
2 Setup
- The study runs CIVeX’s triage procedure, certificate checker, and adversarial Causal-ToolBench generator, changing only the committed graph across 1,050 actions per cell.Each workflow has binary treatment T, continuous utility outcome Y, two observed covariates, and a hidden confounder whose strength is varied.Source: §2 · p. 2
- For identifiable actions, CIVeX estimates effects from 500 observational rows and executes only when the lower end of a 95% interval is at least zero; otherwise, it rejects.When adjustment is unavailable, it experiments if safe experimentation is available (70.6% of instances) and otherwise abstains; each randomized experiment costs 0.05 utility.Source: §2 · p. 2
- The threat model lets an adversary or ordinary mistake alter the committed graph, but not the verifier, estimator, data, or certificate format.The study does not model how graph-editing access is obtained.Source: §2 · p. 2
- At the published strength s = 2.5, the honest graph yields +2.27 utility and zero false executions, compared with +2.23 reported.The honest graph includes observed confounders and a bidirected edge for the hidden confounder; the denied graph differs only by omitting that edge.Source: §2 · p. 2
3 Attack 1: the omitted edge
Omitting the hidden-confounder edge can turn CIVeX’s graph-based identification into harmful execution while leaving its certificates internally valid. The damage depends sharply on confounder strength.
3 Attack 1: the omitted edge
- The denied graph is harmless below strength 1.5, falls sharply between 2.0 and 3.0, and returns negative utility from strength 3.0; the honest graph stays flat.Figure 1 plots the omission attack against hidden-confounder strength, with the published strength and always-abstain floor marked.Source: Figure 1 · p. 3, §3 · p. 3
- With the bidirected edge omitted, backdoor adjustment uses an insufficient set, so the estimate absorbs omitted confounding and the certificate remains internally valid for the wrong graph.The published test instead represents hidden confounding explicitly, causing identification to fail and routing affected actions to experiment or abstention.Source: §3 · p. 3
- At s = 2.5, the omitted-edge graph causes 15.3% false executions, 91% harmful executions, and +0.35 utility, versus +2.27 for the honest graph.At strength 2.0, false executions first appear at 1.5%; from 3.0, 41–47% of actions are false executions and utility is negative.Source: §3 · p. 3, Figure 1 · p. 3
4 Attacks 2 and 3: bad controls and a reversed arrowhead
Two further graph errors have distinct failure modes: admitting a collider as a confounder suppresses correct action, while reversing a mediator’s arrowhead inverts the estimated effect and causes harmful execution.
4 Attacks 2 and 3: bad controls and a reversed arrowhead
- Admitting a collider as a confounder cuts correct executions from 52.9% to 7.9% and utility from +2.79 to +1.39, with zero false executions.Conditioning on a descendant of the outcome makes this bad control fail into reduced action rather than wrongful action.Source: §4 · p. 3
- Figure 2 compares each misspecification with its own correct-graph control because the attacks use different data-generating regimes.The bad control causes paralysis, while the omitted edge and reversed arrowhead cause wrongful action.Source: Figure 2 · p. 4
- With a mediator’s arrowhead reversed, every execution is harmful, every beneficial action is rejected, and utility is negative.The graph includes the real mediator but adjusts for it, blocking the indirect path and recovering the direct effect, whose sign is inverted.Source: §4 · p. 3, Table 1 · p. 4
5 A defence: attestation against a bounded experiment
The paper tests observationally certified executions against a bounded randomised sample, then compares refusal with fallback when the test falsifies the committed graph. The test detects generated harmful executions, but its usefulness depends on response choice and experiment availability.
Attestation procedure
- Attestation draws 500 randomised rows for each observationally identified EXECUTE, compares its 95% interval with the certificate interval, and flags disjoint intervals as graph falsification.Overlapping intervals do not establish that the graph is correct; the test is a discrepancy diagnostic, not a calibrated hypothesis test.Source: §5 · p. 4
Attestation results
- 0 false executions were observed in 1,050 actions after attestation, bounding the true rate below about 0.29% at 95% confidence by the rule of three.Every attested execution from strength 2.0 upward was falsified; zero observed events is not a guarantee.Source: §5 · p. 4
- On the honest graph, attestation never runs because all actions take the experiment path, which uses 741 experiments per 1,050 actions.At strength zero, where the denied graph is correct, attestation flagged 2 of 555 executions (0.4%) and utility fell from +2.79 to +2.76.Source: §5 · p. 4, Table 2 · p. 5
- At strength 1.0, attestation falsified 547 of 554 executions despite harmless denial; refusal yielded +1.05 utility and lost 98.7% of beneficial actions, while fallback yielded +2.76.The authors report that refusal would be worse than no defence across the whole harmless range.Source: §5 · p. 4
Experiment availability
- When only 70.6% of actions had experiments available, honouring availability kept false executions at zero but lowered true-graph utility from +2.79 to +2.29; allowing unattested executions restored false executions.The false-execution rate was 4.3% at strength 2.5 and 12.4% at strength 3.0 when unattestable executions were allowed through.Source: §5 · p. 5
Direction flip
- With an experiment for every action, attestation detected all 513 direction-flip executions; fallback still produced 48.9% false executions, while refusing falsifications gave zero.The fallback reused the committed adjustment set, which blocks the causal path; using the raw difference in means instead brought false executions to zero.Source: §5 · p. 5, Table 3 · p. 5
6 The cost asymmetry
Execution-only attestation removes false executions but leaves beneficial actions lost when the misspecified graph rejects them before attestation. Auditing rejections recovers value, at a substantially higher experiment cost than recovering safety.
6 The cost asymmetry
- At published confounding strength, execution attestation takes false executions to zero and raises utility from +0.35 to +1.12, yet 97.1% of beneficial actions remain lost with fallback and 100% with refusal, versus 28.1% under the honest graph.Figure 3 shows execution attestation leaves those lost actions unchanged; auditing rejections brings beneficial actions back roughly in proportion to experiments spent.Source: §6 · p. 5, Figure 3 · p. 6
- Of 555 beneficial actions at published strength, 539 are rejected at first triage with certificates; execution-only attestation never examines them.The REJECT certificate carries the graph, adjustment set, estimate, and interval, so rejected actions can be tested like executions.Source: §6 · p. 6
- Recovering safety took 127 experiments per 1,050 actions, while recovering value required 614 more; auditing every auditable rejection reproduced the honest graph’s execute decisions and its 741-experiment budget.The reported safety/value experiment split at strength 3.0 was 298 and 443, respectively.Source: §6 · p. 6, Table 4 · p. 6
- Execution-only audits expose wrongful actions but not wrongful inaction: rejected actions change no state and leave no downstream outcome to reveal their loss.Finding lost actions requires re-examining rejections, most of which were correct, so the audit cost scales with rejection volume.Source: §6 · pp. 6–7
7 Related work
The paper situates its attacks among causal identification, safety-constrained decision methods, sensitivity analysis, and verifier robustness. It argues that corrupting a verifier’s trusted graph input is a distinct exposure from challenging the identification theory itself.
Identification and sensitivity analysis
- The attacks hold do-calculus and the Shpitser–Pearl ID algorithm [Pearl, 2009, Shpitser and Pearl, 2006] fixed, corrupting the committed graph rather than challenging identification theory.Backdoor and frontdoor adjustment are described as usual identification arguments.Source: §7 · p. 7
High-confidence deployment
- Per-action certificates connect the verifier to high-confidence off-policy evaluation and the Seldonian framework [Thomas et al., 2015a,b; Thomas et al., 2019], but leave rejections unexamined when audits cover only executions.The paper contrasts an auditable per-action gate with bounds on learned policies.Source: §7 · p. 7
Identification and sensitivity analysis
- Sensitivity intervals for unmeasured confounding could soften the omitted-edge attack, but would not address a direction flip or recover wrongly rejected actions.The paper connects its collider and reversed-mediator attacks to the good- and bad-control taxonomy [Cinelli et al., 2024].Source: §7 · p. 7
Verifier robustness
- Unlike reward-hacking work on agents exploiting evaluators [Amodei et al., 2016, Skalse et al., 2022], this paper examines corruption of evaluators’ trusted inputs.The authors describe that exposure as growing as verification uses structured, assumption-carrying artefacts.Source: §7 · p. 7
8 Limitations
The evaluation is synthetic, and its attacks and defence rely on assumptions that limit how far the results generalize. It measures damage from graph corruption, not the likelihood or difficulty of causing it.
8 Limitations
- The synthetic benchmark demonstrates that observational and interventional signs can disagree but does not claim to match a production system.The collider and mediator regimes are designed to expose their respective failure modes.Source: §8 · p. 7
- Attestation assumes a bounded randomised sample exists and is cheap, excluding irreversible actions where verification matters most; its disjoint-interval test has no calibrated error guarantee.Source: §8 · p. 7
- The attacks directly edit the graph, so the study measures corruption damage rather than attacker access or how often elicited graphs contain such errors.The rejection audit is uniform; a targeted audit might recover value more cheaply, but is untested.Source: §8 · p. 7
- The study evaluates one verifier on a synthetic benchmark, not real tool-agent systems, and does not test whether other verification schemes share the safety–value cost split.Source: §8 · p. 7
9 Conclusion
The conclusion is that causal action verifiers trust a committed graph they cannot check, so graph corruption can yield internally valid certificates for wrong executions. Attestation removes generated false executions, but recovering beneficial actions requires separately auditing rejections.
9 Conclusion
- One omitted edge or reversed arrowhead can make certified executions wrong while leaving their certificates internally valid.The verifier certifies actions against a graph it cannot check.Source: §9 · p. 7
- Attestation removed every generated false execution at a cost proportional to the number of executions, but recovered none of the beneficial actions rejected by the same graph error.Source: §9 · p. 8
- Auditing rejections can recover beneficial actions, but its cost scales with the number of rejections and recovering all of them costs what the honest graph would have spent.The authors argue that verification work should address wrongful inaction as well as wrongful action.Source: §9 · p. 8
A Benchmark, verifier and defence specification
The benchmark specifies causal regimes and a verifier that makes decisions using the committed graph, with attestation and audit procedures evaluated against those decisions.
A Benchmark, verifier and defence specification
- Each instance draws a binary label b and an effect τ across six families with family-specific beneficial and harmful effect means.The label is b ∼ Bernoulli(0.5), and τ ∼ N(µ_b, 0.42).Source: A Benchmark, verifier and defence… · p. 9
- At the published confounding strength, hidden U biases observational contrasts toward the wrong sign; safe randomised experiments are available for 70.6% of instances.The randomised sample draws treatment independently of the covariates, U, and noise.Source: A Benchmark, verifier and defence… · p. 9
- The mediator regime has no hidden confounder, and its direct effect has the opposite sign from its total effect for both beneficial and harmful actions.The stated total effects are +2.5 for beneficial actions and −2.0 for harmful ones.Source: A Benchmark, verifier and defence… · p. 9
- The verifier searches the committed graph for backdoor sets, uses least squares and a 95% interval, and executes only when the lower bound is nonnegative and the risk bound meets its threshold.Attestation compares randomised-sample and certificate intervals; rejection audits sample eligible observational rejections, and each experiment costs 0.05 utility.Source: A Benchmark, verifier and defence… · p. 9
B Per-seed variability
Table 5 reports seed-level spread for the headline cells.
B Per-seed variability
- Table 5 presents seed-level spread for the headline cells.Source: Table 5 · p. 10