Source-linked AI summary
Look Before You Leap: Pre-Action Verification for LLM Agents
Asaad Althoubi
TL;DR
LLM agents can silently produce incorrect effects when shell commands or code edits appear to succeed, exposing a gap in ordinary oversight. The paper fixes intended effects by construction and applies cheap deterministic, abstention-capable checks before execution across both modalities. It finds that oracle-exact checks avoid false positives, content-anchored edits fail cleanly, and pre-action guards can trade applicability for sharply reduced silent failure.
Problem
LLM-agent actions may produce plausible but incorrect effects without errors, while introspection and post-hoc checks cannot reliably detect such silent failures.
Method
The paper fixes each action’s correct effect before execution and evaluates deterministic pre-action verification with success, clean-failure, and silent-failure outcomes across commands and code edits.
Results
Across both modalities, oracle-exact checks avoid false positives, content-anchored edits fail cleanly, and abstention-based guards sharply reduce silent failures at a tunable applicability cost.
Takeaways & Limitations
Action interfaces that expose oracle-checkable properties and content anchors support cheap, model-agnostic guards that convert silent errors into recoverable refusals.
Takeaways & Limitations
The synthetic perturbations may not match real model-error distributions, and the limited real-model probe and single Linux environment constrain estimates of deployment transfer.
Abstract
from arXiv · showhide
An LLM agent acts on the world by emitting actions: shell commands to run, edits to apply. A wrong action does not always fail loudly; it can fail silently, producing a plausible but incorrect effect that raises no error. We argue that a cheap deterministic check, run before an action takes effect, is an effective and underused form of agent oversight, and we study it across two action modalities in one framework. The idea is to fix an action's correct effect by construction, before any executor runs, so that silent failure is measured directly and the verifier may abstain rather than guess. For shell commands, a static verifier over 9930 commands and 482 tools catches 95.8% of invalid commands at a 10.0% false-positive rate. Its syntax and binary checks are oracle-exact, giving zero false positives while catching half of all errors; the flag check is bounded only by help-text coverage and accounts for every false positive. For code edits, a benchmark of 640 edits over 224 files isolating the apply step exposes a sharp split. Content-anchored formats such as search/replace and diff fail cleanly, whereas location-anchored formats fail silently: line numbers corrupt 99.1% of files under a one-line shift, and function-name edits hit the wrong function 12.7% of the time. In both settings a refuse-when-unsure policy turns silent failures into recoverable ones at a tunable cost in applicability: selective grounding reaches 0.958 recall at 7.0% false positives, and an anchor-and-verify applier records one silent misapplication in 8320 trials (0.01%). We release both benchmarks, the verifiers, and the guards.
1 Introduction
The paper frames silent failures in LLM-agent actions as an oversight problem and proposes cheap deterministic checks before execution. It evaluates a shared pre-action verification framework across shell commands and code edits, with abstention converting uncertain cases into recoverable outcomes.
- 1 Introduction: A silent failure occurs when an action appears to succeed but produces the wrong effect without alerting the agent or downstream checks.Such errors can propagate through long, unattended agent trajectories.
- 1 Introduction: Pre-action verification admits or refuses actions before execution, providing a model-agnostic oversight mechanism that avoids relying on introspection or post-hoc detection.Refusal returns a recoverable signal rather than allowing an unobserved error to propagate.
- 1 Introduction: The framework fixes each action’s correct effect by construction and classifies outcomes as success, clean failure, or silent failure.This construction makes silent failure directly measurable across action modalities.
- 1 Introduction: The study benchmarks shell-command verification and code-edit application, separating oracle-exact checks from coverage-bounded checks and comparing content-anchored with location-anchored representations.The contributions include benchmarks, verifiers, and deployable guards for both modalities.
- 1 Introduction: The work positions these guards as safe-by-design oversight that quantifies failures ordinary error handling misses and supplies practical evaluation tools.The released benchmarks and reproducible code support empirical robustness evaluation of LLM-agent actions.
2 Pre-Action Verification: A Common Framework
The common framework verifies whether an action can be realized validly and unambiguously before execution, using construction-defined targets and allowing abstention. It measures safety separately from applicability, trading recoverable over-refusal for prevention of silent corruption.
- 2 Pre-Action Verification: A Common Framework: A verifier inspects the action and local environment before execution, then either admits or refuses it without evaluating whether the intended change is semantically correct.Its scope is realizability and unambiguity, not intent or semantic correctness.
- 2 Pre-Action Verification: A Common Framework: Construction-defined targets let the framework label each trial as success, clean failure, or dangerous silent failure independently of the executor.Commands use behaviorally independent validity oracles, while edit targets are synthesized before rendering and perturbation.
- 2 Pre-Action Verification: A Common Framework: The taxonomy separates safety from applicability, making the silent rate among admitted actions the direct safety measure.This avoids conflating refusal behavior with detection accuracy.
- 2 Pre-Action Verification: A Common Framework: Abstention thresholds trace an applicability–safety frontier, with oracle-exact checks producing zero false positives and coverage-bounded checks carrying residual errors.The deployed gate can maximize applicability subject to a safety floor.
- 2 Pre-Action Verification: A Common Framework: A command verifier checks the executable, syntax, and flags, while an abstaining flag check can avoid rejecting valid aliases absent from extracted help text.The worked grep example illustrates why withholding judgment is preferable to a false alarm.
- 2 Pre-Action Verification: A Common Framework: Pre-action refusal prevents unverifiable actions from taking effect, exchanging recoverable false alarms for prevention of unrecoverable silent corruption.This is the central design trade-off shared by the paper’s guards.
3 Grounding Shell Commands
The shell-command verifier combines exact syntax and binary checks with a coverage-bounded flag check, exposing where false positives arise. Selective grounding preserves recall while reducing false positives by abstaining on ambiguous flags.
- Verifier design and decomposition: Syntax and binary checks form an oracle-exact core with zero false positives and recall 0.500, while the flag check supplies the remaining detection.The core catches malformed syntax and nonexistent-binary errors exactly; the flag check is the only source of false positives.
- Benchmark result: 95.8% of invalid commands are detected at a 10.0% false-positive rate across 9,930 commands and 482 installed tools.The benchmark uses independently oracle-confirmed mutations across nonexistent binaries, malformed syntax, and invalid long and short options.
- Recall by error category: Invalid nonexistent-binary and malformed-syntax commands are caught at recall 1.000 each, while invalid long and short flags are each caught at 0.916.Flag recall is bounded by 91.1% help-text extraction coverage rather than by the verifier’s structural method.
- Selective grounding and operating points: 0.958 recall at 7.0% false positives, versus 95.8% recall at 10.0% for the full configuration; selective grounding Pareto-dominates the naive policy.It abstains on ambiguous single-dash multi-character options, which removes false positives without excluding injected invalid errors.
- False-positive sources: All 198 false positives are valid flags missed during extraction, with the ambiguous single-dash multi-character class comprising 29.8% of them.The remaining errors are extraction gaps distributed across 95 tools, so richer help-text parsing or per-tool completion specifications are the nearest repair.
4 Applying Code Edits
The benchmark isolates whether edit appliers reach fixed target files under realistic drift, revealing a sharp safety divide between content- and location-anchored formats. Refuse-when-unsure verification nearly eliminates silent misapplication, with a tunable applicability cost.
- Benchmark: 640 edits across 224 Python files are rendered into four formats and subjected to reindentation and line-shift perturbations, producing 23,040 apply-step trials.The setup separates generating the intended change from applying it and includes search/replace, unified diff, line ranges, and whole-function-by-name.
- Safety dichotomy: 0.000 silent misapplication for content-anchored search/replace and unified diff contrasts with 99.1% corruption for line ranges after a one-line shift and 12.7% wrong-function edits unperturbed.The benchmark fixes each target before applying edits, making wrong-location outcomes directly observable; Figure 2 summarizes the applicability–safety trade-off.
- Safety is format and policy: 0.020 exact and 0.023 whitespace-normalized silent rates under permissive first-match policies show that zero silent failure requires refusing ambiguous matches.Safety therefore depends on both the edit format and the applier policy, not either factor alone.
- Multi-hunk and generalization: 39% silent failure for multi-hunk fuzzy matching emerges as per-hunk anchors shrink, while exact, whitespace-normalized, and diff matching remain safe and line ranges remain catastrophic.Unified diff stays silent-free because patch validates each hunk’s context independently, though it has the lowest applicability under heavy reindentation.
- Robust-Apply: 0.01% silent misapplication across 8,320 trials is achieved by Robust-Apply, which verifies anchors, rejects ambiguity, and commits accepted fuzzy matches through real patch.With parameters (Kmin, τhi, δ)=(2, 0.90, 0.20), applicability is 0.51 versus 0.98 for perturbed single-hunk edits; across τhi ∈[0.85, 0.95] and δ ∈[0.10, 0.30], silent rate stays at or below 0.05%.
- Operating points: Choosing an operating point depends on whether applicability or avoidance of silent edits matters more and whether reliable downstream tests, review, or self-repair exist.Exact or whitespace-normalized matching is presented as the simplest zero-tuning safe default.
5 Cross-Domain Synthesis
Across shell commands and code edits, cheap pre-action checks combine an oracle-exact core with abstention to convert silent failures into recoverable clean failures. The synthesis favors content-checkable action representations while recognizing that semantic correctness remains outside these guards' scope.
- A cheap check with an oracle-exact core and abstention converts silent failures into recoverable clean failures across both action modalities.The exact components provide a zero-false-positive safety floor, while softer checks trade applicability against residual error.
- Action representations are safest when their effects can be matched against content; location-only anchors such as line numbers or function names can fail silently.The dangerous representations lack surrounding content that distinguishes a correct target from a wrong one.
- The proposed oversight principle is to combine an oracle-exact blocker with an abstaining soft check and tune applicability to the recoverable failure a deployment can tolerate.This design is model-agnostic, auditable, and incurs no model inference call.
- The recipe extends to file operations, schema-validated API calls, and staged database writes when they expose checkable properties and content anchors.The recurring design question is whether a violated property can be identified definitively before the action takes effect.
- Pre-action verification guards realizability rather than semantic correctness, leaving harmful valid commands and logically incorrect edits to tests, review, or learned critics.It is therefore narrower than sandboxing, rollback, or model-based review, despite being cheaper and more predictable.
- Both guards run inline without model calls, GPU, or network, and abstentions return structured signals that agents can use to retry or repair.The command verifier's cached median latency is 2.55 ms, while Robust-Apply is dominated by one similarity scan.
6 Threats to Validity
The evaluation uses synthetic, construction-defined commands and edits, so its failure mechanisms may not match real model-error distributions. Deployment and generalization are further bounded by tool, environment, model-sampling, and language-setting limitations.
- Synthetic invalid commands and perturbed edits are confirmed by oracles or fixed by construction, but need not match real model-error distributions.The authors argue that location drift and extraction gaps remain intrinsic to the tested representations.
- Only 42 commands and 9 valid edits from one Anthropic Claude model were probed, limiting evidence about recall under real error distributions.The authors identify multi-model studies over agent trajectories as the key next step.
- The results come from one Linux host with Python and shell tooling, so exact rates may not transfer to other languages, larger functions, or non-function code.The authors expect the qualitative dichotomies to be more robust than the precise rates.
- Syntax and binary checks are exact on complete commands, but bash -n rejects incomplete fragments and which misses aliases, shell functions, and session-updated PATH entries.A deployed gate should treat these cases as abstentions.
7 Related Work
The paper targets a gap in agent evaluation by isolating action validity and application steps rather than measuring only end-to-end task success. It positions its approach as deterministic realizability verification that complements, rather than replaces, semantic oversight and model verification.
- Unlike agent benchmarks that mainly measure end-to-end task success, this work directly isolates action-validity and apply steps with construction-defined ground truth.That design makes silent failure observable and separates why an action fails.
- Tool- and API-use studies examine selecting and calling the right function, whereas this paper evaluates whether emitted actions are validly and unambiguously realizable.The distinction concerns the action-checking layer rather than model behavior alone.
- The work is a narrow, deterministic instance of scalable oversight that verifies action realizability against the environment, not properties of the model.It complements formal verification approaches such as SMT-based checking of learned components.
8 Conclusion
The paper concludes that construction-defined evaluation makes silent action failures directly measurable and reveals a shared structure across shell commands and code edits. Its broader conclusion is that content-checkable interfaces enable cheap, model-agnostic safeguards, while ungroundable representations preserve silent risk.
- Construction-defined targets turn silent failure from an inferred symptom into a directly measured quantity across shell commands and code edits.This supports a common comparison of safety-relevant action behavior.
- The checks split into an oracle-exact core with no false alarms and a softer, coverage-bounded remainder that carries all the error.The paper identifies this pattern as common to both modalities.
- The resulting guards are model-agnostic, run in microseconds to milliseconds, and are released with benchmarks for deployment before agent actions.The conclusion frames them as practical front-end safeguards.
- The wider lesson is to design agent interfaces around actions that can be checked against content, because ungroundable representations leave silent failures harder to recover downstream.The conclusion presents this as the reason the method may extend beyond the two studied modalities.
Ethical Statement
The work is presented as defensive: its guards prevent silent file corruption and unintended command execution, while refusal wastes a step rather than damaging state.
- The guards prevent silent file corruption and unintended command execution, with refusal as a non-damaging failure mode.The benchmarks use public tools and source code without human subjects or personal data.