When May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMs

• Stopping too early can turn an unsupported intermediate state into a final answer, while continuing indefinitely wastes calls and can compound risk.Source: Introduction · p. 1
• Common DONE signals, checklists, heuristics, and plausibility judgments do not necessarily identify claim-supporting observations, requested scope, or replayable derived values.The gap matters because termination is part of the control system and terminal evidence may support later assurance arguments.Source: Introduction · p. 1
• ECT separates certificate proposal from stop authorization by checking a typed certificate against a trusted task contract and immutable evidence ledger.Task and ledger digests bind the certificate to the trace; receipt checks validate value path, status, and scope.Source: Introduction · p. 1
• ECT combines digest binding, receipt-level validation, and exact closed-transform replay, while failed checks return continue with reason codes.Source: Introduction · pp. 1–2
• The contribution is completion-specific evaluation across a six-family, 48-task benchmark with controlled faults and frozen comparisons against critic-based baselines.Source: Introduction · p. 2

• Earlier systems gate DONE on executed checks, audited task state, termination verifiers, or certificate-backed coverage, so ECT does not claim the first successful-completion gate.Source: Related Work · p. 2
• Certificate-oriented systems already address evidence, hashes, scope, replay, claim decomposition, emission gates, and conformance obligations.Source: Related Work · p. 2
• ECT contributes a narrower completion-specific integration of required-slot contracts, receipt ledgers, digest-bound certificates, exact scope and cardinality checks, and closed-transform replay.Source: Related Work · p. 2
• Existing evaluation work scores final answers, provenance, final state, trajectory artifacts, solvability, missing information, or help seeking; ECT studies unsupported authorization of positive COMPLETE.Source: Related Work · p. 2

• A certificate contains task identity, descriptor and ledger digests, and claims pairing each required slot with a value, evidence IDs, and a transform.Source: Contract, Ledger, and Certificate · p. 2
• ECT uses a closed, non-executable transform language covering collection, aggregation, differences, ratios, extrema, sorting, set operations, top-k, abstention, and windowed differences.Source: Contract, Ledger, and Certificate · p. 3
• The verifier rejects missing or extra slots, duplicate or conflicting records, invalid receipts, scope mismatches, impermissible transforms, and replay discrepancies.Source: Deterministic Verification · p. 3

Deterministic Verification

• COMPLETE is returned if and only if all required claims pass every check; verifier failures return CONTINUE, while STUCK requires a separate recovery-availability rule.Source: Deterministic Verification · p. 4
• For fixed requirements, evidence, and certificate, the verifier is deterministic, order-independent in ledger hashing, and locally reproducible with reason codes.Source: Deterministic Verification · p. 4

Relative Soundness

• If verification accepts, every required slot is present, cited receipts are valid and in scope, and replay reconstructs each certified value within tolerance.Source: Deterministic Verification · p. 4

Trust Boundary

• The guarantee assumes adequate requirements and faithful adapter records; a malicious adapter or incomplete contract can yield internally consistent but incomplete support.Source: Trust Boundary · pp. 4–5
• ECT certifies reconstructibility from the bound trace under declared assumptions, not external truth, authenticity, effects, safety, or alignment.Source: Trust Boundary · p. 5

Tasks, Faults, and Split

• Eight generated worlds cover lookup, aggregation, top-k, temporal comparison, hierarchy join, and missing-data abstention, with 36 held-out tasks and dual independent oracles.Source: Tasks, Faults, and Split · p. 5
• Each base task has one clean snapshot and eight controlled faults, producing 48 × 9 = 432 snapshots, including 288 held-out fault snapshots.Source: Tasks, Faults, and Split · p. 5
• Table 1 applies the clean condition and eight faults to every base task, with Gold denoting the permitted terminal decision at each snapshot.Source: Table 1 · p. 5
• Fault construction changes certificates, receipts, or recovery state while retaining task descriptors and gold answers, isolating declared termination defects without assuming only one verifier predicate fires.Source: Tasks, Faults, and Split · p. 5

Policies and Frozen Inputs

• The comparison includes six baselines and ECT, with fixed sources, prompts, models, decoding, parsers, retries, and failure mappings before scoring.Source: Policies and Frozen Inputs · pp. 5–6

Endpoints and Inference

• The primary endpoint is unsafe completion rate among 288 held-out fault snapshots, analyzed with paired task indicators and task-cluster bootstrap intervals.Source: Endpoints and Inference · p. 6
• Clean false continuation is reported separately, while secondary endpoints include recovery, quality, fault breakdowns, resource use, and reason localization.Source: Endpoints and Inference · p. 6
• Single-check ablations are descriptive, and the exact-transform-contract ablation lacks a matched frozen fault, limiting its interpretation.Source: Endpoints and Inference · p. 6

Static Evaluation

• 0/288 unsafe completions occurred with ECT versus 252/288 for the critic core, a difference of −87.50 pp with a 95% task-cluster interval of [−87.50, −87.50] pp.H1 passed; the six-world sensitivity interval was also [−87.50, −87.50] pp.Source: Static Results and Ablations · p. 6
• 0 clean false continuations occurred for both ECT and the comparator across 36 clean snapshots.Clean false continuation was reported separately from unsafe completion.Source: Static Results and Ablations · p. 6

Ablations

• Single-check ablations localized validation families containing their matched injected faults while retaining unmatched and residually caught cases.These ablations were descriptive and did not establish that any validation check was unnecessary.Source: Static Results and Ablations · p. 6, Table 3 · p. 7

Study Design

• 576 trajectories were generated from 24 held-out tasks, two checkpoint conditions, four policy pipelines, and three seeds.The study used worlds 10–13, with twelve fault assignments and matched nominal inputs rather than same-proposal interventions.Source: Closed-Loop Confirmation · p. 7

Study Integrity

• All 1289 persisted attempt records succeeded, with no recorded parse/provider failure or transport retry.The final manifest validated all 576 cells and provider-audit groups.Source: Closed-Loop Confirmation · p. 8

Primary Outcomes

• 0/66 premature unsupported terminations occurred with ECT versus 40/66 for the faithful controller, a difference of −60.61 pp with a 95% interval of [−78.79, −40.91] pp.The confirmatory analysis covered 22 primary task clusters and H2 passed.Source: Closed-Loop Confirmation · p. 8
• Supported completion was 97/132 for ECT versus 92/132 for the controller, a difference of 3.79 pp with interval [0.00, 9.09] pp.The lower bound exceeded the prespecified −10-point noninferiority margin, so H3 passed.Source: Closed-Loop Confirmation · p. 8

Recovery

• ECT achieved successful recovery in 18/66 eligible incomplete trajectories, with 17 subsequently completing with support.The prespecified event-level H4 gate passed.Source: Closed-Loop Confirmation · p. 8

Control Cost

• ECT used 0.90 additional decision turns, 4,653 additional tokens, 2.34 additional provider seconds, USD 0.00246 additional estimated cost, and 0.045 additional tool calls per trajectory.These paired secondary estimates retained all primary rows and were not efficiency claims.Source: Closed-Loop Confirmation · p. 8

Scope

• Generated worlds cover six task forms rather than deployment traffic, and one planner/model stack with 24 closed-loop tasks does not establish cross-framework generality.The V2 confirmatory analysis used only 22 primary task clusters despite three seeds and fresh worlds.Source: Limitations, Ethics, and Reproducibility · p. 8

Assumptions

• A trusted contract can omit requirements, an adapter can fabricate a consistent receipt, and exact replay covers only a closed transform language.The certificate therefore does not establish external truth, authenticity, effects, safety, or alignment.Source: Limitations, Ethics, and Reproducibility · p. 8

Reproducibility

• The V2 store retained hashes, decisions, usage, and outcomes but not raw planner proposal text, preventing replay of certificate scoring from captured provider outputs.Provider-attempt claims therefore apply to persisted audits, and a process death before commit could leave an unjournaled call that resume would repeat.Source: Limitations, Ethics, and Reproducibility · pp. 8–9

Fault Design

• Static faults were deliberately aligned with declared certificate obligations, so the static study does not establish robustness to unknown error classes.Its 36 held-out task clusters and six synthetic worlds also limit precision claims.Source: Limitations, Ethics, and Reproducibility · p. 8

Interpretation

• Unseen-fault and conflicting-receipt rows were descriptive; internally consistent receipt conflicts did not establish which receipt was externally true or define a general recency policy.The claim is limited to the tested descriptive strata.Source: Limitations, Ethics, and Reproducibility · p. 9

• 0/288 unsafe ECT completions versus 252/288 for the critic core in the locked static study.H1 passed.Source: Conclusion · p. 9
• 0/66 premature unsupported ECT terminations versus 40/66 for the faithful-controller pipeline in the fresh V2 closed loop.H2–H4 passed.Source: Conclusion · p. 9
• ECT met the completion noninferiority margin and recovered successfully in 18/66 eligible trajectories, with 17 subsequently completing with support.Source: Conclusion · p. 9
• ECT establishes trace support under declared assumptions, not external truth, universal task success, safety, or alignment.Source: Conclusion · p. 9