From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution

1 Introduction

• Praxa separates proposals, deterministic admission, brokered execution, external-effect evidence, reconciliation, and serving promotion rather than treating fluent output as authority.The harness treats source conformance, runtime validation, provider evaluation, promotion, and production observation as separate evidence classes.Source: §1 · p. 2

2.1 Research questions

• The research questions span fail-closed state representation, snapshot conformance, reliability benefits, bounded candidate selection, and unresolved adversarial, reliability, and deployment outcomes.Those confirmatory questions remain protocol targets; the paper reports no checked model-checking result.Source: §2.1 · p. 3

3.1 Reasoning, acting, tools, and multi-agent systems

• Praxa treats an optimizer’s output as an untrusted candidate, unable to modify tools, secrets, budgets, approval policy, tenant scope, or rollout authority.The paper adopts the premise that harness structure matters, while restricting what an optimizer-produced candidate may change.Source: §3.1 · p. 3

3.2 Agent evaluation

• Evaluation work motivates fixed task and model bindings, separate effectiveness and resource metrics, explicit invalid-outcome handling, and sealed adversarial trials.Praxa’s repository-local test counts are not agent capability scores, and its Terminal-Bench pilot does not validate production deployment.Source: §3.2 · p. 4

3.3 Risk, adversarial environments, and human oversight

• Praxa is broader in lifecycle span but weaker in adversarial evidence than security-focused systems, and reports neither comparable guarantees nor a security/utility comparison.The paper explicitly distinguishes its positioning from CaMeL-style guarantees and Progent- or MiniScope-scale comparisons.Source: §3.3 · p. 4

3.4 Distributed effects, durable workflows, and artifact promotion

• Praxa composes established durable-effect practices with an LLM-specific authority boundary; the study validates pieces, not novel distributed-systems guarantees.The paper does not claim to invent compensation, idempotency, durable execution, reconciliation, provenance, or signed promotion.Source: §3.4 · pp. 4–5

3.5 Governance and provenance

• Authority-bearing transitions link an authenticated principal, canonical action, deterministic policy decision, broker result, and evidence record; hashes identify artifacts but do not certify them.Praxa operationalizes a narrower claim grounded in attributable transitions.Source: §3.5 · p. 5

4.1 Actors and trust boundaries

• Six logical actors separate proposal from authority: principal, proposer, gateway, policy and budget logic, credential broker, and evidence path; rollout authority separately selects reviewed candidates.Models, optimizers, tool descriptions, and external content are untrusted for authority.Source: §4.1 · p. 5

4.2 Proposal, authority, and effect states

• Admission requires a conjunction of identity, tenant binding, delegated scope, approval, budget, freshness, and action-target policy; effective tools intersect scope layers and explicit denials cannot be undone.A prompt cannot override a failed admission term.Source: §4.2 · p. 6
• After admission, dispatch can end as verified, failed, or uncertain; a timeout remains uncertain until a separate read path supports reconciliation.Acknowledgment alone does not verify an external effect; tested invariants also prevent proposals from self-authorizing or scores from directly activating candidates.Source: §4.2 · pp. 6–7

4.3 Capabilities and invariants

• Table 3 summarizes public-safe capabilities and governing invariants, describing architecture rather than production availability.The public capability map is not evidence that those capabilities are available in production.Source: Table 3 · p. 7, §4.3 · p. 7

4.4 Public and access-controlled surfaces

• The public SDK, CLI, contract library, and gateway examples omit the private action executor; excluded operational material limits independent reproduction of private authority paths.The public artifact excludes credentials, customer data, private fixtures, and unredacted provider transcripts.Source: §4.4 · pp. 7–8

4.5 Multi-runtime utilization case study

• Cloud text, on-device, and realtime voice share an agent identity but retain different capability catalogs; parity is not assumed.Voice excludes visual or interaction-heavy tools, while on-device execution is limited by platform and privacy capabilities.Source: §4.5 · p. 8, Figure 2 · p. 8
• The application audit is secondary utilization evidence, not the primary reproducible result; its dirty tree led the artifact schema to classify the specialist source evaluation as non-publishable.The evaluation inventory has heterogeneous units and oracles that must not be summed; observed feedback-path wiring does not establish invocation completion, task lift, or user benefit.Source: §4.5 · pp. 8–9, Table 4 · p. 9

5 Evidence model

• Evidence does not transfer across lanes: repository tests do not establish provider or production behavior, and benchmark results do not authorize service.A benchmark on one revision is not evidence for another revision or production authorization.Source: §5 · pp. 9–10
• The recursive specialist lane reaches a two-order provider-backed development comparison and counterbalance qualification, but not serving promotion or production observation.Benchmark qualification is distinct from production qualification.Source: Figure 3 · p. 10
• The evidence model distinguishes six stages: static contract, local conformance, controlled provider experiment, benchmark qualification, serving promotion, and production observation.The claim-by-evidence matrix attaches each mark to a named artifact and revision; marks are not cumulative across rows.Source: §5 · pp. 9–10, Figure 3 · p. 10

6.1 Study A: pinned repository-local conformance

• The pinned repository audit separated unit and Workerd boundary tests, checked coverage over a declared source population, and lacked public raw test transcripts.All 363 expected source files appearing in instrumentation means population completeness, not 100% line coverage.Source: §6.1 · pp. 10–11

6.2 Study B: Terminal-Bench reliability-layer pilot

• The 12-task pilot ran three attempts per arm; parser failures were conservatively counted as non-passes, and no inferential test was reported.Pass@1 equals trial accuracy, while pass@3 is the proportion of tasks solved at least once.Source: §6.2 · pp. 11–12

6.3 Study C: recursive specialist benchmark governance

• The coordination-proxy comparison used 30 fixed fixtures, three repetitions, and opposite endpoint orders; its retained result followed evaluation-affecting development changes.The gate required matched context and zero protected violations, among other absolute and relative resource criteria; this was development evidence, not an untouched confirmatory evaluation.Source: §6.3 · p. 12
• $1.578215 baseline and $1.044142 candidate were estimated endpoint costs; production qualification and serving authority remained absent.The comparison was counterbalanced across two endpoint orders; private completions prevent public re-adjudication of every raw trial.Source: §6.3 · pp. 12–13

6.4 Study D: secondary learning-loop and tool-selection audit

• The deployed source/configuration audit mapped four bounded feedback paths, while inspected tool-selection cases remained non-confirmatory and insufficient evidence.The 133-tool catalog and 114 inspected queries were partitioned into 70 calibration and 44 inspected holdout cases; labels were inspected during tuning and lacked independent adjudication.Source: §6.4 · p. 13

6.5 Prospective confirmatory studies (frozen; not executed)

• Prospective protocols specify governance attack/fault testing and a full 89-task reliability study, but their counts are planned rather than observed.The reliability design specifies 890 valid trials; publication of protocol artifacts does not constitute preregistration, and each stage requires its own signed, timestamped ACTIVE manifest.Source: §6.5 · pp. 13–14, Table 5 · p. 14

7.1 Repository-local conformance

• 1,027/1,027 unit tests and 89/89 Workerd tests passed in the author-run release record at the pinned revision.All declared coverage floors passed, and all 363 expected source files appeared in instrumentation; the author-run summaries lack raw per-test transcripts and independent reproduction.Source: §7.1 · pp. 14, 16, Table 6 · p. 15, Figure 5 · p. 15

7.2 Terminal-Bench pilot

• Both arms passed 17 of 36 strict end-to-end trials (47.22% pass@1), while the reliability layer used 37.49% more input and 50.73% more output tokens.The paper reports H3 was not supported under this model, curated task subset, prompt, and budget; the task-level changes were heterogeneous and offset in the aggregate.Source: Table 7 · p. 16, §7.2 · pp. 16–17

7.3 Recursive benchmark gate

• 37.11% fewer tokens, 33.84% lower estimated endpoint cost, and 11.63% fewer steps for the source-authored candidate, with both arms completing 180/180 trials.Both arms had mean accuracy 1.000, recovered all 18 hermetically injected crash-after-commit cases, and recorded zero protected violations; the result does not support latency improvement or production qualification.Source: Table 8 · p. 18, §7.3 · p. 18

7.4 Learning-loop wiring and hybrid tool selection

• Checked-in configuration enabled reflection, recall accounting, and tool-health feedback on the chat worker, plus memory compilation on the router and AI workers.No invocation or completion was observed, and the audit contained no production estimate of task success, tool success, or user benefit.Source: §7.4 · p. 19
• On the 44-case inspected development holdout, the bounded candidate generator recorded 100.0% point-estimate top-1 accuracy and required-tool recall, with zero missed requests in both configurations.Candidate-stage macro precision increased from 18.5% to 55.9%, while mean candidate count decreased from 8.00 to 3.64.Source: §7.4 · p. 19
• After the full deterministic runtime tool-scope union, macro precision was 5.0% versus 5.5%, despite 100.0% point-estimate recall and zero misses in both configurations.The paper characterizes the 0.5-point same-stage precision increase as descriptive, not high end-to-end precision or task-quality lift.Source: §7.4 · p. 19, Figure 8 · p. 19

8.1 The principal result is a boundary, not a leaderboard number

• The harness separates a useful model proposal from authority to enact it through distinct client, gateway, policy, broker, evidence, and rollout boundaries.The paper says this makes narrower questions testable but does not prove the system safe.Source: §8.1 · p. 20

8.2 Efficiency without quality separation

• The coordination-proxy comparison found lower resource totals at matched observed conformance, but equal ceiling-level quality measures prevent a quality-separation claim.Estimated cost used the run-bound operator-supplied price schedule, and the source-authored candidate does not demonstrate optimizer-generated self-improvement.Source: §8.2 · p. 20

8.3 A descriptive benchmark tie is design evidence

• The Terminal-Bench tie and token overhead are descriptive evidence against assuming that planning, verification, checkpointing, and recovery necessarily improve accuracy in this pilot.The paper identifies budget use, shared actor/verifier model binding, and stochastic trajectory divergence as possible mechanisms, not statistical equivalence or a null-effect finding.Source: §8.3 · p. 21

8.4 Why a scalar score is insufficient

• Praxa keeps authorization, safety, contract, grounding, and structured-output violations as protected dimensions that a weighted reporting score cannot offset.Candidates must also remain non-inferior on effectiveness, stay within resource ceilings, satisfy review requirements, and strictly improve at least one held-out or adversarial raw metric.Source: §8.4 · p. 21

8.5 Recursive improvement as governed search

• Recursive harness optimization is bounded candidate generation: an optimizer cannot edit the tools, credentials, policies, approvals, memory boundaries, model binding, or promotion mechanism used to judge it.The retained provider comparison evaluated a source-authored candidate and did not invoke the configured optimizer.Source: §8.5 · pp. 21–22

8.6 Implications for agent publication practice

• The paper recommends claim–evidence matrices, separate reporting for tests and task success, and independent reporting of promotion and production evidence.It also recommends binding provider runs to exact model identifiers, prompts, budgets, tasks, timestamps, scorers, and raw-result hashes.Source: §8.6 · p. 22

Construct validity

• Test pass rate and code coverage are proxies for implementation conformance, not direct measures of agent safety or utility.Coverage records executed syntax and branches, not assertion adequacy.Source: §9 · p. 22

Internal validity

• The release audit was project-run, and the pilot’s fixed arm order, provider stochasticity, asymmetric rebuild setting, and incompletely charged verifier calls limit causal interpretation.The archive also omits several run details, including provider-side defaults, host/runtime versions, and container-image digests.Source: §9 · p. 22

Statistical and external validity

• The Terminal-Bench pilot covered 12 curated tasks with three attempts each and no preregistered sample-size calculation; the coordination-proxy fixtures do not create a new task population.The pilot reports descriptive values without confidence intervals or significance tests, and results need not generalize across models, providers, topologies, environments, risk levels, or users.Source: §9 · p. 22

Synthetic, provider, and learning-loop evidence

• The successful provider comparison used a source-authored candidate on a visible, development-informed partition, while production spawn-agent and web/mobile serving paths were not executed.Learning-loop evidence shows configuration paths but not completed invocations, correct reflections, improved outcomes, or a real-corpus retrieval lift.Source: §9 · p. 23

Deployment, human-subject, and temporal boundaries

• The paper reports no live availability, queue-delivery rate, production throughput, active-user count, incident reduction, user satisfaction, or human-subject study.Preliminary specifications had no reported model-checking output; models, dependencies, prices, standards, and hosted systems also change over time.Source: §9 · pp. 23–24

Reproducibility and provider evidence

• Private authority paths and provider trajectories cannot be fully independently rerun from public materials, which omit raw test reports, command transcripts, per-test transcripts, and a trusted independent signature.The retained provider artifact is time-limited, and the first two development executions retained no result archive.Source: §9 · p. 23

Ethics, safety, and responsible disclosure

• Consent, approval, purpose, tenant identity, resource budget, target, and revocation remain server-owned constraints for consequential agent actions.The design preserves uncertainty rather than presenting incomplete external evidence as success.Source: §10 · p. 24
• The paper states that repository counts are not deployments, formal verification is not claimed, and the pilot did not improve aggregate accuracy.It also reports that production qualification, serving authority, and field outcomes remained absent after the coordination-proxy run.Source: §10 · p. 24
• Raw transcripts are excluded because they may contain benchmark prompts, model completions, credentials, or operational details.The paper recommends explicit license, privacy, authority, and redaction review before future trace releases.Source: §10 · p. 24

11 Reproducibility and artifact availability

• The publication materials include the canonical source, machine-readable aggregates, frozen protocols, schemas, build scripts, and compiled formats.The immutable publication target is preprint-v1.4.0, and release assets are checksummed in the publication manifest.Source: §11 · p. 24
• The repository-local result is tied to a named source revision, evidence bundle, and invariant/source/test/oracle map.The build validator recomputes central quantities and checks that benchmark qualification remains separate from serving authority.Source: §11 · p. 24
• Re-running the full private conformance suite requires authorized access to the pinned implementation, while rebuilding the publication requires no private credential.The provider pilot additionally requires benchmark containers and a model provider; public repository availability does not itself grant reuse or redistribution rights.Source: §11 · pp. 24–25

12 Conclusion

• The pinned release audit reported all 1,027 unit tests and 89 Workerd tests passing, with all 363 expected source files instrumented and four coverage floors met.Raw per-test transcripts and independent reproduction are unavailable.Source: §12 · p. 25
• The Terminal-Bench pilot tied baseline at 17/36 strict passing trials while the reliability layer used more input and output tokens.The reported increases were 37.49% for input tokens and 50.73% for output tokens.Source: §12 · p. 25
• The post-debug coordination-proxy comparison found equal observed conformance while the source-authored candidate used fewer tokens, lower estimated endpoint cost, and fewer steps.The reported reductions were 37.11%, 33.84%, and 11.63%, respectively; the configured optimizer was not invoked and the production specialist topology was not executed.Source: §12 · p. 25
• The conclusion supports explicit, attributable transitions among model proposals, authorization, external effects, evaluation evidence, and serving authority.It says current evidence does not establish adversarial security, production safety, universal reliability, or model superiority.Source: §12 · p. 25

• The author-run release contract reported 1,027/1,027 unit and 89/89 Workerd tests passing, with 363/363 expected files and a complete coverage inventory.Raw per-test transcripts and independent reproduction are unavailable; the result is not 100% statement, branch, function, or line coverage.Source: A Claim–evidence matrix · p. 28
• 17/36 strict trials passed in each Terminal-Bench arm, and the claim of improved aggregate pilot accuracy was rejected for this configuration.Task-level counts index pass@1, pass@2, and pass@3, but do not repair missing raw scorer output or support inference.Source: A Claim–evidence matrix · p. 28
• The recursive conformance gate accepted its synthetic candidate against a deterministic paired artifact and expected gate decision, without demonstrating real quality or resource gains.The evidence does not establish real latency, token, cost, or recovery gains.Source: A Claim–evidence matrix · p. 28
• 37.11% fewer tokens, 33.84% lower estimated endpoint cost, and 11.63% fewer steps accompanied equal observed conformance across 180/180 trials per arm in a development proxy comparison.The comparison does not establish improved quality, latency, production behavior, user benefit, or general-specialist superiority.Source: A Claim–evidence matrix · p. 29
• The retained coordination-proxy candidate satisfied the recorded counterbalance gate, although one source workflow ended red and serving authority remained none.A later credential-free verifier repaired the comparison; raw private trials cannot be publicly re-adjudicated.Source: A Claim–evidence matrix · p. 29
• At revision 705abdce, bounded reflection, recall, and tool-health paths were configured on chat, with memory compilation on router/AI.This version/configuration evidence does not establish invocation, measured self-learning, task lift, or user benefit.Source: A Claim–evidence matrix · p. 29
• On the 44-case inspected holdout, candidate-stage macro precision rose from 18.5% to 55.9%, while full runtime-union macro precision was 5.0% versus 5.5%.The result is non-confirmatory; no confirmatory trial exists and the preregistration manifest remains HOLD.Source: A Claim–evidence matrix · p. 29
• The evidence does not constitute formal verification, a controlled deployment or human-subject study, or proof against compromise of trusted policy, broker, or signing components.The evidence supports implementation and conformance claims, not adversarial security or production safety.Source: A Claim–evidence matrix · pp. 28–29

• Table 11 reports passes in three attempts for each pilot task.Source: Table 11 · p. 30
• The three cron-broken-network records in each arm were parser-null and count as non-passes.Source: Table 11 · p. 30
• The task identifiers are inherited from Terminal-Bench Core 0.1.1.Source: Table 11 · p. 30

• Table 12 is a publication-artifact manifest.Source: Table 12 · p. 30
• The table identifies the roles associated with the publication artifacts.Source: Table 12 · p. 30
• The caption does not specify individual artifact names or roles.Source: Table 12 · p. 30