PaperlayerView exact revision on arXiv

Source-linked AI summary

Guardrailed Meta-Agent Loops: Stress-Testing Policy Pinning, Budget Bounds, and Crash Recovery

Qinzhen Ma, Jialin Wu

arXiv:2609.12216v1cs.RO

TL;DR

Self-improving agents create an audit problem when the controller can change both behavior and the criteria used to judge it. GuardrailLoop tests bounded adaptation alongside policy, compute-accounting, and recovery contracts; experiments find useful round growth and show that successful state recovery can still include repeated execution.

  • Problem

    When a controller can change both agent behavior and evaluation criteria, it is unclear what may improve, what must remain fixed, and what evidence is needed after interruption.

    Source: §1 · p. 1

  • Method

    GuardrailLoop couples a hash-pinned human policy and bounded code-owned evolution with prefix compute accounting and separate scientific-state and normalized-trace recovery checks.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across 240 crash cases, all recovered the specified outcome but only 210 preserved the normalized trace; round growth changed target attainment by +1.00 and restricted mean compute by −56.97 GPU-hours.

    Source: §3.2 · p. 4, §4 · p. 5, §6 · p. 7

  • Takeaways & Limitations

    Reports of successful recovery should state both which state was recovered and what execution was repeated.

    Source: §6 · p. 7

    Evidence is limited to one calibrated shirt-folding simulator, one simulated VLA, synthetic data, an offline deterministic planner, and simulated GPU-hours.

    Source: §5 · p. 6

Abstract

from arXiv · show

Self-improving agent workflows create an audit problem when the same controller can change both its behavior and the conditions under which that behavior is judged. We present GuardrailLoop, a simulation-based testbed that makes three operational contracts jointly testable: preservation of human-defined policy, compute accounting at every recorded execution prefix, and recovery of a specified scientific state after crashes. A hash-pinned policy fixes goals, scope, evaluation identity, budget, and release conditions; machine-directed evolution is restricted to a code-owned feature catalog and bounded knobs. The contribution is an executable boundary and an evaluation protocol that separates useful adaptation, state recovery, and repeated execution. In a paired 50-seed 2 x 2 study, round-stage growth changes target attainment by +1.00 and restricted mean compute to target by -56.97 simulated GPU-hours (95% paired-bootstrap interval [-58.91,-54.70]); idle growth has zero measured utility effect. Across 240 enumerated crash injections, all runs recover the defined outcome, but only 210 preserve the normalized trace: 30 pre-commit crashes repeat a planner call. Resource-drift, kill-switch, integrity, and output-guard matrices satisfy their specified checks. These findings show why successful outcome recovery is insufficient evidence of exactly-once execution. They establish conformance within one calibrated deterministic testbed, rather than general safety or real-world self-improvement.

1 Introduction

Self-improving workflows create an audit problem when a controller can change both behavior and evaluation criteria, while crashes can alter execution, accounting, and replay. GuardrailLoop makes authority preservation, prefix accounting, and recovery equivalence jointly testable in a bounded orchestrator.

1 Introduction

  • The central question is what may improve, what must remain fixed, and what evidence is needed after interruption.The three contracts address distinct failure surfaces: a valid final budget total does not establish prefix bounds, and matching outcomes do not establish identical execution.Source: §1 · p. 1
  • A hash-pinned policy fixes goals, scope, evaluation identity, budget, and release conditions, while machine evolution selects only code-owned catalog features.The planner cannot supply a feature's numeric effect or write policy fields; new-agent suggestions remain inert records.Source: §1 · pp. 1–2
  • The contribution couples these contracts with paired utility tests and enumerated conformance checks, rather than proposing a new hash-chain primitive or general safety guarantee.Round growth is useful in the calibrated simulator, idle growth is not, and all 240 crash cases preserve the specified outcome although 30 repeat a planner call.Source: §1 · p. 2
  • The joint specification links human-pinned policy, prefix compute accounting, and scientific-state recovery while separately tracking normalized-trace identity.It distinguishes recovery of scientific state from whether execution requests were repeated.Source: §1 · p. 2

2 Related work and scope of the contribution

Prior work studies self-improvement, runtime restrictions, budget-constrained planning, and durable execution. GuardrailLoop narrows adaptation to code-mediated configuration changes and evaluates that boundary alongside accounting and recovery, without claiming stronger security or exactly-once execution.

Self-improvement and the object of adaptation

  • Prior self-improvement systems explore modification of optimizers, agent programs, or agent implementations; GuardrailLoop instead restricts adaptation to catalog-resolved changes inside a fixed orchestrator.This makes authority and recovery behavior tractable to inspect, but the experiments do not compare performance with open-ended code-search systems.Source: §2 · p. 2

Runtime constraints and privileges

  • AgentSpec [7] and Progent exemplify established enforcement and controlled policy-update ideas; this paper pins experimental governance and permits catalog-bounded learning-configuration changes.It evaluates that boundary with accounting and recovery, without claiming stronger security than prior enforcement frameworks.Source: §2 · p. 2

Resource-constrained planning

  • INTENT [3] studies budget-constrained tool use under priced, stochastic executions, whereas this paper checks simulated training/evaluation compute charges against a pinned cap.The accounting contract covers settled and unfinished charges, not learning tool-call value or predicting monetary cost.Source: §2 · p. 2

Durable execution and recovery observables

  • Durable Functions studies record-and-replay equivalence, and AgentRewind [11] restores aligned context and environment state; this work separates scientific-state equality from normalized-trace equality after crashes.Its evidence is not a general replay theorem or an exactly-once guarantee for external effects.Source: §2 · pp. 2–3

3 Guardrailed meta-agent testbed

GuardrailLoop fixes human policy while restricting machine-directed changes to bounded, code-owned interfaces. Its operational contracts separately test authority, prefix accounting, and recovery of scientific state versus execution trace.

3.1 Execution and enforcement mechanisms

  • The fixed orchestrator separates a hash-pinned human policy from machine-selected catalog identifiers interpreted by trusted code.The policy and evaluation hashes are checked before every stage, and the model has no policy-field write path.Source: §3.1 · p. 3, Table 1 · p. 3
  • A hash-linked ledger is replayed only after full-chain verification; torn final records may be truncated, while edits to complete prefixes fail closed.Round changes commit through one durable stage_done event, and idle changes through one idle_finished event; a pre-commit crash can repeat a planner call.Source: §3.1 · p. 3

3.2 Operational contracts and observable evidence

  • Table 2 maps contracts to reported evidence and boundaries; its counts describe enumerated experiments, not deployment-failure probabilities.Source: Table 2 · p. 4
  • Machine-directed transitions preserve the pinned policy while limiting evolution to code-owned features, bounded knobs, and other constrained interfaces.The model selects an admissible operation but does not determine its numeric effect or write policy fields.Source: §3.1 · p. 3, §3.2 · p. 4
  • At every ledger prefix, settled and in-flight compute must remain within the policy budget plus tolerance, including after resource changes.On resume, unfinished work is repriced from recorded charges while evaluation cost is reserved; the check covers ledger-accounted work, not unrecorded physical work.Source: §3.2 · p. 4
  • Recovery separately checks scientific-state equivalence and normalized-trace identity; all 240 crash cases recover the outcome, but only 210 preserve the trace.The suite uses an offline deterministic planner and unchanged execution settings; pre-commit crashes can repeat planner calls.Source: §3.2 · p. 4

4 Evaluation

The evaluation tests utility, accounting, and recovery separately within a calibrated simulator and enumerated fault model. Round-stage growth improves target attainment and compute-to-target, while crash tests distinguish recovered outcomes from repeated execution.

Interpretation

  • The utility ablation varies growth availability, not guards or recovery mechanisms, so it does not estimate the enforcement components’ performance cost or necessity.The study frames the ablation as a mechanism check in a simulator calibrated so catalog-bounded round growth can reach the target.Source: §4 · pp. 4–5

Protocol

  • The paired utility study uses 50 seeds in a 2×2 growth design, with observed target attainment and restricted mean GPU-hours to target as primary outcomes.Each pinned evaluation uses 1,000 Bernoulli episodes across five shirt-folding tasks; the seed-run, not an episode, task, or round, is the experimental unit.Source: §4 · p. 5

Growth and budget results

  • +1.00 target-attainment effect and −56.97 GPU-hours in restricted mean compute to target for round growth; idle and interaction contrasts are zero in these runs.The paired-bootstrap interval for compute is [−58.91, −54.70]; the figure reports observed hits changing from 0/50 to 50/50 and compute falling from 160 to 103.03 GPU-hours.Source: §4 · p. 5, Figure 1 · p. 5
  • At budgets 20, 40, 80, and 160, target hits are 0, 0, 0, and 50 of 50; maximum recorded prefix excess is 1.42×10−14 GPU-hours.The stated tolerance is 10−9 GPU-hours, and settlement reconciliation error is 2.84×10−14; these are internal ledger checks, not independent metering.Source: §4 · p. 5

5 Responsible-use statement and limitations

The paper identifies dual-use and safety-theater risks, and limits its evidence to a specific simulated testbed. It recommends additional safeguards before real use and cautions that the tests do not establish general safety or self-improvement.

Responsible use

  • Meta-agent infrastructure can support responsible experimentation or make autonomous optimization easier to operate at scale, while also enabling harmful goals, abusive tasks, centralized workplace decisions, or resource use without meaningful consent.The system has no robot or over-the-air release path, automatic release, or mechanism to instantiate proposed agents.Source: §5 · p. 6
  • Before real use, operators should independently cap spend, protect sensitive data, authenticate approvals, externally anchor audit records, use a sealed holdout, and require domain-specific review.The recommended review includes canary, rollback, and physical-safety review; the listed controls do not remove operator responsibility or make a harmful human policy acceptable.Source: §5 · p. 6

Limitations

  • Evidence is limited to one calibrated shirt-folding simulator, one simulated VLA, synthetic data, an offline deterministic planner, and simulated GPU-hours; tests establish modeled conformance, not general safety or self-improvement.Fifty seeds sample simulator noise, not model-provider, hardware, or organizational variation; crash sites cover selected boundaries, not arbitrary storage, kernel, or network failures.Source: §5 · p. 6

6 Conclusion

GuardrailLoop makes bounded adaptation auditable through separated policy and evolution planes, prefix compute accounting, and distinct recovery observables. The enumerated crashes recover the specified outcome, but some repeat planner calls, so outcome recovery alone does not establish identical execution.

Conclusion

  • GuardrailLoop makes bounded adaptation auditable through an explicit policy/evolution split, prefix compute accounting, and a recovery contract tracking scientific outcome and normalized execution trace.The experiments demonstrate a useful permitted growth path in the calibrated simulator and conformance at the tested boundaries.Source: §6 · p. 6
  • All enumerated crashes recover the specified outcome, while some repeat a planner call; reports should state both the recovered state and repeated execution.Extending the result to remote planners and physical workloads requires separate evaluation of nondeterministic retries, external side effects, authenticated state, and independently metered resource use.Source: §6 · p. 7

A Implementation details

The implementation bounds machine-directed evolution to code-owned choices and records durable, verifiable state transitions. Its logs and artifacts are checked before replay or use.

A Implementation details

  • Machine feature selections contain a catalog ID, evidence, and rationale; code constructs the feature and supplies its numeric deltas from the code-owned catalog.At most two features may be merged per round and one during an idle pass; the bounded evolution state has seven optimization/data knobs and a self-check flag.Source: A Implementation details · p. 8
  • Agent-catalog outputs contain an ID and rationale, and code creates pending proposals with no execution path in the prototype.Neither pending nor approved records are executable in this implementation.Source: A Implementation details · p. 8
  • The event log is the source of truth, and sequence numbers, predecessor hashes, and event hashes are verified before replay or append.Policy and evaluation files are checked against recorded hashes, while training snapshots and final checkpoints are content-address checked before use.Source: A Implementation details · p. 8
  • Round growth is installed by the durable stage_done event and idle growth by idle_finished; new runs do not emit legacy fine-grained event types.Dashboards and candidate manifests are derived views of the event log.Source: A Implementation details · p. 8

B Experimental protocol and extended results

The protocol uses paired efficacy and budget studies, with tables covering a 2 × 2 Feature-growth ablation and a budget sweep under the full loop.

Experimental protocol

  • Efficacy and budget studies use seeds 0–49, while reliability studies use seeds 0–9; evaluations cover five equally weighted tasks with 200 episodes each.Each evaluation costs 0.2 simulated GPU-hours; a default complete round requests 16 GPU-hours plus evaluation, versus 1.2 GPU-hours for the minimum affordable round.Source: B Experimental protocol and extended… · p. 8

Extended results

  • Table 4 reports point estimates for the full 2 × 2 Feature-growth ablation, with principal-contrast and attainment intervals reported in Section 4.The supplied caption does not provide the table’s cell values.Source: Table 4 · p. 8
  • Table 5 presents a budget sweep under the full loop and defines “Spent” as mean terminal accounted compute.The supplied caption does not provide the sweep’s point estimates.Source: Table 5 · p. 8

C Fault model and interpretation

The fault tests separate recovery of projected scientific state from identity of the normalized execution trace, while also checking resource drift and integrity handling.

Crash comparison

  • Thirty trace mismatches occur at three pre-commit crash sites, each adding one offline-planner call, while the final projected state remains identical.The planner is deterministic and each state plan is committed atomically; a nondeterministic remote planner could return a different retry plan.Source: C Fault model and interpretation · p. 8

Resource drift

  • Across 40 resource-drift runs, terminal and prefix usage stay within the 20-hour policy, tick gauges remain monotone, settlements reconcile, and evaluation completes.The suite starts with four GPUs and 0.25-hour steps, then resumes under changed configurations; trajectories are not compared with the fixed-config baseline.Source: C Fault model and interpretation · pp. 8–9

Integrity

  • All 70 integrity attack trials stop at specified verification boundaries, and ten torn final JSON records truncate to the last complete prefix and resume successfully.The matrix models localized corruption; an attacker with complete write access could replace the full record and recompute hashes, so stronger trust mechanisms remain future requirements.Source: C Fault model and interpretation · p. 9

D Reproducibility artifacts

The reproducibility package describes a guarded experiment driver and test suite, but its scripts and raw run data are not included in the preprint source package.

Artifacts

  • The supplementary materials comprise the experiment driver, figure generator, compact per-trial CSV files, and a machine-readable summary.These materials and raw run directories are not included in the preprint’s LaTeX source package.Source: D Reproducibility artifacts · p. 9

Execution safeguards

  • The driver rejects raw-data roots containing an existing study manifest or measurements by default and records source hashes and protocol in a manifest.It uses independent run directories per seed and condition and fails on incomplete paired blocks.Source: D Reproducibility artifacts · p. 9

Validation

E Conditional reasoning about run contracts

The section derives run-contract checks under explicit execution assumptions and distinguishes local state recovery from guarantees about external effects. It also records the generative-AI assistance used in manuscript preparation.

E Conditional reasoning about run contracts

  • Under recorded-charge, serialized-allocation assumptions, reserving evaluation cost permits a bounded number of further training ticks when accrued charge plus reservation fits the cap.If accrued charge plus evaluation cost exceeds the cap, the combined training-and-evaluation sequence cannot be admitted.Source: E Conditional reasoning about run… · p. 9
  • Reapplying admission checks at resource changes bounds every intermediate prefix, but the argument does not analyze floating-point accumulation or unrecorded physical cost.It assumes no concurrent unreserved work, no other new charges during allocation, and evaluation cost no greater than the reservation.Source: E Conditional reasoning about run… · p. 9
  • Without an external protocol to resolve or deduplicate an operation, local commit atomicity cannot ensure both eventual completion and at-most-once external effects.A retry may duplicate an effect, while declining to retry may leave it incomplete; the offline deterministic planner experiments do not establish mechanisms for side-effecting services.Source: E Conditional reasoning about run… · pp. 9–10

Use of generative AI tools

  • OpenAI Codex assisted with manuscript restructuring, language revision, formalization of stated assumptions, reference checks, and LaTeX preparation; no new experiments were run for the revision.Source: Use of generative AI tools · p. 10
Loading 2609.12216v1…