PaperlayerView exact revision on arXiv

Source-linked AI summary

Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control

Jek Huang, Jeffery Hsia, Jiayi Sun, Freddie Shi, Wei Huang, Ian H. White

arXiv:2607.14890v1cs.AIcs.SE

TL;DR

Autonomous coding agents can make lifecycle states such as tested, reviewed, or done appear established without current evidence tied to the code being advanced. Proof-or-Stop makes lifecycle transitions depend on fresh, tracked-source-state-bound evidence gates, and its evaluation reports lower visible-pass/hidden-fail amplification when review is enforced as a gate.

  • Problem

    Autonomous coding agents produce lifecycle claims, but existing execution, review, and handoff systems do not establish whether those claims are safe for downstream lifecycle automation to act on.

  • Method

    Proof-or-Stop treats agent outputs as claims and permits lifecycle advancement only when fresh, structured, tracked-source-state-bound evidence satisfies the relevant gate.

  • Results

    31/1800 versus 2/1800 visible-pass/hidden-fail injected cells were amplified in the A4-vs-A2′ contrast, a +1.6pp not-amplified gain with 95% CI [0.8, 2.5].

  • Takeaways & Limitations

    Within the evaluated scope, Proof-or-Stop provides a practical route for making autonomous software lifecycles act on admitted evidence rather than agent output alone.

  • Takeaways & Limitations

    The evidence is bounded to one model family, 24 ablation tasks, and a self-hosted corpus, with broader generalization remaining future work.

Abstract

from arXiv · show

Autonomous coding agents increasingly execute multi-step software work, but lifecycle states such as reviewed, tested, DONE, and ready-to-merge remain claims unless supported by current evidence. We present Proof-or-Stop Lifecycle Control, a method that permits lifecycle transitions only when fresh, tracked-source-state-bound, mechanically verifiable evidence satisfies the relevant gate. The method treats agent outputs as claims rather than lifecycle state, and uses proof operationally to mean gate-admissible evidence under a stated trust model, not semantic program correctness. We evaluate an open-source implementation through mechanism tests, a powered control-policy ablation, and operated self-application evidence. The unattended-loop engine passed 10 of 10 scenarios with zero false-DONE, and local-key receipt bundles rejected 18 tamper classes with zero false accepts. In a 9,240-cell ablation, the pre-registered A4 versus A2-prime comparison reduced visible-pass/hidden-fail amplification from 31 of 1,800 injected cells under a compute-budgeted naive loop to 2 of 1,800 under the gated loop, a 1.6 percentage-point improvement in not-amplified rate with a 95 percent confidence interval of [0.8, 2.5]. A near-compute A3 versus A4 comparison, 14 of 1,800 versus 2 of 1,800, indicates that the gain is associated with enforcing review as a lifecycle gate rather than merely adding a reviewer. The self-application corpus contains 565 stories and 1,007 review findings, with 94.8 percent resolved, plus a 68-row high/critical cross-vendor exhibit. These results support Proof-or-Stop as a model-agnostic, host-neutral control layer for deciding which autonomous-agent claims a lifecycle may act on. The evaluation is limited to one model family, 24 ablation tasks, and a self-hosted corpus.

1 Introduction

Autonomous coding agents can produce lifecycle claims without evidence that the claimed state is safe to advance. Proof-or-Stop addresses this by making fresh, source-bound evidence gates—not agent reports—the basis for lifecycle transitions.

  • 1 Introduction: The control problem arises because self-reports, green logs, and reviewer approvals may not establish that evidence corresponds to the code being advanced.A green pipeline or successful handoff can coexist with stale, incomplete, or unsupported lifecycle claims.
  • 1 Introduction: Proof-or-Stop treats agent outputs as claims, allowing review, test, done, and merge-relevant state to advance only after gate-consumable evidence is admitted.The method distinguishes agent-as-claim from lifecycle state and makes claim admissibility the control decision.
  • 1 Introduction: Admissible evidence must be fresh, bound to the tracked source state, and satisfy integrity, authorization, and accepted-outcome checks under the stated trust model.The implementation protects evidence with authenticated integrity, producer identity, freshness, and policy or command-set bindings.
  • 1 Introduction: The evaluation combines engine tests, powered ablation, injected-failure recovery, self-application, and cross-vendor review evidence while separating demonstrated tiers from claims that remain gated.The paper reports a 9,240-cell ablation, a 565-story/1,007-finding corpus, and a 68-row cross-vendor exhibit, while reserving strong cross-vendor quorum claims for fresh independent-host verdicts.
  • 1 Introduction: The unattended loop advances on admissible evidence and otherwise performs bounded repair, honest degradation, escalation, or stopping rather than converting unsupported claims into done.Its plan→execute→review→reflect→gate→done workflow makes evidence gates responsible for consequential transitions.

2 Background and Problem

Existing execution, coordination, and release-gating systems improve agent workflows but do not establish whether lifecycle claims are admissible for the exact source state advancing. Proof-or-Stop addresses this gap by binding structured evidence to source identity and checking integrity, authorization, freshness, and execution claims.

  • Prior systems and gap: Existing durable execution and coordination preserve state or transport verdicts, but neither establishes that a claim remains verified against the code being advanced.The paper identifies admissibility against current artifact state as the missing control.
  • Problem: A green pipeline, passing author test, or agent handoff can coexist with a real defect when downstream automation treats status as sufficient evidence.The powered ablation amplified 31/1800 visible-pass/hidden-fail cases under the naive loop versus 2/1800 under the gated loop.
  • Method: Proof-or-Stop treats actor outputs as claims and permits lifecycle transitions only when gate-consumable evidence is admissible.The abstraction is domain-neutral, while domain-specific evidence packages and separate evaluations remain necessary for other domains.
  • Scope boundary: The tracked-source hashes do not by themselves attest executable state, so dependencies, toolchains, environments, external services, and generated files require additional policy or digest coverage.This is the stated boundary of the source-identity binding.
  • Evidence gate: Admissibility requires fresh source-state bindings, complete policy and command-set bindings, verified integrity, and authorized producers, with execution attestation and accepted outcomes when required.Evidence records bind material, head, story, policy, and command-set hashes; receipts bind command, arguments, working directory, exit code, and output digest.
  • Mechanism evidence: Offline receipt bundles rejected 18 tamper classes with zero false accepts and zero false rejects, but the assurance remains local-key single-host rather than an independent multi-host quorum.The contract verifies self-contained bundles offline and does not establish stronger cross-host assurance.

4 Lifecycle Instantiation

Proof-or-Stop instantiates lifecycle advancement as a sequence of evidence-gated story transitions from initialization through done. Each transition persists structured artifacts, checks current-state bindings and assurance conditions, and blocks or degrades honestly when proof is missing, stale, or insufficient.

  • Lifecycle flow: A story advances through init, init-check, plan, dev, review, test, and done, with every arrow implemented as a concrete evidence-gated transition.The lifecycle matrix names each structured artifact, gate check, and fail-closed behavior.
  • Early gates: Review planning freezes contracted file paths, scope checks reject out-of-scope edits, and review-to-test requires independent verdicts, blocked high/critical findings, and an admissible test receipt.These checks turn planning, scope, review, and testing into explicit lifecycle gates rather than informal status claims.
  • Test-to-done gate: Before done, a freshness-bound authenticated receipt must show that the configured build and full test suite succeeded at the current code state.The receipt records commands, exit codes, output digests, source hashes, policy hash, and command-set hash, and is rejected after source or command-set drift.
  • Review assurance: High-risk material changes require at least three review rounds with at least two independent passing verdicts per round over the current material hash.Independence requires differing host, session, and signing-key identities.
  • Degraded operation: When a second host is unavailable, the system records an explicit degraded single-host fallback rather than synthesizing a quorum.Degraded assurance remains local-only and cannot be upgraded to full assurance.
  • Merge readiness: Merge consumers must verify the certificate against the exact source commit and refuse merging if either source or target branch changes before the protected compare-and-swap.A re-merge during development invalidated prior host verdicts until they were re-attested over the new material hash.

5 The Unattended Loop

The unattended-loop evaluation combines verified engine-contract tests with a powered ablation of weaker control policies. Proof-or-Stop reduced visible-pass/hidden-fail amplification, but the benefit came with higher cost and was concentrated in fault-bearing cases.

  • 5.1 Engine contract (verified): The engine-contract suite passed 10/10 unattended-loop scenarios with zero false-DONE, demonstrating contract enforcement rather than improved outcomes against a baseline.The powered ablation supplies the separate comparative outcome evidence.
  • 5.2 Reflection-loop ablation: 2/1,800 versus 31/1,800 injected cells: Proof-or-Stop reduced visible-pass/hidden-fail amplification against the compute-budgeted naive A2′ control, with H1 not-amplified +1.6pp [0.8, 2.5].The CI excludes zero, but A4 spent approximately 1.2× A2′ tokens, so this is not a clean equal-spend causal estimate.
  • 5.2 Reflection-loop ablation: A4 used 204,553 mean tokens and 81.2 seconds per cell versus 170,545 tokens and 54.8 seconds for A2′, establishing a reliability–cost trade-off rather than a speed-up.The clean-task pilot likewise found no completion benefit and approximately doubled cost and latency for the review-gated loop.
  • 5.2 Reflection-loop ablation: A4 amplified 2/1,800 cells versus 14/1,800 for near-compute A3, supporting an enforcement-isolation comparison between advisory review and an enforced review gate.A3 used nearly the same tokens as A4 and the same one-review signal, but treated review as advisory.
  • 5.2 Reflection-loop ablation: The powered ablation measures recovery or not-amplification, not terminal stop activation, because safe-stop and unsafe-action detectors were zero in these records.Stop behavior is instead supported by separate engine-contract and recovery-pilot evidence.
  • 5.2 Reflection-loop ablation: The ablation does not instantiate the full story-level materialHash/receipt-gated lifecycle for each cell; those lifecycle checks were validated separately.The powered harness tests review-and-redo enforcement under a fixed task/scenario matrix, while story-level tests validate freshness, binding, receipts, verdict admissibility, and the done gate.

6 Recovery under Injected Failure

The powered recovery study compares a compute-budgeted naive loop with evidence-gated variants under injected failures, measuring whether visible-test-passing wrong artifacts are amplified, safely stopped, or repaired. The full matrix reports a small but statistically supported not-amplified advantage for A4, while the pilot verifies the block-to-revise-to-rereview mechanism with strict scope limits.

  • Powered recovery comparison: 99.9% [99.6,100] not-amplified for A4 versus 98.3% [97.6,98.8] for A2′, a +1.6pp [0.8,2.5] primary H1 gain.The powered run covered 9,240 cells, and the confidence interval excludes zero.
  • Study design: A2′ is a compute-budgeted bare loop, A4-C adds one independent review gate, and A4b-B isolates bounded block-to-revise-to-rereview recovery.The pilot scores visible acceptance, hidden ground-truth correctness, and whether the arm ships the artifact.
  • Cost and trade-off: The pilot required extra work: A2′ averaged 1.00 attempts, A4-C added one review gate, and A4b-B averaged 2.067 attempts and 1.067 review rounds.The added evidence and review fidelity move outcomes from amplification toward safe-stop or bounded repair.
  • Pilot mechanism: 15/15 wrong artifacts were amplified by A2′, safely stopped by A4-C, and recovered to green by A4b-B in the three-task pilot.The pilot used parse-range, median, and text-stats tasks with five injection cells and three repeats per cell.
  • Scope and limitations: The pilot is directional rather than definitive because it used one model, three tasks, n=3 per task-cell, a B-fidelity proxy, and no fault exceeding the two-review-round cap.Independent V3/V4 audits verified that recovery was not a classifier artifact.

7 Host-Neutral Transfer

Host-neutral transfer is implemented as an evidence property: git-native work units can move between hosts, while verdicts remain admissible only when their receipts match the current code state. The demonstrated mechanism reconstructs worktrees from git alone, but powered independent-host claims remain gated.

  • Evidence-based portability: Cross-host verdicts are admissible only when the receipt materialHash matches the current code; freshness, binding, and receipt predicates are host-independent.Host identity affects provenance and independence checks, not gate semantics.
  • Handoff architecture: Handoff packs contain baseSha, headSha, the canonical story, and allowed paths rather than a worktree image.Machine B fetches, reconstructs the worktree, and resumes from checkpointed git state.
  • Safety boundaries: Takeover hosts are blocked from auto-merge, evidence submission, and production-done actions; stale mirrors, missing capabilities, and single-host degradation produce explicit warnings or local-only evidence.These behaviors are verified by HN-5, HN-7, HN-9, and HN-10 tests.
  • Git-native reconstruction: A fresh second machine exactly reconstructed the first machine’s HEAD and tree content-hash from git-tracked deltas plus a small pack.The demonstration used a bare remote and no filesystem access from machine B to machine A.
  • Scope and deferred claims: The reconstruction result is a single-story proof of mechanism, while broader powered transfer across many stories, worktree sizes, and physical machines remains pending.The stronger claim of a genuinely distinct cross-vendor verdict quorum is explicitly deferred pending a powered independent-host campaign.

8 Self-Application and Audit

Self-application provides operated evidence that independent review finds issues missed under passing author-side tests, while cross-vendor review catches additional high-severity defects and exposes measurement flaws. The corpus also shows that DONE can coexist with unresolved advisory findings, and dogfooding surfaced both real defects and justified refusals to weaken gates.

  • Division of review labor: Review lanes were nonredundant: the quality lane supplied volume, while the test-coverage lane had the highest high/critical density at 70%.The cross-vendor row was selectively invoked and should not be read as a corpus-wide rate estimate.
  • Independent review under green tests: 26 of 28 deep-set findings (93%) were filed while the author’s smoke or tests were passing, including six behavior-changing defects with confirmed fixes.The 28 findings covered production bugs, logic defects, test-coverage gaps, and documentation-accuracy findings.
  • Measurement audit: An independent Codex host-2 reviewer found denominator bias in the paper’s headline ablation rates, leading to the matched-cell invariant that scores both arms on the same data cells.Same-vendor review lanes had passed the statistics before this correction.
  • Cross-vendor review: The cross-vendor exhibit records 68 high/critical catches across 26 stories, all resolved, including 2 critical and 66 high findings.The exhibit is an existence and soundness demonstration, not an unbiased marginal-rate estimate.
  • Corpus and finding status: The 565-story, 1,007-finding corpus contains 955 resolved, 41 open, and 11 dismissed findings; all open findings are on already-done stories under severity- and round-graded gates.Thus DONE records unresolved advisory issues rather than meaning zero known issues.
  • Dogfooding friction: Dogfooding found two genuine tooling defects and several correct refusals to lower assurance, including degraded single-host escalation and rejection of weak evidence correspondence.The system also exposed concurrency artifacts, demonstrating that self-gating forces visible repair or refusal rather than silent weakening.

9 Experience Reuse and Honest Metrics

Proof-or-Stop’s experience layer reuses verified past findings as advisory hints without allowing them to satisfy lifecycle gates. The corpus shows recurring defect classes, but the pilot does not establish that surfacing those hints improves review.

  • Advisory boundary: Experience hints influence attention but are marked gateEvidence:false, excluded from materialHash, and omitted from the review prompt digest.They are consumed at lifecycle entrypoints while remaining unable to satisfy review, validation, or done gates.
  • Observed recurrence: 989 of 1,007 findings carry curated signatures across 211 canonical area|failureMode classes, with 77.1% held-out forward recurrence.These figures describe observational recurrence, not causal improvement from experience reuse.
  • Honest metric interpretation: The pre-registered ON/OFF pilot stopped at its pilot gate, so the paper does not claim that surfacing hints improves review.The authors report the recurrence substrate as an observational fact rather than an efficacy result.

10 Related Work

Proof-or-Stop is positioned as an assurance layer that makes autonomous-agent lifecycle claims admissible only when evidence is bound to current code state. It complements execution, coordination, orchestration, benchmarking, reflection, and provenance systems rather than replacing them.

  • Assurance-layer distinction: Proof-or-Stop decides which lifecycle claims are admissible, while durable systems preserve state, coordination protocols move messages, and benchmarks score task success.The systems compose with Proof-or-Stop but do not replace its evidence-admission rule.
  • Evidence binding: Freshness-bound code identity distinguishes Proof-or-Stop from durable execution, cross-vendor coordination, and quorum systems.Receipts are admissible only when their materialHash matches the relevant code state, with honest degradation to single-host operation.
  • Agent frameworks: Proof-or-Stop is an assurance layer, not a conversation pattern or agent framework: it determines which orchestrated agent claims become admissible lifecycle evidence.Agent frameworks assemble agents and workflows; Proof-or-Stop controls which resulting claims a lifecycle may act on.
  • Benchmarks and reflection: Agentic software-engineering systems measure task success, whereas Proof-or-Stop instruments lifecycle claims such as reviewed, tested, and done for independent checking.Reflection and iterative refinement remain useful for improvement, but only freshness-bound evidence advances a phase.
  • Assurance foundations: Related assurance traditions contribute proof, monitoring, claims, provenance, and release-gating ideas that Proof-or-Stop combines with lifecycle-wide state-bound receipts.The method binds evidence to materialHash, headHash, storyFilesHash, authenticated integrity digests, and receipt identity.
  • Closest related work: Contemporaneous evidence-gated systems converge on gating agent claims rather than trusting agent assertions, while Proof-or-Stop emphasizes changing-code freshness and honest quorum degradation.The paper presents cross-vendor review as a diversity case study rather than a consensus-voting claim.

11 Threats to Validity

The threats analysis limits claims through evidence-integrity defenses, explicit corpus and labeling caveats, and boundaries on causal, cross-host, and generalization conclusions. It distinguishes verified mechanisms from observational or curated evidence.

  • Failure modes: Stale, forged, missing, or over-claimed evidence is rejected, blocked, repaired, escalated, or stopped by Proof-or-Stop controls.Hash mismatches block stale evidence, receipt and digest checks reject forged artifacts, and prohibited-wording guards limit unsupported capability claims.
  • Corpus and labels: The self-built corpus is not an independent population, and reviewer-style labels such as smoke_would_miss are not ground truth.Hard per-row facts instead concern named findings, passing verification at filing, and subsequent fix commits.
  • Sampling: The 12-story, 28-finding deep set is curated rather than random, so its 93% smoke_would_miss rate does not estimate corpus-wide prevalence.Most rows concern coverage or documentation/claim mismatches; six production or logic defects are reported separately.
  • Cross-host evidence: Cross-vendor catches are concrete but selection-described: 68 high/critical findings over 26 stories support motivation, not a systematic Tier-C base rate.The separate provider-execution batch contains 25 tracked trials and is not a powered cross-host rate claim.
  • Causality: Observational experience reuse and an under-powered causal A/B do not establish causal model improvement from experience.The analysis explicitly separates observational reuse from the causal comparison.
  • Counterfactuals: The study observes what Proof-or-Stop caught rather than what direct single-pass runs would have shipped, and its parallel-versus-sequential pilot found no measurable time benefit at about 1.8× cost.The pilot is a boundary case, not support for the thesis.
  • Status evidence: Descriptive lifecycle-status comparisons do not provide hidden-oracle adjudication, true-positive validation, or multi-model generalization.They show that terminal completion and admissible delivery can diverge on the same cell keys.
  • Scope: The corpus is smaller than population studies of agentic pull requests, so conclusions remain bounded to a gated, self-hosted setting.The study trades scale for end-to-end audit depth.

12 Future Work

Future work extends the verified evidence spine toward powered recovery, broader host-neutral validation, real-work base rates, causal experience studies, and deployment integration. The paper also preserves its current boundaries while proposing domain packages that retain the same control abstraction.

  • Tier-B evaluation: The recovery pilot should be scaled into a fully powered stratified injection study, while git-native handoff should be tested across multiple stories and varied worktrees.These efforts would strengthen Tier-B evidence without changing the current claim boundaries.
  • Real-work validation: Future work should estimate the real-work green-but-wrong base rate, because the injected-fault result is conditional on a visible-pass/hidden-fail trap.That denominator is needed for cost-benefit claims about when the control overhead is worthwhile.
  • Host neutrality: A powered independent-host campaign is proposed so cross-vendor verdicts form a fresh material-hash-bound quorum.Live cross-host execution has been exercised; the remaining gap is scale and independence.
  • Evidence integration: A gate-grade replay would connect selected powered-ablation cells to materialHash, commandSetHash, and receipt evidence without changing the ablation’s status as a control-policy experiment.The proposal includes A4 amplified and repaired cells alongside representative A2′ cells.
  • Experience layer: A powered causal A/B with a formal two-proportion test is planned for the experience layer.The current evidence therefore claims only the verified spine, recovery pilot, powered ablation, and self-application evidence.
  • Deployment: Merge deployment requires a GitHub App or Action, branch-protection integration, rejection tests, independent-host validation, path-specific policies, audit trails, and cross-repository replication.The paper treats merge as a future deployment claim rather than a current result.
  • Domain extension: Future domain packages can instantiate the same evidence, gate, and transition abstraction in dry computational workflows.Current PINN- and Quantum-style schema smoke tests do not validate scientific or hardware correctness.
  • Boundary of extension: The central scope remains that agent outputs initiate claims, while admitted evidence advances lifecycle state.This principle is the intended invariant as the method is extended.

Reproducibility

The reproducibility package centers on public artifacts, mechanically extracted records, verification harnesses, and explicit caveats about what each artifact supports. It exposes the implementation and evidence needed to re-derive the paper’s strongest claims without upgrading case studies into powered estimates.

  • Extraction: All quantitative figures are mechanically extracted from lifecycle metadata and git history rather than hand-transcribed.The public address links the implementation repository, verifier tests, and re-extraction entrypoint.
  • Reproduction materials: Reproduction commands are provided in Appendix E, while deep-finding tables, experiment summaries, and figure sources are released through the arXiv or release artifact bundle.Additional corpus figures are included in the release artifact but not embedded in the paper workspace.
  • Artifact index: The public artifact bundle identifies the primary files reviewers can inspect to re-derive the paper’s strongest quantitative and audit claims.The arXiv v1 artifact maps to the arxiv-v1 tag or recorded release-note commit hashes.
  • Evaluation harness: The powered-ablation harness supports pluggable A1, A2, A2′, A3, and A4 engines, fixed-point injection, raw evidence, and local DONE closure.It includes 51 verification intents and 7/7 recovery-runner selfchecks, but the adapter is not itself a powered-result claim.
  • Cross-vendor artifacts: Cross-vendor handoff materials document concrete proof-looking-metadata and receipt-boundary cases without establishing host-intent or powered cross-host rates.The evidence includes four exhibits plus corpus context and a main-paper cross-vendor exhibit.
  • Supplemental evidence: The supplemental paired readout shows that terminal completion and admissible delivery can diverge, alongside a 3.80× token readout for the bundled gated run versus no-review control.It is lifecycle-status and resource evidence rather than an accuracy or cross-vendor effect estimate.

A Pre-registered protocols

The pre-registered evaluation used a five-arm ablation, recovery injections, host-transfer simulation, paired execution-status comparisons, and reproducible corpus exhibits under controlled validity and data-hygiene rules.

  • A Pre-registered protocols: The five-arm ablation compared prompt-only, naive retry, compute-budgeted naive, review-only, and Proof-or-Stop loops across stratified tasks with shared tools and randomized execution.The primary contrast was pre-registered as A4 versus A2′; A1, A2, and A3 were reference cells.
  • A Pre-registered protocols: Recovery evaluation used 15 injections across naive and Proof-or-Stop loops, classifying each run as recovered, safe-stop, or amplified, with not amplified as the primary endpoint.Tier-A contract reassertions were excluded to avoid double-counting.
  • A Pre-registered protocols: The protocol included a two-machine Git-native handoff that reconstructed worktree state after a host failure without transferring a worktree image.The implementation also released reproduction entry points, experiment records, and validation commands.
  • A Pre-registered protocols: Infrastructure-fault runs were voided, pilot runs calibrated per-run time, and arms were executed back-to-back within repetitions to reduce environmental-load effects.These validity filters were applied before scoring the committed waves.
  • A Pre-registered protocols: The paired Cell03/Cell06 comparison was a descriptive execution-status matrix over 1,152 matched cells, not a hidden-oracle correctness result.The no-review condition reported terminal runner status, whereas Proof-or-Stop reported delivery decisions after completion, recovery, or safe-stop.
  • A Pre-registered protocols: The corpus exhibits were recomputed from live metadata and supported reproducible findings, including 565 stories, 518 done stories, 1,007 findings, and a 94.8% resolution rate.Exhibits preserved reviewer lanes, filing-time verification state, rationale, and confirmed fix commits; the implementation and artifacts were publicly identified.

F Experience-reuse validation details

The experience-reuse analysis separates advisory memory from lifecycle proof and tests whether curated defect signatures recur, while stopping a causal study when its pilot baseline failed the planning criterion.

  • F Experience-reuse validation details: The advisory memory layer exposes prior lessons without allowing them to satisfy evidence gates.Runtime validation and the controlled memory pilot treated memory consumption as advisory rather than proof.
  • F Experience-reuse validation details: 989 of 1,007 findings mapped to 211 canonical area|failureMode classes, yielding 0.700 retrodictive recall and 77.1% held-out forward recurrence.A coarse 31-class taxonomy reached 0.98 recall but was judged vacuous, so the more specific taxonomy was retained.
  • F Experience-reuse validation details: Figure 10 shows rising defect-class recurrence alongside positive novelty per window, meaning new classes continue appearing and recall remains bounded below 1.The reported recurrence trend is Spearman ρ ≈0.72 [22].
  • F Experience-reuse validation details: The ON/OFF memory pilot ran 90 arm-runs with zero isolation violations, but its control-arm baseline was 1/30 = 0.033 versus the planning band [0.5, 0.8], so the powered study was refused.The pilot is therefore reported as instrumentation and stop-rule evidence rather than a causal result.
  • F Experience-reuse validation details: A parallel-development pilot found no supported speed advantage: 1-host parallel took 113.0 s versus 118.7 s sequentially, used approximately 1.8× the tokens, and had identical frozen-test quality.An independent adversarial check refuted an earlier claim that parallel execution was consistently slower, retaining the corrected null.
Loading 2607.14890v1…