PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models

Mingyue Cui, Zheyuan Liu, Yihan Zhu, Zheyuan Zhang, Meng Jiang

arXiv:2609.39820v1cs.ROcs.AI

TL;DR

Runtime shields can leave the underlying policy unchanged, allowing policy–shield mismatch to persist. FAILBANK converts runtime feedback into outcome-aware records and guarded LoRA updates; across two backbones, it improves the joint success–cost operating point over base policies.

  • Problem

    Runtime shields can correct actions without updating the nominal policy, leaving a persistent policy–shield mismatch that may block task progress.

    Source: §1 · p. 1

  • Method

    FAILBANK logs observe-only CBF proposals, admits corrections and successful uncorrected actions as weighted records, and uses guarded LoRA updates to improve the policy.

    Source: §1 · p. 2, §4.3 · p. 5

  • Results

    Across two backbones, FAILBANK improves the joint success–cost operating point over base, raising task success rate by 8.5 and 6.9 percentage points and reducing policy-induced cumulative cost by 35.6% and 23.8%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Runtime safety feedback can serve as supervision for improving future policy behavior, rather than only as a temporary intervention mechanism.

    Source: §7 · p. 9

    The implementation targets continuous flow-matching policies and static obstacles; other action formulations and dynamic scenes require adapted interfaces or additional capabilities.

    Source: §6.3 · p. 9

Abstract

from arXiv · show

Vision-language-action (VLA) models generalize broadly across robotic manipulation tasks, but complex environments require balancing task success with unintended contact. Runtime shields can correct individual actions, but they leave the underlying policy unchanged, so repeated disagreements may create a persistent policy-shield mismatch that blocks task progress. To address this challenge, we introduce FailBank, a four-stage self-evolving framework that converts runtime feedback into persistent policy improvement. During collection, a fixed CBF-based safety module serves as an observe-only teacher, producing counterfactual corrections while the policy remains in control. Outcome-aware admission then converts useful proposals into corrective targets and retains successful uncorrected actions as quiet anchors for guarded LoRA updates. We evaluate FailBank on the VLA-Arena benchmark across two difficulty levels and two VLA backbones. Compared with the base policies, FailBank improves the joint success-cost operating point. Across the two backbones, FailBank improves task success rate by 8.5 and 6.9 percentage points, while reducing policy-induced cumulative cost by 35.6\% and 23.8\%, respectively. Compared with runtime shielding, FailBank raises task success rate by 25.4 and 9.5 percentage points, while maintaining comparable policy-induced cumulative cost. These results show that runtime feedback can serve as persistent policy supervision rather than only as a temporary action constraint.

1 Introduction

Complex manipulation requires balancing task completion with unintended contact, while runtime shields can leave a persistent mismatch with the unchanged policy. FAILBANK converts runtime feedback into policy updates and improves the joint success–cost operating point across the evaluated settings.

1 Introduction

  • VLA deployment in complex scenes requires balancing task completion with unintended contact, since greater success may increase contact and excessive conservatism may prevent completion.Source: §1 · p. 1
  • Runtime shields can repeatedly intervene while leaving the nominal policy unchanged, sustaining policy–shield mismatch and potentially pushing the robot into unfamiliar states.Source: §1 · pp. 1–2
  • FAILBANK uses observe-only CBF proposals, outcome-aware records, quiet anchors, and guarded LoRA updates to turn runtime evidence into persistent policy improvement.Source: §1 · p. 2
  • FAILBANK improves task success rate by 8.5 and 6.9 percentage points and reduces policy-induced cumulative cost by 35.6% and 23.8% over base policies.These gains are reported across two VLA backbones.Source: §1 · p. 2

2 Motivation

Runtime shielding may correct individual actions without changing the nominal policy, allowing repeated disagreement to impede task progress. FAILBANK instead uses runtime corrections as supervision for policy updates.

2 Motivation

  • Runtime shielding corrects individual actions but leaves the nominal policy fixed, so repeated disagreement can persist and impair task progress.Source: §2 · p. 2
  • On harder tasks, repeated shielding can reduce local cost yet redirect a capable policy until the episode times out.Source: §2 · p. 2
  • On the Level 2 onion task, AEGIS redirects the gripper away from both nearby hazard bottles and the target, while FAILBANK completes the task using runtime feedback.Source: §2 · p. 2

3 Related Work

The related work spans generalist VLA policies and evaluation, runtime safety mechanisms, and learning-based safety adaptation. The paper positions FAILBANK as converting runtime corrections into supervision for future policy behavior.

3 Related Work

  • Prior work spans generalist VLA policies and evaluation, runtime safety mechanisms, and learning-based safety adaptation.Source: §3 · p. 2
  • On representative Level-2 tasks, FAILBANK has the highest success across Mango, Apple, and Tomato while reducing policy-induced cost relative to Base.Source: Figure 1 · p. 3
  • The Level 2 onion trajectory contrasts AEGIS's repeated intervention and timeout with FAILBANK's observe-only supervision and successful placement.Source: Figure 2 · p. 3
  • π0 uses flow matching for continuous action generation, π0.5 extends this family toward broader generalization, and VLA-Arena evaluates policies across several controlled settings.Source: §3 · p. 3
  • Runtime safety approaches include CBF-based constraints, AEGIS's visual-grounding and CBF projection, and safety guidance during action generation; FAILBANK uses corrections as supervision for future behavior.Source: §3 · p. 3
  • Learning-based safety adaptation includes constrained learning, failure detection from internal representations, and privileged supervision with low-rank adaptation.Source: §3 · p. 4

4 Method

FAILBANK keeps execution under the current policy while turning CBF proposals and rollout outcomes into accumulated training records. Each round fits a fresh LoRA adapter and accepts it only after a held-out guard.

4.1 Problem setting and overview

  • The framework uses a fixed CBF teacher for counterfactual supervision, accumulates admitted records across rounds, and accepts a fresh LoRA update only if it passes a held-out guard.Source: §4.1 · p. 4

4.2 Stage 1: Observe and Label

  • During collection, the policy's nominal action is executed while the observe-only teacher logs proposals based on privileged simulator geometry.Episode outcomes are attached to records for later admission; privileged geometry is used during collection, not deployment.Source: §4.2 · p. 4

4.3 Stage 2: Admit and Weight

  • Admission assigns teacher proposals as targets for retained corrections and nominal actions as targets for successful uncorrected quiet anchors, while failed checks are discarded.CBF-triggered record weights reflect later safety, progress, and recovery evidence as well as repeated triggers, short-horizon cost, and barrier violations.Source: §4.3 · p. 5

4.4 Stage 3: Accumulate

  • The training bank combines newly admitted records with evidence retained from prior rounds, while a held-out validation batch remains separate from training.Source: §4.4 · p. 5

4.5 Stage 4: Update

  • Each update fits a fresh adapter from the base checkpoint using first-action flow-matching loss against assigned targets, then checks held-out loss and first-action drift.A failed candidate is discarded while the prior collecting policy and training bank are retained; an accepted policy collects the next round and deploys without the teacher or privileged geometry.Source: §4.5 · p. 5

5 Experiments

Experiments test FAILBANK on VLA-Arena static-obstacle tasks across difficulty levels and two flow-matching backbones. FAILBANK improves success and policy-induced cost relative to base policies, with gains extending beyond the collection setting.

5.1 Experimental setup

  • Experiments cover VLA-Arena static-obstacle Levels 1–2, collecting on Level 1 mango and comparing Base, AEGIS, and FAILBANK across both backbones.Evaluation uses matched initial states, task definitions, and conditions; Level 2 update data are not used.Source: §5.1 · p. 6

5.2 Main results

  • FAILBANK achieves joint SR and CCpolicy improvements on 8 task–backbone pairs, compared with 3 for AEGIS.Figure 4 shows AEGIS more often reducing cost while also lowering success; FAILBANK more often improves both objectives.Source: §5.2 · p. 6, Figure 4 · p. 7
  • SR gains of 8.5 and 6.9 percentage points and CCpolicy reductions of 35.6% and 23.8% versus base occurred on π0.5 and π0, respectively.Against AEGIS, mean SR gains are 25.4 and 9.5 percentage points on π0.5 and π0, respectively, at additional policy-induced cost.Source: §5.2 · p. 6, Table 1 · p. 7

5.3 Generalization across states, tasks, levels, and backbones

  • On π0.5, mean SR rises from 50.5% to 59.4% on Level 2 without Level 2 rollouts entering the update bank.FAILBANK also raises SR from 71.1% to 93.3% on held-out Level 1 mango states and from 82.5% to 88.7% across four unseen Level 1 tasks.Source: §5.3 · p. 7
  • On π0, mean SR rises from 51.0% to 62.7% across four unseen Level 1 tasks and from 35.7% to 40.0% on Level 2.Together, the reported results span states, tasks, difficulty levels, and both VLA backbones.Source: §5.3 · p. 8

6 Discussion

Ablations examine how collection mode and learning signals affect policy updates, while accumulated rounds show changing trade-offs between success and cost. The discussion situates FAILBANK among VLA, runtime-shielding, and safety-learning approaches, and states its current scope boundaries.

6.1 Collection interface and learning signals

  • Shield-in-loop collection turns eight otherwise successful rollouts into failures while rescuing five, and 88.9% of its failure records are shield-induced.Observe-only collection avoids this rollout-distribution shift and better preserves failures of the nominal policy.Source: §6.1 · p. 8
  • Adding quiet anchors keeps SR at 89% while lowering CCpolicy from 20.88 to 16.52 after correction-only training.The authors interpret corrective records as driving most success gains and quiet anchors as preserving successful behavior while further reducing safety cost.Source: §6.1 · p. 8

6.2 Number of Accumulated Self-Evolution Rounds

  • Across five rounds, SR peaks at Round 2 while CCpolicy reaches its minimum at Round 3; both objectives remain improved over Base in every round.The bank grows from 3.7k records at Round 1 to 22.7k at Round 5, while later-round outcomes fluctuate.Source: Figure 6 · p. 9, §6.2 · p. 9

6.3 Limitations

  • The implementation targets continuous flow-matching policies; other action formulations require adapted supervision and guard objectives.The evaluation focuses on static obstacles; dynamic scenes additionally require temporal obstacle prediction and teacher corrections for moving hazards.Source: §6.3 · p. 9

7 Conclusion

  • FAILBANK accumulates observe-only feedback as outcome-aware records and uses guarded LoRA updates to improve the underlying policy without a deployment shield.The conclusion reports improved joint success–cost performance across two flow-matching backbones and generalization beyond the collection setting.Source: §7 · p. 9

A.1 Generalist Vision-Language-Action Policies

  • Generalist VLA work spans large-scale pretraining, open policies, and varied action-generation approaches; FAILBANK leaves the π0 and π0.5 action-generation architecture unchanged.The experiments study runtime evidence as persistent supervision for these flow-matching backbones.Source: §A.1 · p. 14

A.2 Runtime Shields and Constrained Action Generation

  • Unlike CBF-based action projection before execution, FAILBANK records the CBF correction as an observe-only teacher signal and updates the policy for later rollouts and shield-free deployment.The related-work discussion describes other constraint-enforcement approaches during learning or execution.Source: §A.2 · p. 14

A.3 Safety Alignment and Failure Monitoring

  • FAILBANK combines teacher proposals and rollout outcomes as learning records, adds successful uncorrected actions as quiet anchors, and guards adapter updates against held-out flow-loss degradation and first-action drift.The discussion contrasts this approach with safety learning and runtime failure-monitoring methods.Source: §A.3 · p. 14

A.4 Safety Benchmarks and Evaluation

This section positions VLA-Arena among complementary robot-learning benchmarks and describes how the paper uses its task definitions and metrics. It also outlines runtime feedback as policy supervision and specifies the scope and reporting conventions for the evaluation.

A.4 Safety Benchmarks and Evaluation

  • VLA-Arena provides controlled Safety, Distractor, Extrapolation, and Long-Horizon categories, while ForesightSafety-VLA adds a diagnostic taxonomy of safety failures.The paper retains VLA-Arena’s official task definitions, success predicates, and cumulative-cost aggregation, reporting policy-induced CC as a diagnostic decomposition.Source: §A.4 · pp. 14–15

A.5 Runtime Feedback as Policy Supervision

  • FAILBANK’s loop observes and labels policy-controlled rollouts, admits and weights outcome-aware records, accumulates them in a failure bank, and accepts updates through a held-out guard.The collection-time teacher has privileged simulator geometry, whereas the deployed policy receives RGB observations and proprioception.Source: §A.5 · p. 15

B.1 Benchmark Hierarchy

  • VLA-Arena contains 170 tasks in 11 suites; its five Safety suites each have five tasks at each of three difficulty levels, from basic L0 to most challenging L2.The paper uses the benchmark’s official definitions, success conditions, and cost aggregation without modifying their semantics.Source: §B.1 · p. 15

B.2 Evaluation Metrics

  • SR measures trial success, CC is the trial-average sum of step costs, and CCpolicy subtracts initial-state cost; BRS equally weights normalized failure rate and policy-induced cost.Failures, timeouts, and zero-cost trials remain in the denominator, and policy-induced CC is a diagnostic decomposition rather than a replacement for official CC.Source: §B.2 · p. 15

B.3 Complete Per-Task Results

  • The reported table covers static-obstacle tasks across two backbones, with official CC alongside diagnostic CCpolicy and BRS; FAILBANK means exclude guard-rejected training orders.The caption notes that FAILBANK averages three π0.5 training orders and the two of three π0 orders that pass the training guard.Source: Table 2 · p. 16, §B.3 · p. 16

B.4 Base-Relative Score: Definition and Limitations

The appendix defines and tests a base-relative score, then audits the evaluation scope and probes where results do and do not generalize. Its analyses also show how collection mode changes the failure and correction records available for learning.

B.4 Base-Relative Score: Definition and Limitations

  • Small base denominators make BRS unstable, including undefined resamples, so it should be read alongside SR, official CC, and policy-induced CC.On L2-T3, only 1,273 of 2,000 bootstrap resamples have a nonzero cost denominator; on L2-T1, all methods have zero SR and BRS is determined entirely by CC.Source: §B.4 · p. 16
  • Across ten tasks, BRS is 0.498 for π0.5 FAILBANK versus 0.350 for AEGIS, but π0 scores of 0.443 versus 0.441 do not support a BRS advantage.For π0, the bootstrap probability that FAILBANK scores higher is 0.55 and the intervals overlap; neither comparison establishes that FAILBANK is safer than AEGIS.Source: §B.4 · p. 17

B.5 Training and Evaluation Levels

  • Level 1 measures adaptation beyond the checkpoints’ Level 0 source difficulty, while Level 2 tests transfer without any Level 2 rollout entering the main update bank.On 18 unseen L1-T2 initial states, the host-matched comparison gives 93.3% SR for FAILBANK and 71.1% for base, but the policy-induced CC difference is unsupported by the paired test.Source: §B.5 · p. 17, §B.8 · p. 19

B.6 Backbone Scope

  • The reported method comparison is limited to π0.5 and π0, whose checkpoints support the required matched update-and-evaluation chain.Other candidates have objective-level incompatibilities or, for SmolVLA, zero L1 success and a different serving path; extending π0 supervision to more chunk steps did not improve five-task success.Source: §B.6 · pp. 17–18

B.7 Task Coverage and Names

  • The task audit distinguishes matched comparisons from exploratory probes, and added Level 2 tasks include both a transfer gain and a zero-success coverage case.T4 reproduces a zero-shot transfer gain in two of three training orders with no significant regression in any order; T1 meets the registered floor criterion because base and all three updated policies have zero SR.Source: §B.7 · p. 18

B.9 Scope Boundary

  • The complete three-arm method evaluation uses the static-obstacle suite; the tested dynamic baseline has little headroom and the method does not establish transfer to dynamic tasks.Static Distractor, Extrapolation, Long Horizon, and LIBERO suites lack a benchmark cost predicate, while dynamic distractor was not evaluated; CC comparisons are limited to matching cost semantics.Source: §B.9 · p. 19

B.9.1 Hazard-Avoidance Validity and Headroom

  • Hazard Avoidance was not given a FAILBANK update: its repelling barrier can conflict with grasping or placing, and the withdrawal-teacher pilot failed the preregistered two-task gate.The object already lies in the cost region in 42–50 of 50 initial states per task, and several destination containers lie there as well; reduced L2-T4 cost reflected failed completion, not safer successful manipulation.Source: §B.9.1 · pp. 19–20

B.10 Dynamic-Obstacle Stress Tests

  • The static-trained adapter’s transfer to dynamic tasks is inconclusive, and the controlled L2-T0 round study declines from 86.0% SR in round one to 70.0% in round two.The paired test for the round comparison gives p=0.0386 even with the first round matched to the later 700-step budget; the dynamic-barrier diagnostic reports only limited short-horizon prediction headroom.Source: §B.10 · p. 20

C.4 Adapter-Level Collection Ablation

The collection ablation favors observe-only supervision: its bank passes the held-out guard and improves Level 2 apple success, while the larger shield-in-loop bank yields no deployable adapter. The merged sections also examine attribution, teacher execution, deployment, record construction, and bank accumulation.

C.4 Adapter-Level Collection Ablation

  • Level 2 apple SR rises from 8.0% to 34.0% with the 3,657-record observe-only bank, while the 5,099-record in-loop bank yields no adapter passing both guard conditions.Both banks use the same policy checkpoint and 800-step recipe with zero quiet-anchor weight; the comparison uses one training order and held-out batches of different sizes.Source: §C.4 · p. 21, Table 10 · p. 21

C.5 Attribution Controls

  • FAILBANK outperforms SHAM and SFTPOS in every training order and SFT0 in two; controls account for 11.3 and 12.0 of the 26.2-point Level 2 apple gain, limiting attribution to a universal residual effect.The three-order averaged p-values are 0.0011 against SFT0, 0.0029 against SHAM, and below 0.0001 against SFTPOS; other task comparisons are mixed or inconclusive.Source: §C.5 · pp. 21–22, Table 11 · p. 22, Table 12 · p. 22

C.6 Privileged Teacher Executed In Loop

  • The privileged teacher executed as a shield collapses onion success to zero, reduces apple success to zero, and sharply degrades mango, but inactive counters cannot verify individual projections.The matched-outcome divergence check finds outcomes differ from base in 70%, 87%, and 82% of apple, mango, and onion cells; collapse with privileged geometry does not require perception errors as an explanation.Source: §C.6 · p. 22, Table 13 · p. 22

D.1 Deployment Requirements

  • After acceptance, deployment uses only the updated VLA policy; privileged geometry, the teacher, runtime shield, QP, retrieval, memory lookup, and test-time update are not required.The limited wall-clock audit reports folded FAILBANK close to base and AEGIS approximately 24% higher, but does not establish isolated inference latency or adaptation speedup.Source: §D.1 · p. 23, Table 14 · p. 23

E.1.1 Barrier Teacher; E.1.2 Outcome-Aware Record Weight; E.1.3 Record Admission

  • The CBF teacher projects nominal translations to satisfy protected-object barrier constraints, and outcome-aware admission weights feasible triggered corrections while rejecting emergency and post-crossing records.Successful uncorrected actions become quiet anchors; failed trajectories retain only early pre-contact corrections, and an audit rejects failure targets identical to nominal actions.Source: §E.1.1 · p. 23, §E.1.2 · p. 24, §E.1.3 · p. 24

E.3 Host Matching and Hardware Confounding

  • Comparisons are pinned to one hostname because a historical audit found 66.9% versus 96.2% SR across different, unpaired offset subsets, which cannot isolate a hardware effect.A runtime assertion rejects cells from an incompatible host.Source: §E.3 · p. 24

E.2 Bank and Evaluation Ledger; E.4 Update Recipes and Training Budgets

  • The main static round-2 training fold has 6,535 records, and its 800-step update passes the guard at a 1.0065 full-chunk flow-loss ratio; the 12,343-record pre-filter pool is not trained directly.For the dynamic bank, 700 steps pass at 1.0985 while 800 fail at 1.1016; increasing quiet weight can restore the ratio while inflating triggered loss by up to 48×.Source: §E.4 · pp. 24–25, §E.2 · p. 24

E.5 Bank-Construction Audit

  • The bank-construction audit supports record retention, not a claim that a larger accumulated bank improves success; its supported benefit is lower L1-T1 policy-induced CC for the size-matched accumulated bank than the first-round bank, 10.61 versus 24.00.The controlled multi-round result uses reconstructed accumulated banks.Source: §E.5 · p. 25

E.6 Constraint-Activation Analysis

Observe-only activation changes across rounds and tasks, but activation and held-out guard checks are diagnostics rather than task-level outcomes. The guard thresholds are empirical, and sensitivity checks show the flow-loss limit binds more decisions than the drift limit in the archived range.

E.6 Constraint-Activation Analysis

  • Round-three observe-only activation is 14.36%, 19% below base, after rising to 19.47% in round one; contact-step, success, and cost changes against base remain null.The paired round-three directions include 40 decreases and 10 increases; task directions vary, with mango decreasing in all orders, onion increasing in two, and apple mixed.Source: §E.6 · p. 25, Figure 8 · p. 26

E.7 Held-Out Guard Diagnostics

  • The fixed guard rejects adapters above a 1.10 flow-loss ratio or 0.05 first-action drift, but these empirical limits are not calibrated bounds or task-performance criteria.All five reported adapters pass both limits, yet mixed-batch guard checks alone do not establish which round best balances success and cost.Source: §E.7 · p. 25

E.7.1 Guard-Threshold Sensitivity

  • Tightening the drift limit to 0.01 changes four archived decisions, while limits from 0.02 to 0.10 leave them unchanged; tightening the loss limit to 1.05 flips eleven entries across seven runs.Relaxing the loss limit to 1.15 admits a shield-in-loop update with worse Level 2 apple results than the accepted observe-only update; the audit supports a conservative empirical guardrail, not an optimal threshold.Source: §E.7.1 · pp. 26–27, Table 18 · p. 26

E.7.2 Training Duration and Guard Conservatism

Duration, quiet-anchor, and threshold diagnostics qualify what the held-out guard establishes: passing it is not itself evidence of a useful task update, and the tested flow-loss range does not validate it as a predictor of task degradation.

E.7.2 Training Duration and Guard Conservatism

  • No tested duration significantly lowers SR relative to the 800-step reference; 2,400 steps improve Level 2 apple under the registered paired test, but the guard is conservative over the measured 1.004–1.110 flow-ratio range.The 4,800-step adapter exceeds the 1.10 limit despite apple SR among the highest observed; selecting 2,400 steps from this evaluation set would require an independent held-out test.Source: §E.7.2 · p. 27, Table 19 · p. 27

E.7.3 Quiet Anchors and Chunk Supervision

  • On matched L1-T3, changing quiet-anchor weight from 0 to 0.2 preserves mean SR at 89.0% while lowering policy-induced CC from 20.88 to 16.52; multi-step supervision passes the guard but does not improve pooled SR over base.The quiet-anchor paired test gives p=0.0039; both multi-step variants remain below the accepted first-action-supervision adapter with quiet-anchor weight 0.5.Source: §E.7.3 · p. 27, Table 21 · p. 28

E.7.4 Record and Teacher Threshold Sensitivity

  • Threshold perturbations meet the low-sensitivity criterion for temporal boundaries, but not for risk or η-bin thresholds; changing η bins preserves records while altering their effective weights.These are CPU-side reconstruction diagnostics without policy retraining, so they do not establish corresponding SR or CC changes.Source: §E.7.4 · pp. 27–28, Table 22 · p. 28

E.7.5 Rejected-Update Diagnostics

  • With the guard disabled for diagnosis, the rejected shield-in-loop update is worse than its accepted observe-only counterpart on Level 2 apple, while the rejected round-2-only update does not differ significantly from the size-matched accumulated update.The latter comparison makes guard rejection a training-side decision, not evidence that accumulation improves task success.Source: §E.7.5 · p. 28

E.8 Diagnostics of the Dynamic-Obstacle Null Result

Dynamic-bank diagnostics find stable correction directions and more teacher triggers on benchmark-cost steps, but these intermediate signals do not explain the dynamic-task null result or substitute for matched task outcomes.

E.8 Diagnostics of the Dynamic-Obstacle Null Result

  • Mean correction-direction cosine is 0.814 for dynamic banks versus 0.747 for static banks, and teacher triggers occur on 28.6% of cost steps versus 11.8% of non-cost steps.These results do not establish that an unexecuted correction would prevent cost, and the diagnostics do not support failure-mode churn or sparse triggering as explanations for the dynamic null.Source: §E.8 · pp. 28–29

F.1 Behavior Overview

  • Level 2 apple, mango, and onion timing and policy-induced cost distributions retain all evaluated trials, including failures; blue shading represents three training orders, not a confidence interval.The distributions show timing and cost patterns that mean success rate and cumulative cost can conceal.Source: §F.1 · p. 29, Figure 9 · p. 29

F.2 Policy–Shield Coordination and Collapse

Reattaching a runtime shield can erase learned task progress, while timing and cost distributions show that gains are task-dependent rather than uniformly faster or cheaper.

F.2 Policy–Shield Coordination and Collapse

  • Stacking AEGIS onto FAILBANK lowers SR from 94.0% to 86.0% on mango and from 84.7% to 2.7% on onion.On both tasks, paired tests do not detect an SR difference between the stacked system and AEGIS alone.Source: Table 24 · p. 30, §F.2 · p. 30
  • In a matched onion replay, AEGIS redirects 80 of 300 controller steps and times out, while shield-free FAILBANK grasps and places the onion at step 189 with zero policy-induced CC.The CBF flags 33 of 180 controller steps in observe-only mode; the replay illustrates the mechanism, not its frequency.Source: §F.2 · p. 30

F.3 Completion Timing

  • By 200 steps, FAILBANK completes 17.3% of apple trials versus 4.0% for base and 93.8% of mango trials versus 88.7%, but onion reverses the timing pattern.Onion completion by 200 steps is 33.3% for FAILBANK versus 46.7% for base despite similar final SR; failures remain in the completion-curve denominator.Source: §F.3 · p. 30

F.4 Policy-Induced CC Distributions

  • On apple, FAILBANK reduces trials with policy-induced CC above 100 from 24.7% to 18.0%, but zero-cost trials fall from 36.0% to 6.7%; the distribution does not dominate.On mango, the fraction above 100 falls from 11.3% to 4.7%.Source: §F.4 · p. 30

F.5 Completion-Coupled Cost on Onion

  • On onion, at least 99.3% of trials in every arm have zero policy-induced CC, and nearly all CC differences belong to the benchmark’s initial-state/completion-coupled component.The cost gap does not establish a substantial contact-reduction benefit on this task.Source: §F.5 · p. 30

G.1 Out-of-Sample Breadth and Ablation Tests

The breadth and paired tests show uneven transfer: some task-level gains coexist with null or inconsistent aggregate tests. The section also distinguishes completion evidence from floor-task cost comparisons and documents the eligible result archive.

G.1 Out-of-Sample Breadth and Ablation Tests

  • At Level 2, AEGIS has lower SR than base on both backbones, while FAILBANK has more favorable than adverse SR directions versus AEGIS on each backbone.AEGIS versus base SR is 25.1% versus 50.5% on π0.5 and 26.3% versus 35.7% on π0; FAILBANK-versus-AEGIS counts are 144:16 and 84:29.Source: Table 25 · p. 31, §G.1 · p. 31
  • Across four unseen Level 1 tasks, π0.5 mean SR rises from 82.5% to 88.7%, but the paired test is null (p=0.61); onion carries a significant gain, while mean cost is also null.Onion SR is 90.0% versus 72.0% (p=0.0026), and the four-task mean-based cost test gives p=0.44.Source: §G.1 · p. 31, Table 25 · p. 31
  • On π0, the Level 2 SR gain is not significant (p=0.13), ten-task CC tests are null, and tomato’s SR gain accompanies higher policy-induced CC.Seed 1 is rejected without retraining or relaxing the guard; the reported mean uses seeds 0 and 2.Source: §G.1 · p. 31

G.1.1 Effect Sizes with Bootstrap Intervals

  • Positive bootstrap mean-gain intervals do not change null sign-test conclusions; the intervals resample initial states and exclude training-order variation.A positive mean can coexist with inconsistent per-state gains, and the reported intervals may understate recipe-level uncertainty.Source: §G.1.1 · pp. 31–32, Table 26 · p. 32

G.2 Offset-Paired Level 2 Statistics

  • The pre-registered Level 2 readout classifies tomato as transfer and lemon as a floor task because all methods obtain zero SR.Lemon’s cost differences are descriptive only.Source: §G.2 · p. 32, Table 28 · p. 33
  • Pooling five Level 2 tasks gives mean SR of 59.4% for FAILBANK versus 50.5% for base, but the pooled policy-induced CC difference is driven by lemon.Without lemon, policy-induced CC is 29.27 for FAILBANK versus 38.24 for base, with a nonsignificant paired difference (p=0.4159).Source: §G.2 · p. 32, Table 29 · p. 33

G.3 Archive Eligibility

  • The archive review adds 4,315 provenance-checked result cells and excludes comparisons with inconsistent hosts, obsolete branches, or invalid runtime audits.The release archive should include result files, adapter audits, manifests, and recovery logs for reconstructing aggregates.Source: §G.3 · p. 33

Related papers

Loading 2609.39820v1…