DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents

• Existing embodied harnesses support orchestration and self-improvement, but coarse planning can miss intermediate physical events and task-level failures may not identify which component needs revision.Source: §1 · p. 1
• The paper frames its design question as governing physical execution while turning execution evidence into validated revisions of reusable capabilities.Source: §1 · p. 2
• DynaHarness’s shared contract grounds, bounds, and can refuse commands across analytic skills, recovery skills, and a frozen VLA while recording evidence for revision.Source: §1 · p. 2
• Its slow brain proposes capabilities and symbolic arguments, while the fast brain governs execution through monitoring, substitution, recovery, and replanning.Source: §1 · p. 2
• Execution records localize faults and direct targeted revisions of reusable capabilities or execution mechanisms, with paired regression checks governing admission.Source: §1 · p. 2

• Robot-agent systems combine policy or program composition, 3D grounding, executable tools, verification, recovery, and runtime intervention around pretrained policies.The cited approaches include Code as Policies, VoxPoser, ReKep, Harness VLA, EmbodiedSkills, ETA, Show-Harness, Thea, PhyAgentOS, HarnessWAM, and UniIntervene.Source: §2 · p. 3
• DynaHarness records command grounding, refusal, execution bounds, and completion for later improvement and uses command-level provenance for automated diagnosis and change admission.Source: §2 · p. 3
• Self-improvement methods repair traces or programs, acquire skills, rehearse changes, or use validation-gated updates to improve robot agents.Examples include ASPIRE, GaP, InSight, ENPIRE, and Zetta.Source: §2 · p. 3

3.1 OVERVIEW

• The physical harness connects slow planning, fast command governance, and offline self-evolution through a library of bounded analytic skills, recovery skills, and a frozen VLA.Source: §3.1 · p. 3

3.2 SLOW BRAIN: SEMANTIC REASONING

• The slow brain returns a capability and symbolic arguments without poses, leaving geometry resolution, success checks, refusal, and replanning to the fast brain.Source: §3.2 · p. 4

3.3 FAST BRAIN: PHYSICAL GOVERNANCE

• The fast brain grounds advised steps into commands, checks parameters and preconditions, refuses unresolved commands, and can continue, recover, switch capabilities, or request replanning.Source: §3.3 · p. 4

3.4 PHYSICAL EXECUTION CONTRACT

• Every robot-facing command is grounded, bounded by budget and lease, terminated on recorded conditions, and logged with observations, decisions, outcomes, and reasons.Refusals are recorded even when no action is dispatched, distinguishing unresolved commands from failed capability executions.Source: §3.4 · p. 4

3.5 CAPABILITY EXECUTION

• Analytic skills, recovery skills, and the frozen VLA share the same budget, lease, monitoring, and task-verdict interface, while analytic skills can complete entire tasks.Source: §3.5 · p. 5

3.6 FAILURE ATTRIBUTION

• For failed episodes, automated attribution assigns the first matching label among ordered diagnostic layers, covering infrastructure through recovery and prioritizing upstream conditions.The diagnostic granularity is controlled by N.Source: §3.6 · p. 5

3.7 SELF-EVOLUTION AND PAIRED VALIDATION

• A targeted revision changes a reusable capability or execution mechanism, then paired validation and broader regression checks determine whether it is admitted.The revision is expressed in physical quantities rather than keyed to a task.Source: §3.7 · p. 5

4.1 EXPERIMENTAL SETUP

• LIBERO-Pro evaluation covers four 10-task cells with 20 seeds each, using development seeds 21–40 and separate post-selection sets of 800 new states and 261 untouched official states.LIBERO-Plus adds 10,030 tasks across seven perturbation categories in a separate configuration.Source: §4.1 · p. 5

4.2 MAIN RESULTS

• 74.25% on four LIBERO-Pro cells exceeds PhyAgentOS at 20.3% and Harness VLA at 6.0% under matched conditions; the gain over frozen policy is 58.0 percentage points.On hardware, success is 90% for circle pick-and-place, 80% for cup stacking, and 70% for both bread pick-and-place and drawer placement.Source: §4.2 · pp. 6–7, Table 1 · p. 6, Table 3 · p. 7

4.3 POST-SELECTION TRANSFER

• 75.2% on 800 newly sampled states exceeds frozen π0.5 at 17.5%; the 14.3-point development gain transfers as 14.4 points, while the final 1.0-point increment remains uncertain.On 261 untouched official states, DynaHarness reaches 77.0% versus 16.5% for π0.5.Source: §4.3 · p. 7, Table 2 · p. 7

4.4 FAST-SLOW ROBOT AGENT

• Across 5,805 episodes, failures spend 173 steps after last progress versus 51 for successes, but replacing these suffixes yields 22 alternative-only versus 14 control-only wins (p = 0.24).Latching transient success improves replay success from 25% without latching to 95% with latching and a chunk-level read.Source: §4.4 · p. 7

4.5 DYNAMIC PHYSICAL HARNESS

• 570 of 770 archived control episodes never invoke the VLA and succeed 96.7% of the time; analytic execution supplies most successful behavior, while policy calls concentrate in difficult episodes.In retained failures, VLA execution reaches 91.4% of steps in the hardest cells, where no cell exceeds 35%.Source: §4.5 · p. 8

4.6 ATTRIBUTION-DRIVEN SELF-EVOLUTION

• Two drawer-capability changes raised cell success from 55% to 90% and 100% over 20 seeds, and Goal success from 75.25% to 78.0% over 400 episodes.A candidate that passed its paired gate lost 79 successes on five stove-hob cells in full-round evaluation and was reverted, motivating broader validation.Source: §4.6 · p. 9, Figure 7 · p. 9

4.7 WHAT MAKES THE PHYSICAL HARNESS WORK?

• Full reaches 74.0% versus 63.88% for A2static on the same 800 development states; the 10.125-point difference reflects a joint dynamic-execution gain, not individual causal effects.Across 783 trace-matched pairs, Full records 566 failure/escalation replans, 471 substitutions, and 141 recovery insertions, versus zero for A2static.Source: Table 2 · p. 7, §4.7 · pp. 9–10

5 CONCLUSION

• The frozen system reaches 75.2% on new initial states of the same cells versus 17.5% for frozen π0.5, while snapshot gains transfer from development to new states.Removing analytic skills lowers success from 74.0% to 16.6%; closed-loop execution with the same library improves over nominal one-step replanning by 10.125 points.Source: §5 · p. 10

AI USE STATEMENT

• The authors used Claude and ChatGPT for limited language assistance, not for hypotheses, experiment design, implementation, experiments, analysis, or scientific conclusions.The authors reviewed and revised all AI-assisted edits and take responsibility for the paper’s content and conclusions.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

• The supplementary package provides implementation, evaluation configurations and scripts, and run records for reproducing aggregate results and paired comparisons.The paper identifies where benchmarks, evaluation settings, execution contracts, admission rules, and retained per-episode records are specified.Source: REPRODUCIBILITY STATEMENT · p. 11

A METHOD DETAILS

• The execution interface grounds capability parameters and preconditions, records refusal reasons, and bounds commands by budgets, leases, and a velocity envelope.LIBERO uses simulator state for geometry and its task predicate for completion; hardware scene information comes from camera images. The 50 Hz harness checks do not replace the robot controller’s 100–1000 Hz servo loop.Source: A METHOD DETAILS · p. 14
• Analytic skills handle geometry-determined motion, recovery skills restore an earlier pose or release and retreat, and the VLA handles contact dynamics not determined by geometry.The fast brain can stop the VLA when the task verdict latches and applies the same budget to it as to an analytic skill.Source: A METHOD DETAILS · p. 14
• Ordered attribution labels an episode with the first matching diagnostic layer, prioritizing upstream checks; the label is a revision hypothesis, not a causal effect.The evaluated configuration uses 13 diagnostic categories, and the experiments do not establish an optimal granularity or sensitivity to N.Source: A METHOD DETAILS · p. 15
• The evolution loop reproduces named faults from the harness-created state, revises physical parameters, and admits changes to the registry only after paired validation.Rejected changes are reverted; the evidence store also logs fast-brain decisions and outcomes.Source: A METHOD DETAILS · p. 15

B EXPERIMENTAL SETUP DETAILS

• The main LIBERO-Pro comparisons use four task cells, Qwen3-VL-4B, the frozen public π0.5 checkpoint, matched budgets and protocols, and 800 episodes.Each cell has 10 tasks and 20 episodes per task; the matched comparison controls the listed model, policy, and evaluation settings.Source: B EXPERIMENTAL SETUP DETAILS · pp. 15–16
• The evolution registry tracks capability additions after fault probing, targeted revision, and paired validation.Figure 9 depicts the loop and registry.Source: Figure 9 · p. 16
• Tables 5 and 6 document run settings and result sources, distinguishing the authors’ runs from external references and noting author-supplied entries.Candidate-development effort and skill libraries remain system-specific despite the matched listed settings.Source: B EXPERIMENTAL SETUP DETAILS · pp. 16–17, Table 5 · p. 16, Table 6 · p. 17
• The evaluation reports success over all episodes, Wilson intervals where relevant, paired discordant outcomes and exact tests, and step shares by skill type.Capability conversion compares source-cell success counts before and after admission on the same seeds; infrastructure-lost episodes count as failures.Source: B EXPERIMENTAL SETUP DETAILS · p. 17

B.1 REAL-WORLD PLATFORM AND TASK INSTRUCTIONS

• Across 40 physical trials, success was 90% for ring placement, 80% for cup stacking, and 70% each for bread and drawer placement.The platform uses a Qwen 4B language model, SAM3 segmentation, a UR7e arm, and RealSense D435 and D405 cameras; object placements varied across trials.Source: §B.1 · p. 18

C EVALUATION DETAILS

• The analyzed round is stat30, while the reported development aggregate for the Table 1 agent is 74.25%; a complete paired remeasurement scores 74.1%.The new concurrent mechanism-ablation control scores 74.0%; the post-selection results use separate state banks.Source: C EVALUATION DETAILS · p. 18
• The archived frozen LIBERO-Pro run scores 46.7%, but 1,064 tick-limit failures under its 600-tick configuration confound its interpretation as standard-LIBERO transfer.The failures are distributed across Spatial, Object, Goal, and Long suites.Source: C EVALUATION DETAILS · p. 18
• On LIBERO-Plus, camera viewpoint success is 66.5% and robot initial state success is 76.1%; all 1,568 failures share the step-budget-exhaustion label.The proposed explanation for category differences is a hypothesis, and the run used a LIBERO-Plus-specific configuration.Source: C EVALUATION DETAILS · p. 18
• Figure 10 shows four real-world execution sequences, each arranged from an initial scene through four stages with its language instruction below.The sequences depict ring placement, cup stacking, bread placement, and ring placement in a drawer.Source: Figure 10 · p. 19
• 76.0% over 712 episodes, excluding 88 workcell connection losses, compares with 67.6% over all 800 episodes in the analyzed round.Every paper success rate counts those infrastructure losses as failures; Figure 11 compares outcome composition with the round carrying the rejected candidate.Source: C EVALUATION DETAILS · p. 18, Figure 11 · p. 19

C.1 POST-SELECTION EVALUATION

• The champion and its prompt, library, parameters, budgets, and verifier were frozen before evaluation on 800 newly sampled states; a second block uses 261 untouched official states across 31 cells.The official-state block is not a complete benchmark block; both evaluations follow system selection.Source: §C.1 · p. 19, Table 14 · p. 22
• From stat23 to the champion, improvement is 14.25 points in development and 14.375 points on new states; the last increment alone does not establish a positive effect at the 0.05 level.The last increment adds 8 successes over 800 episodes, with 12 versus 4 discordant wins and p = 0.0768.Source: §C.1 · p. 20, Table 15 · p. 22
• Post-selection execution accounting reran every cell-system job assigned to the flagged host, including successful episodes, while preserving original and replacement records.Replacement jobs used the same system and state bank; other hosts’ results were retained.Source: §C.1 · p. 20
• 75.2% for the champion versus 65.8% when five evolved capabilities are withheld, a 9.5-point difference across 800 new states.The withheld arm has 51 flagged failures versus 2 for the champion, so the contrast includes runtime effects of removal.Source: §C.1 · p. 21

D ADDITIONAL RESULTS

• The additional-results section provides the full account behind Sections 4.4–4.7, including records and diagnostic comparisons moved out of the main text.Figure 12 accompanies this account.Source: D ADDITIONAL RESULTS · p. 22

D.1 CAPABILITY AND CLOSED-LOOP EXECUTION ABLATIONS

• The ablation arms shared frozen π0.5, Qwen3-VL-4B, prompts, parameters, and environment-step budgets with their concurrent control; the 7-skill removal tested the final library, not the entire development process.The final library included initially available and subsequently added or refined capabilities.Source: §D.1 · pp. 22–23
• Removing analytic contact skills cost 96, 129, 121, and 113 successes across the four suites, reducing every suite to 27.0% success or below.The ablation arms covered 800 episodes each, with 200 per suite.Source: §D.1 · p. 22, Table 16 · p. 23
• Recovery-capability removal cost 7 successes overall, including 7 in 10-T; the other suites changed by −1, 0, and +1.Removing both analytic and recovery capabilities left planning and the execution contract at 15.8%, near the bare policy’s 16.2%.Source: §D.1 · pp. 22–23, Table 16 · p. 23
• A2static and A2seq lost 81 and 82 net successes to Full, respectively; A2static added only one net success over A2seq, so nominal replanning alone did not reproduce dynamic execution.The same-program inert-switch replicate measured 74.0% versus 74.1% on 800 episodes, with 5/6 discordant pairs and p = 1.Source: §D.1 · p. 23, Figure 13 · p. 23, Table 17 · p. 24
• Among 770 archived control event stores, 570 episodes did not invoke π0.5 and succeeded at 96.7%, versus 6.0% for the 200 episodes that invoked it; these are conditional, not randomized, comparisons.Policy calls concentrated in hard episodes after analytic execution struggled.Source: §D.1 · p. 23

A. Aggregate contrasts

• Full gains 10.125 points over A2static, while nominal replanning adds only 0.125 points over A2seq on the same 800 initial states.The 0.125-point difference is an observed small difference, not an equivalence test; Full’s gain evaluates its dynamic mechanisms jointly.Source: A. Aggregate contrasts · pp. 24–25
• 74.00% for Full versus 63.75% for A2seq over 800 paired episodes, a +10.25-point difference on the development block.This comparison evaluates execution design, not blind generalization to unseen tasks.Source: A. Aggregate contrasts · p. 26

D.2 FAST-SLOW ROBOT AGENT

• 95% success with latching and a chunk-level read compares with 25% without sampling and latching and 80% for the bare policy.That replay changes both sampling and latching; a separate paired ablation holds sampling fixed.Source: §D.2 · p. 27
• The fast brain and planner have distinct median latencies: 0.046 ms for a scheduler decision and 913 ms for a planner call.The fast brain runs on a nominal 0.5 s schedule, while the slow brain is called on demand.Source: §D.2 · pp. 27–28
• The learned fast brain and the rule each score 593 successes over 800 episodes, but the learned policy escalates less often on reachable cells.It has limited counterfactual training data because 99.2% of logged decisions continue the current command; choosing between analytic skill and VLA is outside its action space.Source: §D.2 · p. 28

D.3 DYNAMIC PHYSICAL HARNESS

• Turn-knob object completes only 21% of its invocations even as its source cell rises from 0% to 95%; retained routing records show failures attributed to the planner spend 351 of 363 steps under the VLA.The routing sample emphasizes hard cells and does not estimate the effect of removing the VLA.Source: §D.3 · pp. 30–31

D.4 ATTRIBUTION-DRIVEN SELF-EVOLUTION

• Drawer-focused probes and revisions raised one cell from 55% to 100% and Goal success from 75.25% to 78.0% over 400 episodes, while the earlier local change was retained for development without passing the full admission gate.Development gains are local rather than uniform; 27 of 40 cells gained at least five successes, while seven remained at zero.Source: §D.4 · p. 31
• A generalized cavity-entry candidate caused five cells to lose 79 successes and was reverted; selected case studies complement, but do not replace, the aggregate results.A separate mirror-recovery candidate won on targeted cells but scored 70.75% versus 72.75% in full evaluation, so the incumbent was kept.Source: §D.4 · p. 32, E CASE STUDIES · p. 32

F PER-TASK RESULTS

• Frozen-policy references differ by at least 50 percentage points on nine of 40 tasks, with differences in both directions.For example, goal task[1] is 95% in the cited reference and 15% in the early-seed block, while 10 task[0] is 5% and 55%, respectively.Source: F PER-TASK RESULTS · p. 32, Table 27 · p. 34
• At round 39, the agent succeeds on at least 80% of seeds in 27 of 40 tasks and on none in five.Four zero-success tasks involve a drawer; the fifth closes a microwave.Source: F PER-TASK RESULTS · p. 33
• The selected figures depict an infeasible-capability refusal, an evolved capability beside the frozen policy, and two episodes whose frozen-policy counterparts fail on the same seeds.Figure 16 compares success counts within each seed block, not across the early-seed and development blocks.Source: Figure 17 · p. 33, Figure 18 · p. 33, Figure 19 · p. 34

G BASELINE RECORDS

• The reported Zetta drawer-task diagnostics include failed online recovery calls, a completion report without the drawer target being met, and three same-seed candidate gates that all failed.Eight of nine candidate proposals failed validation.Source: G BASELINE RECORDS · p. 35, Table 30 · p. 36
• In a second campaign, PhyAgentOS scores 21.3% with GPT-4o-mini and 20.3% with Qwen3-VL-4B, while ENPIRE scores 5.4%; Zetta and ASPIRE record no executed success.These are four-cell LIBERO-Pro comparisons with 200 episodes per cell; ASPIRE’s runs again ended without an executed episode.Source: G BASELINE RECORDS · p. 36
• Three diagnostic Harness VLA episodes include one recovery after an off-plate release and failures involving bowl placement and loss of object identity.The episodes use turn-by-turn segmentation, back-projection, motion, and grasp tools, with benchmark termination signals observed after actions.Source: G BASELINE RECORDS · p. 36, Table 31 · p. 36
• Harness VLA with Qwen3-VL-4B and the frozen public π0.5 succeeds in 48 of 800 episodes, compared with DynaHarness at 74.25% and Harness VLA at 6.0% in the matched comparison.The 48 successes are 28 on Goal-T, 14 on Goal-S, six on 10-T, and none on 10-S.Source: G BASELINE RECORDS · p. 36
• The Qwen3-VL-4B CaP-X adaptation scores 5.0%, and the reported ENPIRE run records three successes in 40 tasks.The CaP-X runs are characterized as an adaptation and excluded from Table 1; ENPIRE’s recovered raw files cover only 18 tasks, so its total rests on the run summary.Source: G BASELINE RECORDS · pp. 35–36

H TWO-STAGE ADMISSION OF A RECOVERY CANDIDATE

• C2 scores 61.7% on six targeted cells, versus 55.8% for the incumbent, with no net change on four regression cells.C2’s mirror recovery fires 36 times and reaches all six targeted cells, but two cells show no success change.Source: H TWO-STAGE ADMISSION OF A RECOVERY… · pp. 36–37, Table 32 · p. 37
• In the full 800-episode round, C2 scores 70.75% against the incumbent’s 72.75%, so the incumbent is kept.Twenty of the 22 lost episodes are in goal swap, including losses in cells the candidate did not target.Source: H TWO-STAGE ADMISSION OF A RECOVERY… · p. 37, Table 32 · p. 37
• A later matched comparison on 200 Goal-S task-and-seed pairs scores 76.5% for each build, with two pairs won only by each side.The earlier Goal-S loss did not reproduce.Source: H TWO-STAGE ADMISSION OF A RECOVERY… · p. 37, Table 32 · p. 37

Evaluation scope

• Evaluation tests new initial states in the same 40 LIBERO-Pro cells and 261 untouched official states across 31 cells, not new task families or physical domains.Executor comparisons use development states and jointly evaluate execution mechanisms; A2seq also changes the planning horizon and output schema.Source: I LIMITATIONS · p. 37

Improvement and deployment

• LIBERO completion uses the benchmark predicate, while physical deployment needs a separately specified completion interface and scene description.The deployed fast brain is deterministic; its learned variant matches its success count in the reported comparison.Source: I LIMITATIONS · p. 37

Records and coverage

• Earlier step-share analyses cover 8 of 40 cells, some rounds and baseline campaigns retain only summaries, and the no-evolution arm lacks an attribution field.The final development score is an archived aggregate; a separate remeasurement supplies the historical paired comparison.Source: I LIMITATIONS · p. 37