AdaHVLA: Adaptive Harnesses for Long-Horizon Vision-Language-Action Execution

I. INTRODUCTION

• Long-horizon tasks require robots to retain relevant history, track remaining work, and establish conditions that allow later stages to succeed.A locally appropriate action may still fail to advance the full task.Source: §I · p. 1
• AdaHVLA adapts code-based coordination policies that govern VLA context, stage advancement, responses to deviations, and task completion.Separate working contexts handle evidence analysis, harness revision, and behavioral assessment; testable hypotheses link revisions to predicted effects.Source: §I · p. 2
• A revision graph preserves alternative harnesses and links evidence, hypotheses, revisions, and observed effects to guide refinement within and across tasks and environments.Both successful and unsuccessful attempts can inform later revisions.Source: §I · p. 2
• AdaHVLA couples agent reasoning and memory with VLA execution through editable policies refined using robot experience.The adaptation algorithm separates evidence analysis, harness revision, and behavioral assessment, with hypotheses and a revision graph guiding evaluation.Source: §I · p. 2
• Navigation test success rises from 22.5% with the initial harness to 31.7–57.5% across models; manipulation also improves across three backbones, with real-world deployment illustrating stable stage execution.The evaluation covers simulated quadruped navigation and manipulation.Source: §I · p. 2

A. Long-Horizon Robot Execution

• Long-horizon robot work addresses skill composition and subtask dependencies through phase-aware training, temporal memory, progress estimation, planning, and programmatic or spatial representations.AdaHVLA builds on hierarchical systems by adapting coordination policies between agent reasoning and VLA execution through robot experience.Source: A. Long-Horizon Robot Execution · p. 2

B. Embodied Adaptation and Harness Optimization

• Agent adaptation methods retain feedback or search executable workflows and agent components, but embodied adaptation also requires validating the closed-loop behavior those implementations induce.The cited approaches include Reflexion, automated agent design, AFlow, and GPTSwarm.Source: B. Embodied Adaptation and Harness… · p. 2
• AdaHVLA separates evidence analysis, harness revision, and behavioral assessment; hypotheses and a revision graph connect policy changes to observed effects for continued refinement.The graph retains successful and unsuccessful attempts to guide candidate selection within and across tasks and environments.Source: B. Embodied Adaptation and Harness… · p. 2, Figure 2 · p. 3

A. Overview

• The harness guides VLA execution from observations and task history, while a multiagent process turns rollout evidence into hypotheses, code revisions, and behavioral assessments.A revision graph links these records to alternative policies, supporting repeated refinement and adaptation across tasks and environments.Source: A. Overview · p. 3

B. Problem Formulation

• A task instance comprises an instruction, environment, and initial robot and scene state; at each decision boundary, the harness uses interaction history and runtime state.The interaction history includes the instruction, observations through the current time, and prior actions.Source: B. Problem Formulation · p. 3
• The coordination policy produces a VLA instruction and observation context, an updated runtime state, and a task-termination flag; the VLA produces a command or action sequence.A rollout keeps the source revision fixed, initializes runtime state at its start, and ends when the termination flag is set.Source: B. Problem Formulation · p. 3
• Adaptation uses a finite task set while leaving the robot configuration and VLA parameters unchanged; behavioral assessments guide the search for task completion.Success is represented by a binary benchmark indicator.Source: B. Problem Formulation · p. 3

C. Harness VLA Policy

• Editable coordination policies determine what state persists, what instruction and context reach the VLA, when progress is committed, and when recovery or completion logic takes control.The harness shapes the executor’s request and context; the VLA supplies local action behavior.Source: C. Harness VLA Policy · pp. 3–4
• Task policies advance goals based on observable progress criteria and whether previous actions established the conditions needed for the next goal.They can retain the original instruction or provide a local goal to the VLA.Source: C. Harness VLA Policy · p. 4
• The harness selects observations, progress records, and ongoing corrections for reasoning, with retention rules that can differ between route reasoning and local execution.An earlier visual buffer may remain useful for route reasoning after a turn but become stale for local execution.Source: C. Harness VLA Policy · p. 4
• Recovery guidance can persist through corrective turns into forward motion, while completion checks consider remaining goals rather than relying on a local stopping signal.Within a rollout, observations and runtime-state updates can change instructions; between rollouts, adaptation changes the code while leaving VLA parameters unchanged.Source: C. Harness VLA Policy · p. 4

• Evidence analysis, harness revision, and behavioral assessment use distinct agent contexts that exchange structured records rather than a shared conversation.Source: D. Policy Revision and Assessment · p. 4
• For each rollout, an analysis agent separates observed events from explanations and records testable hypotheses with a suspected mechanism, evidence, expected effect, and observable indicator.One example is retaining a goal when the robot is off-route so guidance remains focused on an unfinished passage.Source: D. Policy Revision and Assessment · p. 4
• Before a new rollout, a revision agent records the patch’s hypothesis and predicted effect, then a separate source and interface review checks execution eligibility.The revised source is formed by applying a code patch to the prior source.Source: D. Policy Revision and Assessment · p. 4
• A separate assessor compares parent and revised rollouts on the same task in a fresh context, evaluating targeted effects, overall behavior, and final success separately.Source: D. Policy Revision and Assessment · p. 4
• The attribution gate updates hypothesis support only when the mechanism was triggered before trajectories diverged; repeat rollouts, not one successful episode, are needed to establish repeatability.Targeted improvements strengthen support, while unchanged or degraded effects weaken it when a candidate is discontinued; otherwise support is retained.Source: D. Policy Revision and Assessment · p. 4

E. Revision Graph and Memory

• The revision graph links code versions, hypotheses, rollout evidence, and behavioral assessments so related attempts remain retrievable across branches.Source: E. Revision Graph and Memory · p. 4
• Within a task, candidates are prioritized by task success, then targeted and overall behavioral effects; active-leaf and revision-depth limits govern further exploration.A candidate can be extended when assessment finds a targeted improvement or newly exposed failure and capacity remains, while inactive candidates and evidence stay available.Source: E. Revision Graph and Memory · p. 4
• Across tasks, condensed adaptation memory retains coordination issues, tested hypotheses, attempted changes, observed effects, and evaluation conditions, while runtime state and local exploration depth reset.The selected harness and retained agent context persist across task switches, and revision contexts are rebuilt for unrelated assignments.Source: E. Revision Graph and Memory · p. 4
• The benchmark notes identify Terminal-Bench 2.1 and tool-free MMMU-Pro scores as references that follow their source evaluation protocols.Source: E. Revision Graph and Memory · p. 5
• The final harness is selected using adaptation evidence alone, while its source stays fixed on held-out tasks and online reasoning and runtime-state updates continue.Source: E. Revision Graph and Memory · p. 5

IV. EXPERIMENTS

• The experiments assess whether execution experience improves coordination on held-out instances across task demands, robots, and model configurations.Source: §IV · p. 5

A. Long-Horizon Benchmarks

• NaVILA-LH uses 50 NaVILA-Bench instances, with 10 assigned to adaptation and 40 to testing; tasks require traversing passages, following turns, and reaching an intended destination.Source: A. Long-Horizon Benchmarks · p. 5
• VLA-Arena Long Horizon uses 50 instances per level with a 10/40 adaptation-test split; L1 combines independent skills, while L2 adds longer workflows and stage dependencies.The manipulation tasks test whether object states resulting from earlier actions allow later goals to be achieved.Source: A. Long-Horizon Benchmarks · p. 5

B. Setup

• Simulation uses a Unitree Go2 in Matterport3D through Isaac Lab and Isaac Sim for navigation, and a Panda robot in RoboSuite with MuJoCo for manipulation.Navigation uses a pretrained locomotion controller; manipulation observations are RGB and actions are seven-dimensional.Source: B. Setup · p. 5
• Comparisons include Single VLA, Initial Harness VLA, and AdaHVLA; the initial harness is not code-revised, and VLA parameters and controllers remain fixed.Runtime reasoning uses Qwen3.7-Flash; navigation uses NaVILA, and manipulation uses SmolVLA, OpenVLA-OFT, and π0.5; published capability references do not guide adaptation.Source: B. Setup · p. 5
• Success rate follows each benchmark’s completion criterion, and test SR measures reuse beyond adaptation instances; reported rates average three runs on the same tasks.The means represent 30 adaptation or 120 test evaluations, not distinct tasks or the total adaptation rollout budget.Source: B. Setup · pp. 5–6
• Figure 3 indexes candidates from the Initial Harness VLA at 0 through candidates 1–15; points report mean test SR over three adaptation restarts, with sample SD added in right panels.The runtime reasoning agent and NaVILA remain unchanged, and test outcomes do not guide revisions or candidate selection.Source: Figure 3 · p. 6

C. Quadruped Navigation

• Mean test SR rises from 22.5% with Initial Harness VLA to 31.7–57.5% across six adaptation models, adding 9.2–35.0 percentage points.The runtime reasoning agent and NaVILA remain unchanged, and final harnesses are selected using adaptation evidence alone.Source: C. Quadruped Navigation · p. 6

D. Robot Manipulation

• Manipulation adaptation improves test success in every backbone-level setting, reaching 25.8–55.8% on L1 and 10.8–38.3% on L2.With π0.5, adaptation adds 30.8 points on L1 to reach 55.8%, and raises L2 success from 10.0% to 38.3%, a 28.3-point gain.Source: D. Robot Manipulation · p. 6
• π0.5 outperforms the larger OpenVLA-OFT on both levels, but this comparison does not isolate model size.Source: D. Robot Manipulation · p. 6
• Across all three backbones, absolute gains and final SRs are lower on L2 than L1, but workflow length and stage dependencies vary together, so their individual effects cannot be separated.Source: D. Robot Manipulation · p. 6

V. DISCUSSION

• Adaptation adds value beyond the initial harness by using behavioral evidence and separately managed contexts to refine coordination for each robot.The overall gains are consistent with cumulative refinement, but progress varies across adaptation models and later candidates can regress.Source: §V · p. 6, A. Cumulative Policy Improvement · p. 6

A. Cumulative Policy Improvement

• Linked revision evidence and retained agent context let later candidates build on earlier attempts across tasks, while preserving alternatives when a recent change proves less useful.The discussion notes that gains vary across adaptation models and later candidates can regress.Source: A. Cumulative Policy Improvement · p. 6
• Fig. 4 compares execution policies after adaptation and adaptation processes from a common initial harness; manipulation uses π0.5 [35].Error bars denote sample SD over three runs.Source: Figure 4 · p. 6
• Fig. 5 spans a real-world Unitree Go2 deployment and simulated navigation and manipulation trajectories comparing Single VLA with harness policies during and after adaptation.The real-world panel marks deviation and goal-directed motion with red and green arrows.Source: Figure 5 · p. 7
• The following ablations examine graph and context choices alongside behavioral versus success-only candidate selection.They assess continued refinement through organized reusable evidence and appropriately scoped working context.Source: A. Cumulative Policy Improvement · p. 7

B. Ablation Study

• The ablations remove execution functions without readaptation or repeat adaptation from a common initial harness using the same candidate budget.Full retains all coordination and adaptation components; execution variants include changes to task planning, visual refresh, recovery, completion checks, and runtime context.Source: B. Ablation Study · p. 7
• Navigation SR falls from 57.5% to 29.2% without completion checks and to 33.3% with compact memory, the two largest execution-ablation losses.The compact-memory condition retains task and concise progress rather than eliminating memory.Source: B. Ablation Study · p. 7
• For manipulation, removing visual refresh gives the largest L1 decrease, from 55.8% to 37.5%; removing recovery lowers L2 from 38.3% to 23.3%.Compact memory lowers L2 to 22.5%, versus 53.3% on L1; the contrasts are consistent with greater recovery and context needs in longer workflows with stage dependencies.Source: B. Ablation Study · p. 7
• Without the revision graph, navigation SR falls from 57.5% to 42.5% and manipulation L2 from 38.3% to 26.7%, despite versions and rollback remaining available.Manipulation L1 changes from 55.8% to 50.8%.Source: B. Ablation Study · p. 7
• Under the same candidate budget, SR Select reaches 38.3% navigation, 48.3% L1, and 25.0% L2, performing worse than selection informed by behavioral evidence.Success alone cannot show whether a patch had its predicted effect or whether another failure masked a local improvement.Source: B. Ablation Study · p. 7
• Shared Context lowers navigation SR from 57.5% to 49.2% and L2 from 38.3% to 30.0%, while L1 changes little, from 55.8% to 55.0%.Together with the graph ablation, this supports organized persistent revision evidence and working contexts with separate scope and lifetime.Source: B. Ablation Study · p. 7
• Assigning analysis, revision, and selection to one agent or merging observations with hypotheses also lowers navigation SR.For L1, W/O Recover and W/O E&H reach 56.7% versus 55.8% for Full; these close means over three runs do not establish a reliable ordering.Source: B. Ablation Study · p. 7

C. Execution Stability

• Simulated navigation and manipulation traces show more goal-directed progression after adaptation, complementing held-out success-rate gains with examples of coordinated execution.Across both robot platforms, the harness maintains task-level progress while the VLA supplies robot-specific actions.Source: C. Execution Stability · p. 7, §VI · p. 8
• The real-world Go2 example preserves the pending goal during corrective motion after a deviation, then continues execution without restarting.The harness source remains unchanged throughout the episode.Source: C. Execution Stability · p. 7

D. Limitations and Future Work

• Gains vary with adaptation model and task demands, and visual or spatial reasoning errors may affect both code revision and behavioral assessment.Separate contexts do not rule out errors shared by the base model; the authors suggest human-guided corrections as future work.Source: D. Limitations and Future Work · pp. 7–8
• The framework edits coordination code while leaving VLA parameters unchanged; future work could use persistent memory to select validated sequences for fine-tuning or post-training.Memory would guide experience selection rather than serve directly as action supervision.Source: D. Limitations and Future Work · p. 8

VI. CONCLUSION

• The paper concludes that code-based coordination adaptation improves task success across adaptation models and VLA backbones, while real-world deployment illustrates sustained progress across task stages.AdaHVLA links testable coordination hypotheses to observed robot behavior through separated analysis, revision, and assessment.Source: §VI · p. 8