PaperlayerView exact revision on arXiv

Source-linked AI summary

RoboDuo: Attempt, Check and Atomic Recovery through Code for Vision-Language-Action Models

Jie Wang, Bo Zhang, Renzhi Lu, Jianyu Chen

arXiv:2610.34108v1cs.ROcs.CV

TL;DR

Long-horizon VLA tasks can remain unfinished when a failed step goes unchecked. RoboDuo checks a full-task attempt and uses an atomic VLA to complete remaining steps; it improves results across benchmarks, especially on long-horizon tasks.

  • Problem

    VLA models do not check whether a task is complete, so an unnoticed failed step can leave remaining steps unfinished even when earlier progress was correct.

    Source: §1 · p. 1

  • Method

    RoboDuo uses code to direct a frozen full-task VLA's attempt, check completion, and, if needed, have a frozen atomic VLA finish the remaining steps.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    Across benchmarks, RoboDuo improves both backbones, especially on long-horizon tasks; on LIBERO-Long, X-VLA rises from 87.4% to 91.6% and π0.5 from 93.6% to 96.2%.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The results suggest that a failed episode is better finished as a sequence of atomic skills than continued under the original instruction.

    Source: §5 · p. 8

    Because both VLAs stay frozen, RoboDuo cannot recover skills neither policy has, and gains are bounded by completion-check accuracy.

    Source: §5 · p. 8

Abstract

from arXiv · show

Vision-Language-Action (VLA) models map a language instruction and the current observation directly to actions, without checking whether the instruction has been completed. For long-horizon tasks, a single failed step therefore goes unnoticed and leaves the whole task unfinished. Such failed episodes, however, often end with the task partially complete. Moreover, the unfinished steps are reusable atomic skills that recur across demonstrations. A failed attempt can thus be recovered by composing with atomic skills. We propose RoboDuo, a closed-loop dual-system framework that uses code as the interface between a coding agent and two VLAs. The agent writes a program that lets a full-task VLA attempt the whole task and checks whether it is complete; on failure, an atomic VLA finishes the remaining steps. Since the agent emits no actions and only decides which VLA for execution, RoboDuo wraps an existing full-task VLA without updating its weights. To train the atomic VLA and the coding agent with the completion-checking ability, we build LIBERO-Atomic, LIBERO-Code, and LIBERO-Judge from a single set of LIBERO demonstrations. RoboDuo consistently improves X-VLA and $\pi_{0.5}$ on LIBERO and LIBERO-PRO, with the largest gains on long-horizon tasks, and recovers failed rollouts of a frozen full-task VLA on a real robot.

1. Introduction

RoboDuo addresses unnoticed failures in long-horizon VLA tasks by checking a full-task attempt and using an atomic VLA to complete unfinished steps. It improves benchmark performance while keeping the full-task VLA frozen.

1. Introduction

  • VLA policies do not check whether an observation satisfies the goal, so an unnoticed failed step can leave a long-horizon task unfinished despite earlier progress.A post-slip state may be rare in demonstrations, and the policy may continue its learned trajectory rather than re-grasping.Source: §1 · p. 1
  • Code control flow can express checks, retries, and executor switches, while code-as-policy methods start with primitives or VLA skills and largely discard full-task competence.Putting judgment inside a large action model also slows every inference step, including steps a faster policy could execute unchanged.Source: §1 · p. 1
  • A coding agent gives a frozen full-task VLA one attempt, checks completion, then has a frozen atomic VLA execute unfinished steps, with at most three attempts per step.Control does not return to the full-task VLA after recovery begins.Source: §1 · p. 1
  • LIBERO-Atomic, LIBERO-Code, and LIBERO-Judge are constructed from the same demonstrations to train the atomic VLA, coding agent, and completion check, respectively.RoboDuo is instantiated on X-VLA and π0.5 without updating either full-task VLA's weights.Source: §1 · p. 2
  • On LIBERO-Long, RoboDuo raises X-VLA success from 87.4% to 91.6% and π0.5 success from 93.6% to 96.2%.On LIBERO-PRO, π0.5 mean success rises from 32.05% to 39.33%; on a real robot, success rises from 36.7% to 53.3% across three fruit-arrangement layouts.Source: §1 · p. 2

2. Related Work

Prior work improves single VLA attempts, changes how a policy is conditioned, or uses code to control skills. RoboDuo instead lets a full-task VLA attempt the task first, then hands remaining work to an atomic VLA.

Vision-Language-Action Models

  • VLA research varies action representations and adds intermediate reasoning, but these approaches improve a single attempt within the action model [19] [22] [37] [39].Autoregressive models predict discretized actions, while continuous-control models regress or denoise future actions [4] [5] [16] [26] [40] [41].Source: §2 · p. 2

Dual Systems and Code as Policies

  • Helix and related dual systems condition one low-level policy through a command, token, or latent vector, typically without an explicit completion check [2] [6] [8] [29] [30] [38].Systems that generate actions directly with a large model incur that model's inference latency at every control step [12].Source: §2 · p. 2
  • Code-as-policy methods write programs over primitives and other skills [13] [14] [20] [31], with newer agents also calling frozen VLA policies as skills [21].Their programs serve as the plan from the first step, so no full-task policy attempts the whole task first.Source: §2 · pp. 2–3
  • RoboDuo uses code as the interface, lets the full-task VLA attempt the whole task, and invokes the atomic VLA only after the completion check fails.Both VLAs stay frozen at test time, and control does not return to the full-task VLA.Source: §2 · p. 3

Failure Correction and Recovery

  • FPC-VLA, Spotter, RoboFAC, and Recova intervene around failures but return control to the original policy for remaining steps [33] [18] [34] [24].RoboDuo instead plans remaining steps as atomic instructions and keeps control with an atomic VLA trained on short successful segments [p25].Source: §2 · p. 3

3. Method

RoboDuo wraps a full-task VLA in a code-controlled loop: it checks completion after an attempt and uses atomic skills for unfinished steps. Its method segments demonstrations into effect-complete clips and uses them to train the atomic VLA, coding agent, and completion check.

3.1. Problem Formulation

  • A fixed-instruction VLA trajectory receives new observations but no goal-satisfaction check, so the method adds an explicit test without changing the policy or its parameters.The completion test addresses a gap in the existing execution setup.Source: §3.1 · p. 4

3.2. Dual-System Execution via Code

  • The coding agent writes a program and atomic plan; after one full-task attempt, a completion check directs failure to atomic execution, with up to three attempts per step.Code supplies function calls, branches, and bounded loops while the VLAs produce the actions.Source: §3.2 · p. 4

3.3. Atomic Action Segmentation

  • LIBERO demonstrations are segmented into effect-complete clips; cleaned VLM verb windows yield 9,214 clips and 5,442 programs across 125 tasks.The clips are paired with atomic instructions and corresponding programs.Source: §3.3 · p. 5, Figure 3 · p. 5

3.4. Training the Atomic VLA, Agent, and Judge

  • The clips, programs, and shared boundaries train the atomic VLA, coding agent, and vision-language completion check, while the full-task VLA remains a released, unchanged checkpoint.The coding-agent targets attempt the full task before fallback; the judge learns from frame and sub-goal labels.Source: §3.4 · p. 5

4. Experiments

RoboDuo improves frozen VLA performance across LIBERO, LIBERO-PRO, and a real robot, with gains shaped by recoverable unfinished work and by the coding agent. The experiments assess overall performance, perturbation robustness, agent choice, and physical-robot transfer.

4. Experiments

  • The LIBERO evaluation tests two frozen backbones across four suites, with 50 trials per task and a fine-tuned coding agent and completion check.The LIBERO-PRO evaluation covers eight position- and task-perturbation conditions; its setup conditions policies on the BDDL language field matching the scored goal predicate.Source: §4.1 · p. 6, §4 · p. 6, §4.2 · p. 6

4.1. Overall Performance on LIBERO

  • Under four-suite training, RoboDuo raises LIBERO success from 87.4% to 91.6% for X-VLA and from 93.6% to 96.2% for π0.5, with no suite reduced.The largest gains are on Long; under five-suite training, residual gains shrink as the bases are stronger on Long.Source: §4.1 · p. 6, Table 2 · p. 6

4.2. Robustness on LIBERO-PRO

  • Under four-suite training, π0.5’s LIBERO-PRO eight-condition mean rises from 32.05 to 39.33, while X-VLA moves from 4.95 to 5.70.The largest π0.5 gains are Object-T (+30.6) and Object-S (+11.0); under five-suite training, π0.5 rises from 43.58 to 45.68, led by Object-T (32.2 →42.2).Source: §4.2 · p. 6, Table 3 · p. 7

4.3. Effect of the Coding Agent

  • With frozen VLAs and the same atomic policy, the coding-agent choice changes the LIBERO-PRO eight-condition mean by up to 5.8 points.Under four-suite training, the fine-tuned agent scores 39.33, GPT-6-Astra 41.50, and GPT-6-Astra with TAMP 43.00; under five-suite training, they score 45.68, 49.75, and 51.50, respectively.Source: §4.3 · p. 7, Table 4 · p. 7

4.4. Real-World Experiments

  • Across three robot layouts, pooled success rises from 11/30 to 16/30 trials, although the 10 trials per layout make the differences indicative rather than conclusive.The atomic VLA commonly takes over after a timeout, detected drop, or scene check reporting no progress; neither VLA is updated on-robot.Source: §4.4 · p. 8, Figure 4 · p. 8

5. Conclusion

RoboDuo wraps two frozen VLAs in a code-driven controller that attempts the full task, checks completion, and hands unfinished work to an atomic VLA. Its gains track recoverable work, but recovery is limited by the skills available to the VLAs and the accuracy of the completion check.

5. Conclusion

  • A coding agent gives the full-task VLA one attempt, checks completion, then hands remaining steps to an atomic VLA without returning to the full-task policy.The same program wraps X-VLA and π0.5 without changing their weights.Source: §5 · p. 8
  • Across benchmarks, gains track recoverable work; on the bimanual robot, RoboDuo improves task success and fruit count across layouts.The accompanying table reports success and mean fruit count over 10 trials per method and layout; success requires all four fruits in the basket.Source: §5 · p. 8, Table 5 · p. 8
  • Because both VLAs remain frozen, RoboDuo cannot recover skills neither policy has, and its gains depend on completion-check accuracy.Source: §5 · p. 8

Related papers

Loading 2610.34108v1…