PaperlayerView exact revision on arXiv

Source-linked AI summary

RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang, Yehang Zhang, Zexi Li, Wenqian Li, Qize Yu, Zetian Song, Leyi Wu, Jinghao Li, Mingxuan Song, Xinrun Xu, Zongyang Qiu, Yangkai Wei, Tianyi Zhang, Kaiwen Zhou, Yinchuan Li, James Cheng

arXiv:2610.11480v1cs.ROcs.AI

TL;DR

Coding agents lack inherent awareness of the divergent outcomes of different skill executors, making responsibility assignment a central challenge. RoboAware defines semantic decision stages and learns family-conditioned values from counterfactual outcomes to coordinate modular and end-to-end skills. Across 100 tasks on three benchmarks, it achieves a 77.0% overall success rate.

  • Problem

    Coding agents can flexibly compose program modules, but lack inherent awareness of the divergent consequences of choosing different skill executors.

    Source: §1 · p. 2

  • Method

    RoboAware uses P5 stages to choose between modular and end-to-end policy families, with same-state counterfactual branches and MCTS-guided Q-learning to learn family-conditioned values.

    Source: §1 · p. 2, §2.2.2 · p. 4

  • Results

    77.0% overall success rate across 100 tasks on three benchmarks, reported as state of the art.

    Source: §1 · p. 3

  • Takeaways & Limitations

    At deployment, observable context guides family selection at each P5 boundary without simulator restoration, privileged inputs, or online search.

    Source: §5 · p. 9

    RoboAware currently integrates only modular and end-to-end policies, and responsibility changes only at P5 boundaries, leaving finer-grained interruption under ambiguous observations open.

    Source: §5 · p. 9

Abstract

from arXiv · show

Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the $P^5$ schema and formulate a hierarchical MDP based on it. $P^5$ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.

1 Introduction

RoboAware addresses coding agents’ difficulty choosing between modular and end-to-end execution by learning responsibility from counterfactual outcomes. It defines P5 decision stages and combines state-locked branching with value learning to coordinate the two policy families.

1 Introduction

  • Coding agents can flexibly compose modules, but lack inherent awareness of the divergent outcomes of different skill executors.The paper frames outcome-aware responsibility over embodied skills as the central research question.Source: §1 · p. 2
  • The P5 Skill Schema organizes skills into semantic stages and grounds a hierarchical MDP that selects a policy family at each stage.A curated API library combines typed modular interfaces with end-to-end policies for contact-rich local execution.Source: §1 · p. 2
  • State-Locked Counterfactual Branching executes each admissible family from the same physical state, while Execution-Aware Learning uses MCTS and Q-learning to learn family-conditioned values.At deployment, the coordinator selects a family from observable context and the coding agent generates the next local code block.Source: §1 · p. 2
  • 77.0% overall success rate across 100 tasks on three benchmarks gives RoboAware a state-of-the-art result.Ablations also confirm the effectiveness of the designs.Source: §1 · p. 3

2 Methodology

RoboAware formulates policy-family selection as responsibility learning at semantic skill boundaries, training a coordinator from counterfactual outcomes while keeping the coding agent and execution backends fixed. At deployment, it selects a family from observable context and lets the frozen coding agent generate the next local code block.

2.1 Problem Formulation

  • Only the coordinator’s value model is trained in a finite-horizon MDP where each decision selects an admissible policy family and the coding agent generates its code block.The coding agent, API interfaces, backend bindings, and low-level backend parameters remain fixed.Source: §2.1 · p. 3

2.2 RoboAware Framework

  • P5 decomposes each skill into perceive, propose, pre-manipulate, perform, and post-manipulate stages, with responsibility reconsidered at each stage.The schema represents both primitive and compositional skills.Source: §2.2.1 · p. 3

2.2.1 Skill Preparation

  • The API library pairs composable modular components with end-to-end local controllers, positioning VLA and WAM policies at grasp–lift granularity rather than for complete tasks.Each stage admits both families except propose; Perform_E2E spans four stages and remains atomic until a postcondition, failure, or step-budget limit.Source: §2.2.1 · p. 4

2.2.2 Execution-Aware Learning

  • State-Locked Counterfactual Branching restores the same simulator state to evaluate admissible policy families, while search and suffix rollouts collect outcomes for Q-learning.The search uses UCB selection and includes failed and nonpreferred evaluated branches in its training data; Bellman updates train family-conditioned values from the fixed transition set.Source: §2.2.2 · pp. 4–5

2.2.3 Test-Time Skill Coordination

  • At test time, the coordinator selects the highest-valued admissible family from observable context, and the frozen coding agent generates and executes a local block before the next semantic boundary.Family selection repeats at subsequent boundaries using updated observable feedback, with Qθ fixed.Source: §2.2.3 · pp. 5–6

3 Experiments

RoboAware is evaluated across three benchmarks against standalone end-to-end, code-as-policy, and VLA-harness baselines, then analyzed through family, training, coordination, and budget ablations. The reported analyses favor state-dependent family coordination, paired same-state outcomes, per-stage decisions, and UCB-guided branching.

3.1 Experimental Protocol

  • The protocol evaluates 100 tasks across LIBERO-Pro, RoboSuite, and RoboTwin 2.0, using seed 0 for SCB branches and Qθ training and seeds 1–10 for 1,000 test episodes.Baselines include standalone end-to-end policies, code-as-policy methods, and VLA harnesses; global cross-episode memory is disabled for memory-based baselines.Source: §3.1 · p. 6, Table 3 · p. 6

3.2 Benchmark Performance

  • Under whole-task distribution shifts, RoboAware generates modular alternatives while retaining end-to-end policies for local skills suited to their capabilities.The evaluation covers 100 tasks across LIBERO-Pro, RoboSuite, and RoboTwin 2.0; the supplied passages do not give aggregate success-rate values.Source: §3.2 · p. 6, §3.1 · p. 6

3.3 Experimental Analysis

  • In group A, single-family variants show that neither modular-only nor end-to-end-only provides broad high-success task coverage; RoboAware switches families across P5 boundaries.On the illustrated LIBERO-Pro task, the variants fail at different stages, and ρθ assigns lower preference to the families they use at those states.Source: §3.3 · p. 7, Table 4 · p. 7, Figure 4 · p. 7
  • RoboAware-UO’s modest success-rate decrease relative to RoboAware supports paired same-state counterfactual executions, while its advantage over RoboAware-TF shows coordinator training helps without them.Executed-only outcomes inherit the agent’s experience distribution and selection bias.Source: §3.3 · pp. 7–8, Table 5 · p. 7
  • RoboAware-FH and RoboAware-ES have lower success rates than per-stage RoboAware, favoring family reselection at each P5 stage over fixed handoff or episode-level selection.The strategies differ in when family responsibility may change.Source: Table 6 · p. 7, §3.3 · p. 8
  • At matched branching budgets in RoboSuite, RoboAware exceeds uniform-allocation RoboAware-UA; diminishing gains with K motivate the adopted elbow budget K=20.Figure 5 compares success rates across K ∈ {5, 10, 20, 50}.Source: §3.3 · p. 8, Figure 5 · p. 8
  • Across four RoboTwin 2.0 tasks, RoboAware shows a small average success-rate gap across three coding-agent backends and four embodiments.The authors attribute the small backend gap to P5 limiting each generation to one code block.Source: §3.3 · p. 8, Figure 6 · p. 8

4 Related Work

Prior embodied coding work develops compositional programs and adaptive execution, while orchestration work combines end-to-end policies with planning or harnesses. RoboAware distinguishes itself by obtaining paired outcomes for modular and end-to-end families from the same restored state.

4.1 Embodied Coding Agents

  • Embodied coding methods progress from language planners over closed skill APIs to executable controllers and adaptive code repair, but their evidence remains conditional on generating a modular block.SCB instead generates and executes modular and end-to-end blocks from the same restored state, and EAL learns family-conditioned values from paired outcomes.Source: §4.1 · p. 9

4.2 End-to-End Policy Orchestration and Execution Harnesses

  • End-to-end orchestration spans generalist VLAs, hierarchical harnesses, and hybrid policy routers, which estimate suitability from traces, probes, or other retrospective signals.RoboAware’s distinction is simulator-restored, same-state execution of modular and end-to-end families within a common skill-stage interface.Source: §4.2 · p. 9

5 Conclusion and Limitations

RoboAware learns state-dependent responsibility between modular composition and end-to-end control using P5 boundaries, paired counterfactual outcomes, and family-conditioned value estimates. Its scope is currently limited to those two policy families and stage-boundary handoffs.

5 Conclusion and Limitations

  • RoboAware keeps the coding agent and API library fixed while using P5 boundaries and paired restored-state outcomes to learn responsibility between modular and end-to-end control.EAL combines UCB-guided MCTS with Q-learning; at deployment, family selection uses observable context without simulator restoration, privileged inputs, or online search.Source: §5 · p. 9
  • RoboAware currently integrates only modular composition and end-to-end policies; adding other policy types may require new counterfactual outcome data and further coordinator training.Responsibility changes only at P5 boundaries, so a single atomic end-to-end invocation can span several stages and keep transient failures within its controller until a postcondition, failure signal, or timeout returns control.Source: §5 · p. 9

A Methodological Details

P5 stage contracts govern admissible families, APIs, postconditions, and failure transitions. Family selection occurs at eligible stage boundaries and remains fixed during an atomic API call.

A.1 P5 Stage Contracts

  • Each P5 stage contract specifies an entry predicate, typed inputs, admissible families and APIs, a success postcondition, and failure or timeout feedback.The coordinator selects a family when it has control and the stage entry predicate holds; an atomic API call keeps that selection until return, failure, or timeout.Source: §A.1 · p. 15, Table 1 · p. 15
  • At propose, only Mod is admissible because Propose_eef_pose exposes the end-effector target, while E2E backends determine grasp and approach targets internally.An E2E call beginning at perceive or a later admissible stage can complete a skill without visiting propose.Source: §A.1 · p. 15
  • After a multi-stage E2E call returns, execution advances to the earliest enterable stage whose postcondition remains unmet; completed stages are skipped.The next stage is determined from observable typed returns, postcondition bits, failure codes, and timeout flags; an empty admissible-family set during continued execution terminates the episode with failure.Source: §A.1 · p. 15

A.2 RoboAware: Training and Deployment

This section describes RoboAware’s training and deployment setup, including how its state-conditioned coordinator learns from counterfactual family outcomes and selects admissible families using observable context and execution feedback. It also covers the coding agent’s grasp-proposal, motion-planning, and gripper-control components, including Panda grasp-screen results and the handling of motion modes and gripper actions.

A.2 RoboAware: Training and Deployment

  • MCTS evaluates family-conditioned branches from restored roots, and Q-learning fits the coordinator on collected transitions; deployment uses the trained coordinator without search or oracle rewards.Each root’s counterfactual branches share the same physical state and observable-history prefix, and all expanded edges—including failed and nonpreferred branches—enter the training set. The coding agent and API library remain frozen; deployment relies on observable information and feedback.Source: §A.2 · pp. 16–17

A.3 Responsibility Preference

  • The normalized preference preserves Q-value ordering over admissible families, so the coordinator can select by either preference or Q-value.With both E2E and Mod admissible, positive value gaps favor E2E, negative gaps favor Mod, and zero gives equal preference; a singleton family receives preference one, while an empty set terminates with failure.Source: §A.3 · p. 19

A.4 Q-Value Model

  • Qθ uses a Qwen3.5-0.8B multimodal backbone and scalar MLP head, while the coding agent, typed APIs, and low-level backend checkpoints remain frozen.Inputs include images, proprioception serialized as text, execution history, current skill and P5 stage, and queried family; history retains recent stage transitions and summarizes older ones using observable information only.Source: §A.4 · pp. 19–20

A.5 Reward Function

  • Training-only oracle rewards score completed skills and terminal task success, but the coordinator does not observe oracle fields at deployment.Modular perception is scored by mask IoU, whereas E2E perception is scored by whether execution acted on the oracle-referenced object; manipulation uses contact/grasp predicates or pose thresholds.Source: §A.5 · pp. 20–21

B.1 Typed API Contracts

  • Typed API contracts specify inputs, outputs, coordinate frames, postconditions, and failures, separating the coding agent’s semantic target from embodiment execution and measured completion.The coordinator observes typed API returns rather than privileged simulator state, and API contracts and backend bindings are frozen before training.Source: §B.1 · p. 21

B.2 Object Perception

  • Perceive_object_pose grounds language references in agent-view RGB-D data, returns a robot-frame target with supporting outputs, and reports failure when perception is unreliable.InstructSAM [17] accepts the coding agent’s detailed free-form prompt; the resulting mask is lifted through a shared geometric pipeline to produce the typed outputs used downstream.Source: §B.2 · p. 21

B.3 Pose Proposal

  • NeuGraspNet+ adds deterministic geometric seeds when native sampling yields too few candidates, then uses cuRoboV2 feasibility and a confidence gate to select a grasp.The released NeuGraspNet checkpoint still predicts seed quality and gripper width; the gate preserves informative network rankings, uses geometry to break near-ties, and weights geometry more when feasible scores are low.Source: §B.3 · p. 22
  • 18/18 Success@1, Success@3, and Success@5 on the 18-task Panda screen; NeuGraspNet+ is frozen as the grasp backend.The selection rules prioritize first-choice success over speed and were developed using the benchmark, so performance on new object distributions requires further validation.Source: §B.3 · p. 22, Table 6 · p. 22

B.4 Motion Planning

  • goto_pose offers auto, ik_checked, and ik_unchecked modes, selected before execution to avoid repeated collision-avoidance attempts for intended contact without silently bypassing requested checks.Auto uses cuRoboV2, then collision-checked multi-seed IK, then bounded recovery; ik_checked skips global planning but checks the scene, while ik_unchecked omits scene-collision checks but retains control and arrival checks.Source: §B.4 · p. 23

B.5 Gripper Control

  • The gripper interface provides binary open/close actions and reports success only after the observed opening or joint signal stays stable for a fixed dwell window.Platform-specific adapters map commands to native coordinates across Franka, RoboSuite, and RoboTwin 2.0; the deterministic, embodiment-local operation is not screened and Qθ does not score the adapter.Source: §B.5 · p. 23

B.6 End-to-End Policy Execution

End-to-end routes support task, grasp–lift, and bounded-prompt instruction modes, with explicit execution stops and recorded postconditions. Route screening and benchmark evaluation use specified checkpoint configurations, and efficiency statistics are interpreted alongside success counts.

B.6 End-to-End Policy Execution

  • Perform_E2E supports task, grasp–lift, and bounded-prompt modes, with invalid argument combinations rejected.Task mode forwards the environment instruction; grasp–lift uses an object-specific local instruction; bounded-prompt forwards an explicit prompt and requires a step budget.Source: §B.6 · p. 23
  • Calls stop on completion, termination, a supported model stop signal, or budget exhaustion, and record execution details and measured postconditions.Grasp–lift mode also stops when its local completion predicate is met; full-task and grasp–lift success use their respective predicates.Source: §B.6 · p. 24
  • Route screening covers 14 LIBERO-Pro, four RoboSuite, and ten RoboTwin 2.0 tasks, with retained routes frozen before coordinator training.RoboTwin full-task evaluation may reject expert-invalid initial states, so realized initial states can differ across methods.Source: §B.6 · p. 24
  • Reported results characterize the specified checkpoint–suite configurations; RynnVLA-002 loading reinitializes part of its VQ encoder, and JointWAM lacks an available runnable checkpoint.FastWAM and FasterWAM use suite-matched checkpoints, while Harness-VLA uses the same LIBERO-trained checkpoint across suites.Source: §B.6 · p. 24
  • Efficiency tables report rollout and timing statistics, but lower all-trial steps or runtime need not mean greater efficiency because failed trials can end early.Interpret these statistics alongside Table 8 success counts; successful-only timing is available only for methods with successful trials.Source: Table 8 · p. 25, Table 9 · p. 26, Table 10 · p. 27

C.1 Benchmark Tasks

The evaluation spans three tabletop-manipulation benchmarks and tests generalization to new scene seeds within the same task pool. A separate RoboTwin study varies coding-agent backend and robot embodiment while holding task instructions and P5 contracts fixed.

C.1 Benchmark Tasks

  • The 100-task pool comprises 80 LIBERO-Pro, four RoboSuite, and 16 RoboTwin 2.0 tasks, covering perturbed single-arm, clean-scene pick–place, and long-horizon bimanual settings.Coordinator training uses scene seed 0, while evaluation uses seeds 1–10; mobile-manipulation suites are outside the stated P5 tabletop scope.Source: §C.1 · p. 28
  • The cross-agent and cross-embodiment study uses a fixed 3 × 4 backend–embodiment grid on four RoboTwin task instances.Claude Code and Aloha–AgileX are defaults; comparisons swap in Cursor or Codex and Franka–Panda, ARX–X5, or UR5–WSG.Source: §C.1 · pp. 28–29
  • The backend sweep keeps the API library and trained coordinator fixed, while each embodiment uses its native adapter and compatible controller under the shared API contract.The supplementary videos provide 16 task–embodiment clips; Figure 6 reports aggregate success rates.Source: §C.1 · p. 29

C.2 Baseline Implementations

Baselines are evaluated under a shared single-episode protocol, with cross-episode memory disabled. The comparison includes standalone end-to-end policies, code-as-policy methods, and VLA harnesses, alongside controlled RoboAware routing ablations.

C.2 Baseline Implementations

  • All methods share a single-episode evaluation protocol with seeds 1–10, one trial per seed, terminal success rate, and no global cross-episode memory.RoboTwin “/” entries indicate that single-arm VLA checkpoints are incompatible with bimanual controllers.Source: §C.2 · p. 29
  • π0, π0.5, FastWAM, and FasterWAM run as frozen released-checkpoint full-episode controllers, while retained π0.5 and FastWAM routes are local Perform_E2E calls in RoboAware.The standalone end-to-end policies are not re-fine-tuned on the coordinator training pool.Source: §C.2 · p. 29
  • CaP and CaP-X use their public code-generation loops, while RATS and ASPIRE retain their published interfaces; none receives the learned Qθ coordinator.The code-as-policy baselines are evaluated without optional cross-episode repair memory.Source: §C.2 · p. 29
  • Pigey and VLS use public harness wrappers, Harness-VLA uses a released π0.5-RLinf-SFT checkpoint, and RoboAware routing ablations vary coordinator training or family allocation.Harness early-stop and reward-steering settings follow the authors’ defaults under the shared protocol.Source: §C.2 · p. 29

C.3 Success and Runtime Metrics

The section defines how success, runtime, and task averages are measured, then specifies the coordinator’s settings, coding-agent backends, execution protocol, and training compute.

C.3 Success and Runtime Metrics

  • Success rate counts terminal binary evaluator success per task–seed episode, including aborted episodes in the denominator; task macro and episode-weighted averages coincide when tasks have equal seed counts.An unweighted average of the three suite-level rates is different because it gives each suite equal weight.Source: §C.3 · pp. 29–30
  • Episode time runs from the post-reset observation used to build the task plan until termination, excluding simulator reset and offline model loading.Times are wall-clock seconds on the evaluation host unless a table names another clock; averages over all trials include failures.Source: §C.3 · p. 30

C.4 Hyperparameters and Experimental Settings

  • The main experiments use MCTS budget K=20; RoboSuite ablations compare K ∈ {5, 10, 20, 50} and show diminishing success-rate gains beyond K=20.K counts completed suffix simulations per search root, not environment steps or semantic decisions.Source: §C.4 · p. 30

C.5 Coding-Agent Backends

  • Sensitivity tests swap coding-agent backends while keeping the API library, P5 contracts, and Qθ fixed.Claude Code is the default; Cursor and Claude Code use Claude Opus 4.8, while Codex uses GPT-5.5 in the reported runs.Source: §C.5 · p. 30

C.6 Code-Execution Protocol

  • Generated blocks run in a restricted Python process, with syntax, name, import, or schema errors triggering up to two repair attempts.Between episodes, the coding-client session is reset; within an episode, feedback is limited to the observable channel, and deployment family selection comes from Qθ.Source: §C.6 · pp. 30–31

C.7 Computation and Software

  • Fitting Qθ took approximately six hours on four 80 GB NVIDIA A100 GPUs, while trajectory and counterfactual data collection took approximately one additional day.Data collection accounts for most of the wall-clock training time.Source: §C.7 · p. 31

D.1 Ablation of Harness-VLA

The Harness-VLA ablation reports multi-seed and screening comparisons, followed by a LIBERO-Pro rollout illustration and an accounting of observed failure categories and their scope.

D.1 Ablation of Harness-VLA

  • 62.5% Full success on 24 tasks; removing the memory package and SAM3 produced 37.5%, a 25.0 pp drop (95% task-bootstrap CI [4.2, 45.8]).The screening used seed 42; the larger joint-removal drop does not by itself establish a positive interaction.Source: §D.1 · p. 32, Table 14 · p. 32
  • Bare SFT achieved 41.7%, 20.8 pp below Full, but this contrast also removes the planner and scripted hybrid execution.The study screens six configurations on 24 tasks, then evaluates Full, w/o Memory, and w/o Memory & SAM3 on 80 tasks with three seeds.Source: §D.1 · pp. 31–32
  • 56.6% valid-completion success for Full, versus 16.9% without Memory and 20.9% without the memory package and SAM3, in the multi-seed study.On jointly completed episodes, removing the memory package yielded −38.9 pp (n=113, 95% CI [−50.0, −27.4]); removing both yielded −35.8 pp (n=109, 95% CI [−48.2, −23.4]).Source: §D.1 · p. 32, Table 15 · p. 33

D.2 Visualizations

  • On the spatial-language LIBERO-Pro task, RoboAware succeeds by switching at P5 boundaries, while EO misgrasps and MO selects the wrong object.The illustrated task asks the agent to pick the akita black bowl between the plate and ramekin and place it on the plate.Source: §D.2 · p. 33, Figure 1 · p. 33

D.3 Failure Modes

  • 29.8% of failed rollouts were grasping or object-holding instability and 16.7% were constrained contact, insertion, or mechanism-actuation failures, together 46.5%.Grounding, transport and placement, and multi-object sequencing accounted for 5.3%, 6.1%, and 4.4%, respectively.Source: §D.3 · p. 33
  • 37.7% of failures remain in Others because saved evaluation records lack traces to distinguish specific causes, limiting characterization of the full failure distribution.The figure caption also attributes 37.7% to records insufficient to identify a specific cause.Source: §D.3 · p. 33, Figure 2 · p. 34
Loading 2610.11480v1…