PaperlayerView exact revision on arXiv

Source-linked AI summary

RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer

Chenxi Li, Zhangrui Zhao, Rui Li, Yuan Gao, Kehui Liu, Jiarui Li, Dong Wang, Tong Si, Minting Pan, Wanli Ouyang, Dongzhan Zhou

arXiv:2610.02717v1cs.RO

TL;DR

Embodied sim-to-real transfer often requires target-domain data and policy fine-tuning, while policy-bound task knowledge is difficult to isolate and revise. RoboBridge transfers explicit task skills through aligned interfaces, evolves them using validated execution feedback, and guides a pretrained action policy at inference time. Experiments report useful initialization for physical tasks, further improvement from real-world feedback, and reduced adaptation effort compared with evolution from scratch.

  • Problem

    End-to-end policies commonly require target-domain demonstrations and fine-tuning for new robotic settings, while their parameter-bound task knowledge is difficult to isolate and selectively revise.

    Source: §1 · p. 1

  • Method

    RoboBridge transfers executable task skills through aligned semantics and interfaces, validates feedback-driven skill revisions, and uses inference-time guidance with a pretrained action policy whose parameters remain fixed.

    Source: §1 · pp. 2–3, §3.1 · p. 5, §3.3 · p. 6

  • Results

    Simulation-evolved skills provide useful initialization for three physical tasks and improve further through real-world feedback, reducing adaptation effort compared with evolution from scratch.

    Source: §4.3 · pp. 8–9, §5 · p. 10

  • Takeaways & Limitations

    When task semantics and interaction interfaces are aligned, simulation skills can provide reusable initial knowledge for physical tasks and continue adapting from execution feedback.

    Source: §4.3 · p. 9

Abstract

from arXiv · show

A key challenge in bringing embodied intelligence into the real world is transferring capabilities from simulation to reality and enabling agents to continually adapt after deployment. End-to-end vision-language-action policies provide strong manipulation capabilities, but their transfer to physical environments typically relies on calibrating simulated visual and dynamical conditions, collecting additional target-domain demonstrations, and optimizing the policy through further training. Tool-using embodied agents offer flexible task orchestration, yet existing systems primarily emphasize task execution and experience reuse within a given environment, with limited support for transferring procedural knowledge and continuously adapting it across simulation and reality. We propose RoboBridge, a framework that treats sim-to-real transfer as the continued adaptation of executable task skills. The agent represents task knowledge as procedures connecting task intent, observations, tool operations, and outcome verification. Interaction feedback is used to generate candidate skill revisions, which are evaluated before being persisted or rejected. A pretrained vision-language-action policy is exposed as a reusable action tool and enhanced with inference-time guidance, enabling fine-grained execution without retraining the underlying policy. RoboBridge grounds transferable skills in task semantics and interaction interfaces shared across simulation and reality. This representation preserves reusable task structure while allowing environment-dependent operations to be selectively revised through real-world execution feedback. We evaluate the framework on LIBERO-PRO and corresponding physical tasks, studying both skill evolution and post-transfer adaptation. Our framework provides a route from one-shot policy deployment to continual procedural learning across environments.

1 INTRODUCTION

RoboBridge addresses the difficulty of transferring embodied capabilities when task knowledge is bound to policy parameters and adaptation requires target-domain data and retraining. It transfers explicit, revisable task procedures alongside pretrained manipulation capabilities, using validated feedback-driven updates and inference-time guidance.

1 INTRODUCTION

  • End-to-end policies encode task knowledge in model parameters, making reusable procedures difficult to isolate and revise as execution conditions change.Adapting pretrained VLA models commonly involves target-domain demonstrations and policy fine-tuning.Source: §1 · p. 1
  • RoboBridge stores procedures connecting task intent, observations, tool operations, and outcome verification, while a pretrained VLA supplies reusable manipulation segments.The agent organizes operations and checks their outcomes; task procedures can be revised through interaction.Source: §1 · p. 2
  • Transferred skills rely on aligned task semantics and compatible observation and tool interfaces, then continue adapting from real-world execution feedback.The action tools are instantiated for each environment.Source: §1 · p. 2, Figure 1 · p. 2
  • Candidate skill insertions, revisions, or deletions are validated before updates are persisted, rejected, or rolled back.The agent also decides when to invoke the pretrained policy and inference-time guidance, configuring subtask rewards while keeping policy parameters fixed.Source: §1 · p. 3

2 RELATED WORK

Prior work develops both end-to-end embodied policies and tool-using agents, but RoboBridge focuses on transferring and revising procedures across environments. Its framework combines explicit skills, execution feedback, and action tools.

2.1 END-TO-END EMBODIED MODELS

  • End-to-end models map visual observations and language instructions to actions, while sim-to-real approaches adapt observations, dynamics, or data but leave adaptation coupled to a policy.The related work includes observation diversification [Tobin et al., 2017], dynamics perturbation [Peng et al., 2018], simulator-distribution adaptation [Mehta et al., 2020], and scaled simulated experience and adaptation data.Source: §2.1 · p. 3

2.2 EMBODIED AGENTS

  • Embodied-agent work uses procedures, memory, and feedback, while RoboBridge treats each transferred skill as a revisable task hypothesis shaped by target-environment feedback.Figure 2 depicts an agent loop, validation-gated skill evolution, and a self-guided action tool.Source: §2.2 · p. 3, Figure 2 · p. 4

3 METHOD

RoboBridge combines skill-guided execution, validation-gated skill evolution, and inference-time action guidance. Skills transfer through aligned semantics and interfaces, then adapt from physical feedback without updating the pretrained policy.

3 METHOD

  • The agent uses the active skill and observations to select tools, while execution checks and logs inform subsequent decisions and candidate skill updates.With aligned task semantics and compatible observation and tool interfaces, simulator-evolved skills can initialize real-world execution and continue adapting through physical feedback.Source: §3 · p. 4

3.1 PROBLEM FORMULATION AND AGENTIC EXECUTION LOOP

  • A skill connects observations to tool operations and verification conditions, and validated updates transfer it from simulation to reality through an interface mapping.The mapping aligns task semantics, observation formats, and tool interfaces; failed validation leaves the current skill unchanged.Source: §3.1 · pp. 4–5
  • Each rollout gives the planner observations, task instructions, prior feedback, and the active skill; one tool operation returns refreshed observations and feedback.Rollouts end when the task condition is satisfied, the agent terminates, or the episode budget is exhausted, and their trajectories inform later skill updates.Source: §3.1 · p. 5

3.2 EMBODIED FEEDBACK-DRIVEN SKILL EVOLUTION

  • Skills are task-level executable hypotheses organized around relevant states and affordances, not particular object identities.Rollout evidence can motivate local additions, replacements, or removals; candidates are retained only when valid and improved task execution is demonstrated.Source: §3.2 · p. 5

3.3 FROM A FROZEN POLICY TO A SELF-GUIDED ACTION TOOL

  • The agent can guide a frozen action model during denoising with a reward derived from the subtask and execution context, steering candidate actions while model parameters stay fixed.This adjusts execution within an invocation, whereas skill evolution revises operation selection, ordering, and verification across episodes.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments first evolve and evaluate skills in LIBERO-PRO, then initialize corresponding real-robot tasks with simulation-derived skills and continue updates from physical feedback. Results cover benchmark performance, transfer, post-transfer refinement, and adaptation efficiency.

4 EXPERIMENTS

  • The experiments use two stages: repeated skill evolution and evaluation in LIBERO-PRO, followed by initializing corresponding real-robot tasks with simulation-evolved skills and adapting them from target-environment feedback.Accepted candidate skills are persisted for subsequent deployment.Source: §4 · p. 6

4.1 EXPERIMENTAL SETUP

  • Across settings, the agent uses a unified observation and tool interface; it performs two simulation skill-evolution rounds with five training seeds each, and one real-robot evolution round after each episode.Different pretrained policy checkpoints are used in simulation and on the real robot while preserving the same action interface.Source: §4.1 · pp. 6–7

4.2 SIMULATION EVALUATION

  • 74.4% overall success with Codex exceeds Harness VLA’s 72.1% by 2.3 percentage points on LIBERO-PRO; the gain is 5 points on GOAL-T.Directly using πRLinf yielded 50.0% overall success under the same setup.Source: §4.2 · p. 7
  • Removing skill guidance, skill revision and deletion, validation, or inference-time guidance lowered SPATIAL-T and OBJECT-T success relative to the full framework, showing complementary contributions.The full framework scored 83% and 94%; without skill guidance it scored 21% and 16%, and without inference-time guidance, 75% and 82%. Append-only updates scored 44% and 57%; removing validation scored 69% and 63%.Source: §4.2 · pp. 7–8, Figure 3 · p. 8

4.3 REAL-ROBOT EVALUATION

  • After 10 episodes of real-robot evolution, adapted skills reached 15/15, 14/15, and 15/15 successes across three tasks, versus direct-transfer results of 9/15, 7/15, and 7/15.The three real-robot tasks differ in objects and manipulation modes from their corresponding simulation tasks.Source: §4.3 · p. 8, Figure 5 · p. 9
  • On cake placement and covering, simulation-initialized evolution reached 15/15 in 12 versus 16 iterations from scratch, using 13.08M versus 52.39M cumulative tokens.Each iteration comprises one physical execution episode followed by one skill update.Source: §4.3 · pp. 8–9, Figure 5 · p. 9

5 CONCLUSION AND LIMITATIONS

RoboBridge combines reusable, revisable procedural skills with inference-time action guidance, and experiments report transfer and continued real-world improvement. The authors identify the coding-agent configuration used in evaluation and broader model and harness testing as a future direction.

5 CONCLUSION AND LIMITATIONS

  • RoboBridge transfers procedural skills from simulation and adapts them through execution feedback; evaluation uses Codex with GPT-5.5, while broader foundation models and agent harnesses remain future work.The authors report that Codex assisted writing and coding but was not used to formulate the core framework or hypotheses.Source: §5 · p. 10, AI USE STATEMENT · p. 10

AI USE STATEMENT

  • Codex assisted with translation, drafting, code, literature searches, figures, and qualitative analysis; authors checked AI-assisted outputs against experimental evidence and take responsibility for the final work.The authors state that generative AI did not formulate the hypotheses, design experiments, generate synthetic datasets, or formulate mathematical claims or proofs.Source: AI USE STATEMENT · p. 10

A PLANNER DESIGN WITH DECOUPLED OBSERVATION AND EXECUTION

RoboBridge separates scene observation construction from task-dependent planning and action selection. Its adapter refreshes observations at action-tool boundaries so the planner can use updated scene information and execution feedback.

A PLANNER DESIGN WITH DECOUPLED OBSERVATION AND EXECUTION

  • The adapter supplies an initial observation after reset and refreshes perception after each action-tool invocation, keeping sensing separate from planner decisions.The planner selects tools and arguments; the adapter handles segmentation, geometric reconstruction, and observation packaging.Source: A PLANNER DESIGN WITH DECOUPLED… · p. 14
  • Scene inventories identify entities, while the adapter associates segmentation masks with metric geometry and records reference-point provenance and uncertainty.Target assignments, grasp offsets, and end-effector goals remain planner decisions.Source: A PLANNER DESIGN WITH DECOUPLED… · p. 14
  • Simulation snapshots combine image metadata, measured scene geometry, robot proprioception, coordinate conventions, and validity information, with task instructions and feedback supplied separately.Positions use the simulation world frame in metres; optional estimates are included when available.Source: Table 2 · p. 14, A PLANNER DESIGN WITH DECOUPLED… · p. 14
  • Each action attempt invalidates the prior frame, and the dispatcher serializes physical execution with publication of the resulting observation before the next decision.Refresh is attempted after simulation errors as well, with unavailable observations reported explicitly; synchronization occurs at tool boundaries even when a call contains multiple low-level steps.Source: A PLANNER DESIGN WITH DECOUPLED… · pp. 14–15

B ACTION TOOLS AND AGENT-GUIDED EXECUTION

RoboBridge exposes a shared action library for planner-selected operations, combining explicit geometric control with learned manipulation. For policy-based actions, the agent can configure inference-time guidance and verifies progress through refreshed observations and feedback.

B ACTION TOOLS AND AGENT-GUIDED EXECUTION

  • The action library links planner subtasks to Cartesian motion, orientation adjustment, gripper control, and policy-based manipulation.The planner expresses intent through positions, poses, gripper commands, or language instructions, while the adapter translates requests into executable actions.Source: B ACTION TOOLS AND AGENT-GUIDED… · p. 15, Table 3 · p. 15
  • Geometric tools handle explicit spatial objectives, while policy tools supply learned behavior for grasping and other contact interactions.Procedures can combine geometric positioning with policy-based manipulation, selected according to the current subtask and observed state.Source: B ACTION TOOLS AND AGENT-GUIDED… · p. 15
  • The agent may configure an inference-time reward from the subtask objective, execution constraints, and observed feedback, steering action generation while policy parameters remain fixed.The policy produces short action sequences from current sensor observations; the agent evaluates progress at tool boundaries.Source: B ACTION TOOLS AND AGENT-GUIDED… · p. 15, Table 3 · p. 15
  • Tool invocations return execution feedback and refresh observations so the planner can continue, adjust, or stop based on intermediate outcomes.Tool completion and task success are assessed separately; physical evaluation records an operator-provided final label.Source: B ACTION TOOLS AND AGENT-GUIDED… · p. 15
  • Physical execution checks workspace and motion constraints, requires verified arrival before associated gripper operations, and requires successful positioning plus refreshed observation before policy invocation.Execution errors halt the physical episode.Source: B ACTION TOOLS AND AGENT-GUIDED… · pp. 15–16

C LIBERO-PRO BENCHMARK AND EVALUATION PROTOCOL

The LIBERO-PRO evaluation covers four task families under instruction-redirection and position-swap perturbations. Skill revisions are validated separately from disjoint test seeds, and test success is determined by the benchmark task-completion predicate.

C LIBERO-PRO BENCHMARK AND EVALUATION PROTOCOL

  • The evaluation spans SPATIAL, OBJECT, GOAL, and LIBERO-10 task families under task-redirection and position-swap perturbations.Redirection changes the valid target or goal while swaps rearrange object starting locations with the instruction unchanged.Source: C LIBERO-PRO BENCHMARK AND EVALUATION… · p. 16
  • Training seeds 11–20 generate candidate skill revisions, validation seeds 21–25 gate retention, and disjoint seeds 1–10 determine reported test success.The selected skill remains fixed throughout testing, and test initial states are excluded from update evidence and validation decisions.Source: C LIBERO-PRO BENCHMARK AND EVALUATION… · p. 16
  • During test rollouts, the planner can respond to refreshed observations and execution feedback, but the selected skill does not evolve further.Rollouts end at task success, execution stop, or exhaustion of the allotted budget.Source: C LIBERO-PRO BENCHMARK AND EVALUATION… · p. 16
  • Each of the 8 family-condition groups contains 10 tasks and 10 test seeds, yielding 100 test rollouts per group.Training and validation episodes are excluded from test counts.Source: C LIBERO-PRO BENCHMARK AND EVALUATION… · p. 16
  • A test rollout succeeds only when LIBERO-PRO’s task-completion predicate is satisfied within the execution budget, regardless of the tool sequence.A primitive returning control or an agent declaring completion does not itself establish success.Source: C LIBERO-PRO BENCHMARK AND EVALUATION… · p. 16

D VLA MODEL INSTANTIATIONS

RoboBridge uses frozen VLA policies as contact-rich action primitives, with separate checkpoints for simulation and physical deployment. The agent can steer action segments through inference-time reward guidance without updating policy parameters.

D VLA MODEL INSTANTIATIONS

  • Simulation uses the RLinf LIBERO checkpoint πRLinf, while physical experiments use the OpenPI DROID checkpoint πDROID.The deployments use different checkpoints because the environments differ in robot embodiments, sensor layouts, state representations, and control interfaces.Source: D VLA MODEL INSTANTIATIONS · p. 17
  • Both deployed policies are frozen, and inference-time reward specifications steer generated action segments without updating VLA parameters.The agent may derive guidance from the current subtask, execution constraints, and observed feedback.Source: D VLA MODEL INSTANTIATIONS · p. 17

D.1 πRLinf: RLINF LIBERO CHECKPOINT

πRLinf is RoboBridge’s LIBERO action policy, supplying short manipulation segments under planner instructions and optional reward guidance. The checkpoint remains frozen, while RoboBridge separates its policy-only benchmark result from the contributions of planning, skill evolution, and guidance.

Architecture and Input, Guidance, and Action Interface

  • The planner specifies each operation, and the agent may add an inference-time reward specification before πRLinf generates a short manipulation segment.The policy combines visual observations, language instructions, and robot state; its output is continuous robot action.Source: §D.1 · p. 17

Input, Guidance, and Action Interface

  • For LIBERO, πRLinf receives external and wrist RGB views, proprioceptive state, and a language prompt, then returns a seven-dimensional action chunk for execution.The planner receives a refreshed observation after the chunk executes.Source: §D.1 · p. 17

Training and Deployment

  • The checkpoint is used without policy training or target-domain fine-tuning; skill evolution changes procedures, while reward guidance steers the selected operation during inference.Policy parameters stay frozen, and the agent’s reward specification is a control signal rather than a training label or parameter-update objective.Source: §D.1 · p. 17

Performance and Role in RoboBridge

  • 50.0% overall success rate is πRLinf’s policy-only LIBERO-PRO result under the reported evaluation protocol.The score is a reference for separating the frozen policy’s contribution from procedural planning, skill evolution, and inference-time reward guidance.Source: §D.1 · p. 17

D.2 πDROID: OPENPI DROID CHECKPOINT

πDROID is RoboBridge’s OpenPI π0.5 checkpoint for physical-robot execution. The agent coordinates its use and supplies optional inference-time guidance, while feedback supports revisions to the transferred procedure rather than policy retraining.

Architecture and Checkpoint

  • The OpenPI π0.5 checkpoint remains unchanged as the agent provides the current manipulation instruction and optional reward guidance for physical execution.The DROID-oriented deployment stack adapts action generation to the current physical context.Source: §D.2 · pp. 17–18

Observation, Guidance, and Action Processing

  • Before a policy call, the physical agent obtains a fresh observation, positions the end effector, and specifies desired outcomes and constraints when additional steering is needed.The generated short action chunk is executed through the physical control interface, after which images and robot state are refreshed for verification.Source: §D.2 · p. 18

Frozen Use and Agent Coordination

  • During real-world skill evolution, the frozen checkpoint receives inference-time reward signals, while revisions can change pre-positioning, action order, recovery, and verification.The agent selects the target and local operation, checks preceding positioning, and decides when to construct reward guidance.Source: §D.2 · p. 18

Role in Physical Transfer

  • πDROID supplies low-level manipulation for physical transfer, while RoboBridge transfers task-level skill organization and adapts local operations to the target robot’s observations and dynamics.Execution feedback supports later skill revision without retraining the full VLA model; the checkpoint is evaluated as part of the complete transfer and adaptation process, not by an isolated policy-only score.Source: §D.2 · p. 18

E PROMPT, SKILL, AND STRUCTURED ARTIFACT ORGANIZATION

RoboBridge separates reusable skills, environment-specific prompts, and structured execution records. Simulation and physical prompts share a planner contract, while skill revisions are bounded and evaluated before persistence, and physical execution adds hardware-aware constraints.

E PROMPT, SKILL, AND STRUCTURED ARTIFACT ORGANIZATION; E.1 SIMULATION PLANNER PROMPT

  • The simulation planner uses the newest observation, selects one operation, waits for feedback, and verifies state; tool completion is distinct from task success.Final outcomes use the benchmark completion predicate, and task-specific behavior comes from the active skill rather than the prompt.Source: §E.1 · pp. 18–19

E.2 PHYSICAL-ROBOT SYSTEM PROMPT

  • The physical prompt adds fresh external and wrist sensing, workspace and motion constraints, pre-positioning before policy calls, and stopping when execution or sensing becomes unreliable.A completed action chunk or tool call does not by itself prove the intended object interaction succeeded.Source: §E.2 · p. 19

E.3 SKILL-EVOLUTION SYSTEM PROMPT

  • Skill-evolution prompts use rollout evidence to propose bounded local changes, then compare candidates with the incumbent before persistence; unsupported or non-improving candidates are rejected.Revisions may change an operation, transition, recovery condition, or verification rule while preserving unaffected skill parts.Source: §E.3 · p. 19

E.4 STORED SKILL STRUCTURE

  • A stored skill is a compact procedure linking intent, conditions, expected observations, ordered operations, verification, and recovery, framed around task-relevant states and affordances.This reduces reliance on object names or scene-specific coordinates as the sole basis for generalization.Source: §E.4 · pp. 19–20

E.5 SKILL VERSIONING AND VALIDATION

  • Versioned skills retain the incumbent as fallback and promote a candidate only when validity checks pass and the selected execution criterion improves.Rejected candidates remain audit records, preventing one failed rollout from permanently changing procedural memory.Source: §E.5 · p. 20

E.6 STRUCTURED OBSERVATION AND EPISODE FILES; E.7 EPISODE ARTIFACT ORGANIZATION

  • Execution records separate procedural skills, observation snapshots, perception audits, and rollout feedback, preserving diagnosis data without placing internal perception details in planner requests.Episode files can help locate failures in object binding, positioning, action generation, tool ordering, or verification.Source: §E.6 · p. 20, §E.7 · p. 21

E.8 SIMULATION AND REAL-ROBOT PROMPT ALIGNMENT

  • Simulation and physical prompts retain the shared skill format and update protocol, while physical execution conditions are adapted through the prompt and feedback records.The simulation prompt emphasizes benchmark interaction and predicate-based completion; the physical prompt adds sensing, pre-positioning, hardware-aware stopping, and operator-based labels.Source: §E.8 · p. 21
Loading 2610.02717v1…