PaperlayerView exact revision on arXiv

Source-linked AI summary

EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

Python Song, Zhixuan Liang, Kelsey Fu, Mengdi Wang, Junfeng Yang, Shilong Liu

arXiv:2610.10498v1cs.AI

TL;DR

Robot foundation models can lose performance when deployment conditions change, while adapting the model can require costly robot-data collection. EmbodiedRSI uses hypothesis-guided experiments to evolve code, skills, and memory around the model; it reports strong results on simulated benchmarks and zero-shot real-world transfer.

  • Problem

    Self-evolving harnesses use costly physical trials inefficiently and often lack principled experiment selection, coordinated code-skill improvement, and memory learned for later adaptation.

    Source: §1 · p. 2

  • Method

    EmbodiedRSI selects physical experiments to distinguish code and skill hypotheses, uses outcomes for joint updates, and learns memory actions by their downstream value to the Fast System.

    Source: §1 · p. 2, §3.3 · p. 6

  • Results

    77.0% overall success on RoboCasa365, including 71.3% on Composite-Unseen, 86.8% overall success on LIBERO-Pro, and 71.3% overall success across multiple challenging real-world tasks after zero-shot transfer.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Continual robot learning can occur in a harness around a frozen robot foundation model when physical evidence shapes experiment selection and knowledge retained for future tasks.

    Source: §5 · p. 10

    The approach improves executable code and skills without updating the robot foundation model’s parameters; future work could use selected trajectories for online model updates.

    Source: §5 · p. 10

Abstract

from arXiv · show

Robot foundation models provide strong visuomotor control, yet their performance can degrade when object positions or task instructions change. Further improvements often require post-training on substantial robot data, which can be costly to collect through methods such as teleoperation. Agentic harnesses can adapt around the model, but current self-evolving harnesses use robot trials inefficiently when deciding which code and skill changes to pursue. We introduce EmbodiedRSI, a self-evolving agentic harness that autonomously decides where to explore next and turns the resulting physical interaction into improved code and skills. EmbodiedRSI realizes this through a Fast-Slow Dual-System Architecture, in which competing code and skill hypotheses are maintained in a Hypothesis Graph. Value-of-Information Experiment Selection chooses physical experiments that can distinguish these hypotheses. Their outcomes guide Code-Skill Co-Evolution. The Slow System builds Hierarchical Memory, and Reward-Grounded Memory Learning selects effective memory according to their value for later Fast-System improvement. On RoboCasa365, EmbodiedRSI reaches 77.0% overall success and 71.3% on Composite-Unseen, compared with 40.1% for the best baseline. EmbodiedRSI also reaches 86.8% overall success on LIBERO-Pro. Beyond benchmark performance, EmbodiedRSI transfers zero-shot to real-world robot, achieving 71.3% overall success across multiple challenging tasks.

1 Introduction

Existing self-evolving harnesses use costly physical trials inefficiently and often evolve code, skills, and memory separately. EmbodiedRSI selects informative experiments and uses their outcomes for code-skill co-evolution and memory, with strong benchmark and real-world results.

1 Introduction

  • Existing harnesses use costly physical trials inefficiently, often pursuing related improvement directions without selecting experiments for their information value or jointly evolving code and skills.Memory is also commonly accumulated for retrieval rather than learned according to its value for later self-evolution.Source: §1 · p. 2
  • EmbodiedRSI proposes and executes experiments that distinguish code or skill hypotheses, updates confidence from outcomes, and records refinement history in a Hypothesis Graph.The resulting evidence guides Code-Skill Co-Evolution; Hierarchical Memory is selected by downstream value to the Fast System.Source: §1 · p. 2
  • A frozen robot foundation model can be surrounded by a self-evolving harness that turns physical interaction into reusable code and skill improvements that generalize to unseen scenes and tasks.The experiments also report preservation of the model’s local control competence.Source: §1 · p. 2
  • 77.0% overall success on RoboCasa365, including 71.3% on Composite-Unseen, and 86.8% overall success on LIBERO-Pro; real-world success reaches 71.3%.The real-world result comes from zero-shot transfer across multiple challenging tasks.Source: §1 · p. 2
  • The introduction frames autonomous selection of where to explore as a way to learn more efficiently from physical interaction.The Fast System selects experiments using hypothesis weights and cost, while the Slow System builds memory from past experience.Source: §1 · p. 2

2 Related Work

Prior work surrounds frozen robot policies with planning, memory, tools, or evolving skills and code. EmbodiedRSI distinguishes itself by linking physical attempts and outcomes in a Hypothesis Graph before jointly updating code and skills, while learning which memory actions support later improvement.

Agentic Execution Around Frozen Robot Foundation Models

  • EmbodiedRSI links physical attempts, candidate code or skill updates, and outcomes in a Hypothesis Graph before Code-Skill Co-Evolution constructs an update.Related approaches include frozen-VLA harnesses and systems that evolve runtime critics, response skills, and tools at multiple time scales.Source: §2 · p. 2

Skill Libraries and Code-Mediated Improvement

  • Code-as-Policies generates robot-control programs, while CaP-X, Playful Agentic Robot Learning, and Voyager build or accumulate executable skills from successful executions or environment feedback.Other approaches revise skills or optimize an external skill and harness layer, while Eureka evolves reward programs.Source: §2 · pp. 2–3

Embodied Memory and Continual Learning

  • Embodied-RAG develops hierarchical memory for embodied retrieval and generation, and MemSkill learns memory-writing skills; EmbodiedRSI learns from later task outcomes which memory actions support self-improvement.EmbodiedRSI builds reusable memory from physical transitions.Source: §2 · p. 3

3 Method

EmbodiedRSI formulates continual harness improvement as selecting physical experiments from current hypotheses and memory, then incorporating their evidence into the harness. Its Fast System selects informative, cost-aware trials and co-evolves code and skills; its Slow System organizes experience and learns how to manage memory for later experiments.

3.1 Problem Formulation

  • Each episode selects a physical experiment from candidates using the current hypotheses and memory, then incorporates its evidence into the evolving harness.With a budget of B episodes, the objective is to maximize task success rate over evaluation episodes.Source: §3.1 · pp. 3–4

3.2 Fast System: Rapid Code-Skill Co-Evolution

  • Value-of-Information Experiment Selection ranks matched trials by expected uncertainty reduction per experiment cost, using outcomes to update hypothesis confidence and subsequent experiment rankings.The cost combines elapsed time, simulator steps, GPU cost, and reset cost; exploration stops at uncertainty or score thresholds, the rollout budget, or the per-hypothesis evaluation cap.Source: §3.2 · p. 4
  • Code-Skill Co-Evolution measures code-only, skill-only, and joint success from the same initial states; positive joint gain records a supported works-with relation.The Hypothesis Graph retains revision history, and physical outcomes inform which executable improvements and code-skill relations to evaluate next.Source: §3.2 · pp. 4–5, Figure 3 · p. 5

3.3 Slow System: Hierarchical Memory and Reward-Grounded Memory Learning

  • Hierarchical Memory preserves experience at three levels: physical execution records, code and skill hypotheses, and reusable cognition distilled from repeated evidence.The Slow System abstracts M1 and M2 evidence into M3 and retrieves hypotheses and cognition to guide candidate experiments.Source: §3.3 · p. 5
  • Reward-Grounded Memory Learning trains the Slow-System policy to select memory actions according to later Fast-System improvement, measured through success, exploration uncertainty, and experiment cost.Candidate actions include Retain, Merge, Abstract, Compress, Forget, and Skip; PPO maximizes expected discounted return.Source: §3.3 · pp. 5–6

4 Experiments

Experiments assess generalization across simulated task suites, zero-shot real-robot transfer, and component contributions. EmbodiedRSI’s strongest reported benchmark gains include unseen task compositions.

4.1 Experimental Setup

  • RoboCasa365 probes generalization across household scenes and task compositions, while LIBERO-Pro tests changed instructions and object positions; evaluation uses disjoint evolution and test seeds.Each task is evaluated with 10 random seeds and 10 trials per seed.Source: §4.1 · p. 6

4.2 Generalization on RoboCasa365 and LIBERO-Pro

  • 77.0% overall RoboCasa365 success; on Composite-Unseen, EmbodiedRSI reaches 71.3% versus Harness VLA’s 40.1%.Composite-Unseen combines familiar skills into task compositions absent from pretraining.Source: §4.2 · p. 6, Table 1 · p. 7
  • 86.8% overall LIBERO-Pro success versus Harness VLA’s 72.1%, with EmbodiedRSI leading in all eight perturbation cells.The benchmark tests changed instructions and object positions.Source: Table 2 · p. 7, §4.2 · p. 7

4.3 Zero-Shot Sim-to-Real Transfer of the Agentic Harness

  • 71.3% overall real-robot success versus 46.0% for SmolVLA alone, with the foundation model frozen during trials.The evaluation covers five challenging tasks, including multi-step manipulation, semantic and arithmetic reasoning, and precision grasping.Source: Table 3 · p. 8, §4.3 · p. 8

4.4 Ablation Study

  • 71.3% Composite-Unseen success for Code-Skill Co-Evolution versus 55.6% for Skill-only Evolution; joint evolution leads on both suites.Code-only Evolution outperforms Skill-only Evolution on Atomic-Seen, but the ranking reverses on Composite-Unseen.Source: Figure 6 · p. 9, §4.4 · p. 9
  • 72.9% overall success versus 27.1%, with learning AUC 0.635 versus 0.309 after 50 physical episodes.In a representative SPAT-T case, Value-of-Information selection reaches sustained 100% eight-episode rolling success at episode 48, while random selection continues fluctuating through episode 100.Source: Table 4 · p. 9, Figure 7 · p. 9, §4.4 · p. 9
  • 71.3% Composite-Unseen success and 0.635 learning AUC for full Reward-Grounded Memory Learning, versus 61.7% and 0.556 for retrieval-based selection and 65.2% and 0.589 for PPO with task reward alone.The ablations attribute improved success and learning efficiency to co-evolution, hypothesis-guided exploration, and reward-grounded memory learning.Source: §4.4 · p. 9, Table 5 · p. 10

5 Conclusion

EmbodiedRSI uses physical outcomes to select experiments, improve code and skills together, and retain experience for later adaptation. The harness improves generalization and learning efficiency across simulated and real-robot tasks, while the foundation model remains frozen.

5 Conclusion

  • Across simulated benchmarks and real-robot tasks, EmbodiedRSI improves generalization and learning efficiency and transfers the simulation-evolved harness to physical execution.Continual robot learning takes place in the harness around a frozen robot foundation model.Source: §5 · p. 10
  • Each physical trial informs the next experiment, joint code-and-skill refinement, and experience retained for later adaptation.The conclusion presents this as a way for a self-evolving agentic harness to gain lasting value from physical trials.Source: §5 · p. 10
  • Code-as-policy and agentic harness methods learn executable behavior from physical trajectories without updating the robot foundation model’s parameters.Online model updates using harness-selected trajectories are proposed as future work so code, skills, and the model could improve together.Source: §5 · p. 10

Related papers

Loading 2610.10498v1…