PaperlayerView exact revision on arXiv

Source-linked AI summary

Playful Agentic Robot Learning

Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, Trevor Darrell

arXiv:2606.19419v1cs.ROcs.AI

TL;DR

Current agentic robot systems generally acquire reusable skills only after receiving explicit task instructions. RATS uses self-directed play to build a persistent skill library, improving held-out downstream success by 20.6 and 17.0 percentage points on LIBERO-PRO and MolmoSpaces over CaP-Agent0.

  • Problem

    Most agentic robot systems remain task-driven, acquiring reusable skills only after receiving external instructions rather than through proactive continual learning.

  • Method

    RATS uses self-directed play to propose exploratory tasks, execute and verify Code-as-Policy programs, diagnose failures, retry, and store successful behaviors in a persistent skill library.

  • Results

    20.6 and 17.0 percentage-point gains over CaP-Agent0 on LIBERO-PRO and MolmoSpaces, respectively, show improved held-out downstream success from play-learned skills.

  • Takeaways & Limitations

    Play-learned code skills provide a plug-and-play way to improve agentic robot systems without finetuning the underlying model.

  • Takeaways & Limitations

    The evaluation remains primarily simulation-based, requiring larger-scale physical deployment to validate robust sim-to-real transfer.

Abstract

from arXiv · show

Current agentic robot systems can write executable Code-as-Policy programs, observe feedback, and revise behavior across multiple attempts, but they remain largely task-driven: reusable skills are acquired only after explicit instructions. We study Playful Agentic Robot Learning, where an embodied coding agent uses self-directed play as a continual skill-learning stage before downstream tasks arrive. We introduce RATs, Robotics Agent Teams designed for play-time skill acquisition. During play, RATs proposes novel yet learnable exploratory tasks, plans and executes robot-code policies, verifies intermediate progress, diagnoses failures, retries with dense, step-level feedback, and distills successful executions into a persistent code skill library. At test time, the agent reuses relevant skills from this frozen library to help solve new tasks. Experiments in LIBERO-PRO and MolmoSpaces show that play-learned skills improve held-out downstream tasks over no-play and random-play baselines, with 20.6 and 17.0 percentage-point gains over CaP-Agent0 on LIBERO-PRO and MolmoSpaces, respectively. Moreover, the learned skills can be plugged into other inference-time Code-as-Policy agents by simply retrieving them into the context, improving RoboSuite and real-world transfer by 8.9 and 8.8 points, respectively, without finetuning the underlying model.

1 Introduction

Playful Agentic Robot Learning uses self-directed play before deployment to acquire reusable robot skills. RATS operationalizes this process through exploratory task generation, Code-as-Policy execution, dense feedback, failure recovery, and persistent skill-library updates that improve downstream performance.

  • Motivation and formulation: RATS turns self-directed play into an explicit continual skill-acquisition stage before downstream tasks arrive.The approach is motivated by children’s play-based acquisition of reusable skills and intrinsic-motivation principles favoring experiences that are novel yet learnable.
  • RATS design: RATS proposes exploratory tasks, plans and executes Code-as-Policy programs, verifies progress, diagnoses failures, retries, and distills successful behavior into a persistent library.Its structured agent team is designed to provide dense, step-level feedback over intermediate subgoals and execution attempts.
  • Evaluation results: 20.6 percentage points over CaP-Agent0 is RATS’s average-success improvement on LIBERO-PRO, while 17.0 points is its improvement on MolmoSpaces.These evaluations test whether skills acquired during play improve held-out downstream benchmark tasks.
  • Transfer and reuse: 8.9 points is the RoboSuite success improvement when LIBERO-PRO skills are plugged into CaP-Agent0, demonstrating cross-environment skill transfer.The skills can be reused by another inference-time Code-as-Policy agent through retrieval into context, without finetuning the underlying model.

2 Related Work

Related work frames robot learning through self-directed play, continual acquisition and reuse of skills, and Code-as-Policy agents that synthesize modular executable programs. These strands motivate learning reusable capabilities through exploration and agentic planning.

  • Play, Curiosity, and Developmental Robotics: Play is viewed as a self-directed curriculum in which agents explore novel, meaningful interactions near their current competence boundary.Developmental robotics links play to discovering controllable effects and practicing emerging motor routines before explicit tasks arrive.
  • Continual Skill Learning and Curriculum in Robotics: Continual robot learning focuses on acquiring, retaining, and reusing transferable skills, while curriculum learning structures experience to support long-horizon learning.Reusable representations include skills, options, motor primitives, affordances, behavioral priors, and hierarchical policies.
  • Code-as-Policy and Agentic Robot Learning: Code-as-Policy methods use large language or multimodal models to synthesize executable robot programs that compose perception, planning, and control APIs.These programs are described as modular, inspectable, and reusable, alongside embodied systems that ground language plans in pretrained skills, affordances, scene representations, visual feedback, or 3D value maps.

3 Robotics Agent Teams for Playful Agentic Robot Learning

RATS is a multi-agent Code-as-Policy system that enables playful, self-directed skill acquisition before downstream tasks. It proposes learnable exploratory tasks, executes and retries them with verification and diagnosis, and stores successful behaviors in a reusable library for test-time solving.

  • Play-Time Task Generation: RATS removes external task instructions during play, autonomously proposing tasks from scene observations, prior attempts, and intrinsic motivation.This play-time formulation targets reusable skills that improve later downstream task solving.
  • Memory and Skill Learning: Successful executions are distilled into documented experimental code skills, while failures become compact lessons in persistent stores that prioritize reliable skills and hide repeatedly failing ones.Periodic curation merges duplicates, removes redundant lessons, and proposes helper functions when failures reveal repeated missing capabilities.
  • Play-Time Task Generation: RATS selects exploratory tasks using a Goldilocks objective that balances novelty with learnability, favoring semi-familiar tasks near a success rate of approximately 0.5.The selection avoids tasks that are trivial or impossible by combining exploration and competence-frontier signals.
  • Agent-Team Execution: The agent team coordinates planning, code writing, execution, goal and per-step verification, failure diagnosis, and retries in a Write-Execute-Verify-Diagnose loop.The planner retrieves relevant skills and the inner loop continues within a retry budget until success or exhaustion.
  • Test-Time Reuse: At test time, RATS disables exploration and memory updates and reuses the frozen learned skill library to solve externally provided tasks.The library serves as the existing knowledge base after play-time development.

4 Experimental Results

Experiments evaluate whether autonomous play improves Code-as-Policy agents through in-domain generalization, cross-environment transfer, ablations, and real-world deployment. RATS substantially improves held-out LIBERO-PRO performance and transfers learned skills across embodiments to RoboSuite.

  • Evaluation setup: The study reports task success in plug-in mode, which retrieves skills into CaP-Agent0’s context, and full-system mode, RATS Exec.Experiments compare against VLA policies and CaP-Agent0 with only the primitive library under the No Play condition.
  • In-domain generalization: RATS raises LIBERO-PRO average success from 23.2% to 43.8%, a +20.6 pp gain over CaP-Agent0.The evaluation covers 60 held-out tasks with 10 initializations each, totaling 600 trials.
  • In-domain generalization: LIBERO-PRO gains are largest on object splits at 61.0% and 63.0%, while also appearing on goal and spatial splits.These results indicate generalization beyond the tasks practiced during play.
  • Cross-environment transfer: Transferred skills raise RoboSuite average success from 40.3% to 49.1%, a +8.9 pp gain over CaP-Agent0.Skills learned during LIBERO-PRO play transfer to RoboSuite, which is unseen during play, across 50 trials per task.
  • Cross-environment transfer: RoboSuite transfer includes +16.0 pp on cube lifting and +24.0 pp on two-arm lifting.The two-arm result is cross-embodiment because skills were practiced on a single-arm LIBERO-PRO robot.

5 Conclusion

RATS is a playful agentic robot learning framework that acquires reusable Code-as-Policy skills before downstream tasks. Across simulated and real-world settings, self-proposed play improves performance over baselines, outperforms random play under the same budget, and supports cross-environment transfer.

  • Framework: RATS acquires reusable Code-as-Policy skills through play before downstream tasks are provided.The framework is designed for pre-task skill acquisition rather than learning only after explicit downstream instructions.
  • Performance: RATS substantially improves over CaP-Agent0 and VLA baselines across LIBERO-PRO and MolmoSpaces.Ablations further show that random play under the same budget produces much smaller gains.
  • Transfer: Skills learned through self-proposed play transfer beyond the original play environment in RoboSuite and real-world evaluations.These cross-environment results suggest that the learned skills are not limited to the environments in which play occurred.

Limitations · Appendix · A Implementation Details about RATS

The paper identifies simulation dependence, limited practice diversity, and mismatched skill reuse as key limitations of RATS, while the appendix details its teams, skill-library evaluation, and prompt interfaces.

  • Limitations: RATS remains primarily simulation-based, so larger-scale physical deployment is needed to validate robust sim-to-real transfer.This limitation concerns the extent to which simulation results generalize to physical deployment.
  • Limitations: Play-time skill acquisition is constrained by simulation diversity, limiting practice across objects, dynamics, and affordances.The available environments restrict the range of exploratory experiences the agent can obtain.
  • Limitations: Improper skill reuse can reduce downstream performance when retrieved skills do not fit the task.The limitation arises during reuse of learned skills for downstream tasks.
  • A Implementation Details about RATS: The appendix introduces implementation details for RATS, including its three constituent teams.These teams are the Task Proposer, Execution, and Memory-Management teams.
  • A Implementation Details about RATS: The Task Proposer team handles play-time task proposal, while the Execution team handles task execution.These responsibilities correspond to two of the three RATS teams described in the appendix.
  • A Implementation Details about RATS: The Memory-Management team updates the skill library and failure memory.This team covers persistent skill and failure-memory maintenance.
  • A Implementation Details about RATS: The appendix describes learned-skill-library use during evaluation and summarizes each agent’s prompt interfaces.These details explain how skills are applied at evaluation time and how agents are prompted.

A.1 Details of Play-Time Task Proposal

Play-time task proposal combines library-aware candidate generation with Goldilocks-driven selection, then converts selected proposals into validated executable environments. The process adapts scene grounding and accessibility constraints across LIBERO and MolmoSpaces before execution.

  • Candidate task generation: The Task Proposer uses scene context, a compact skill-library summary, and recent attempt history to generate exploratory task candidates.The summary contains skill names, descriptions, reliability tiers, and empirical success statistics, but not full source code.
  • Scene grounding: Scene grounding is environment-specific: LIBERO exposes object inventories and manipulation primitives, while MolmoSpaces shows only readable, visible, reachable objects.This prevents proposals from targeting objects present in metadata but inaccessible from the robot’s current state.
  • Goldilocks-driven task selection: Candidates are ranked by the product of object-skill novelty and competence, with competence estimated using Wilson lower bounds rather than raw success rates.Historical object-skill-pair counts provide novelty, while the lower bound avoids treating rarely attempted skills as reliable after one lucky success.
  • Environment creation: The Environment Creator converts selected proposals into executable instances, generating and validating LIBERO BDDL specifications with one bounded repair when validation fails.Static checks cover referenced objects, fixtures, regions, predicates, and assets before MuJoCo instantiation.
  • Environment verification: Before execution, the Environment Verifier checks LIBERO instantiation across two reset seeds and validates rendering, simulator access, object bodies, goal predicates, and severe initial penetrations.MolmoSpaces instead constrains creation through a bridge catalog and verifies the proposed target after environment rebinding.
  • Failure-driven refinement: A play-time attempt can reveal a concrete execution failure, such as grasp-centering, and motivate creation of a new helper skill.The example trace identifies the failed grasp as the reason for adding assistance.

A.1.1 Example Trace of Play-Time Task Proposal … B Additional Real-World Experimental Results

RATS combines competence-frontier play-time task selection with a bounded execution-and-diagnosis loop, persistent skill and failure stores, and frozen-library evaluation through both RATS and plug-and-play Code-as-Policy execution. The system proposes varied exploratory tasks, verifies and repairs policies, extracts successful helpers, and reuses learned skills at test time.

  • A.1.1 Example Trace of Play-Time Task Proposal: RATS proposes five scene-grounded alternatives using primitives, prior skills, recent outcomes, and variation rules, then selects a learnable competence-frontier objective.The proposer avoids recent duplicates, builds one-variable changes from successes, and changes direction after failures.
  • A.1.1 Example Trace of Play-Time Task Proposal: The tissue-box lift receives the highest Goldilocks score but remains unsuccessful after four attempts, revealing a grasp-centering bottleneck and motivating adjust_grasp_to_centroid.The ranker uses competence-frontier learnability after novelty, retry-bonus, and failure-penalty terms are considered.
  • A.2 Details of RATS for Execution: RATS executes accepted tasks through a bounded Write-Execute-Verify-Diagnose loop that localizes failures to planning, code generation, or physical execution.Planning uses observations, failure lessons, primitives, and active skills; verification and diagnosis guide subsequent retries.
  • A.2 Details of RATS for Execution: The Policy Writer converts verified plans into Python, while the Quality Checker rejects static errors before execution and the Goal and Per-Step Verifiers assess outcomes and localized evidence.Retries receive per-step results and working code segments, encouraging local edits instead of full-policy rewrites.
  • A.3 Details of Skill Library and Failure Memory Updates: Successful trajectories yield validated experimental helper skills, whereas repeated failures can prompt anticipatory candidate helpers that earn reliability only through later use.Skills progress through experimental, verified, and deprecated tiers according to usage and empirical success thresholds.
  • A.3 Details of Skill Library and Failure Memory Updates: RATS maintains persistent skill library L and failure memory M, updating and periodically curating both after play attempts.Skills store executable code and metadata, while failures store diagnoses and compact lessons retrievable by task and object overlap.
  • A.4 Details of Evaluation With the Learned Skill Library: At test time, the learned library L is frozen, task proposal and curation are disabled, and evaluation compares plug-and-play CAP-AGENT0 execution with full RATS execution.Plug-and-play exposes selected skill signatures, descriptions, source code, and executable definitions to the baseline context and namespace.
  • A.4 Details of Evaluation With the Learned Skill Library: Full RATS evaluation provides the frozen library to the Planner, prioritizes verified skills, injects selected executable definitions and dependencies, and retains verification, diagnosis, and bounded retry.The Policy Writer generates code conditioned on skills selected for individual plan steps.

B.1 Additional Quantitative Results for Real-World Experiments

MolmoSpaces play-learned skills improve CAP-AGENT0 performance on additional real-world manipulation tasks involving object rearrangement and articulated-object interaction. Across Swap Cubes and Close Drawer, average success rises from 3.3% to 25.0%, a +21.7 percentage-point gain.

  • Evaluation setup: MolmoSpaces play-learned skills are evaluated on Swap Cubes and Close Drawer using the CAP-AGENT0 system.Swap Cubes requires object exchange around a platform, while Close Drawer requires closing an initially open drawer.
  • Swap Cubes: 7 out of 30 trials succeed on Swap Cubes with skills, whereas standard CAP-AGENT0 fails to solve the task.The skill-augmented agent reuses skills learned by RATS during MolmoSpaces play.
  • Close Drawer: 8/30 successes on Close Drawer improve over the standard CAP-AGENT0 rate of 2/30.This task tests whether the robot can close an initially open drawer.
  • Aggregate result: +21.7 percentage-point gain: average real-world success improves from 3.3% to 25.0% across the two tasks.The comparison averages performance on Swap Cubes and Close Drawer.

B.2 Additional Qualitative Results · C Details about Play Process

Additional qualitative results show that CAP-AGENT0, augmented with RATS skills learned through MolmoSpaces play, successfully transfers those skills to newly added real-world tasks. The play-objective distribution is computed from 49 saved proposal records collected during a 50-iteration run.

  • B.2 Additional Qualitative Results: CAP-AGENT0 augmented with MolmoSpaces play-learned RATS skills successfully executes additional real-world tasks.These examples are presented as qualitative evidence of skill transfer.
  • B.2 Additional Qualitative Results: The augmented agent solves the newly added Swap Cubes task.The robot exchanges a cube on the platform with a cube below the platform.
  • B.2 Additional Qualitative Results: In Swap Cubes, the robot exchanges a platform cube with a cube positioned below the platform.
  • B.2 Additional Qualitative Results: The augmented agent solves the newly added Close Drawer task by closing an initially open drawer.
  • B.2 Additional Qualitative Results: The augmented agent also solves an Open Drawer task among the additional qualitative real-world examples.
  • B.2 Additional Qualitative Results: Figure 6 reports play-objective counts from 49 saved proposal records available for the 50-iteration run.

C.1 Play-Time Objectives and Knowledge Accumulation

During a 50-iteration MolmoSpaces play run, RATs proposed diverse, novel-yet-learnable objectives and accumulated reusable skills alongside distilled failure knowledge. Successful and unsuccessful attempts both expanded persistent knowledge through reuse, refinement, and new helper creation.

  • Objective distribution: Across 49 recorded proposals, the proposer covered seven interaction families and more than 30 object categories, with pull and lift dominant.One iteration lacked a saved proposal artifact, so the distribution uses the remaining 49 records.
  • Skill and memory growth: The learned skill library grew from 6 helpers at iteration 10 to 27 helpers at iteration 50.The 50-iteration run saved skill-library and failure-memory snapshots every 10 iterations.
  • Skill and memory growth: Failure memory grew from 14 raw episodes and 8 distilled lessons to 70 episodes and 121 lessons over the same period.The snapshots also track three evolving reliability tiers, though the supplied passage truncates their values.
  • Novel yet learnable objectives: At iteration 25, a novel black-shoe slide combined novelty score 1.0 with frontier score 0.6756, succeeded after two attempts, and yielded a new helper.The task reused perception and pull helpers learned earlier; the new helper was slide_grasped_object_and_release.
  • Knowledge accumulation: Successful and failed play both generated reusable knowledge, including helpers reused for later pushes and drawer pulls and helpers refining grasp and approach strategies.Examples include a black-metal-rail push helper reused for a handheld-tool push, plus failure-derived helpers for pull direction, grasp selection, centroid adjustment, top-down grasps, and approach-axis filtering.

C.2 Learned-Skill Usage in MolmoSpaces Evaluation · C.3 Qualitative Comparison with Direct Code Synthesis

In MolmoSpaces evaluation, play-learned skills are invoked in nearly every trial and compose perception, geometry, and grasp-planning functions. Compared with direct code synthesis, the learned-skill approach succeeds by reusing verified helpers instead of reconstructing brittle low-level procedures.

  • C.2 Learned-Skill Usage in MolmoSpaces Evaluation: 391 of 400 evaluation trials invoke at least one learned skill, using 14 of 27 library skills across 5,169 runtime calls.The frozen library is exposed to the policy model during test-time execution.
  • C.2 Learned-Skill Usage in MolmoSpaces Evaluation: Opening tasks average 19.0 helper calls per trial across 11 distinct learned skills, making them the heaviest users of the library.Skill composition differs by task type despite broad library reuse.
  • C.2 Learned-Skill Usage in MolmoSpaces Evaluation: Object-localization helpers account for 2,806 invocations, led by localize_and_verify_object_point_cloud with 1,873 calls.Opening tasks also combine localization with direction estimation and grasp planning; direction-and-geometry helpers comprise 32.3% of open-task calls.
  • C.2 Learned-Skill Usage in MolmoSpaces Evaluation: A successful cabinet-opening trial composes get_axis_aligned_pull_direction with select_grasp_for_pulling to estimate motion and choose a compatible grasp.The helpers were proposed after a failed play iteration and later reused during play and evaluation.
  • C.3 Qualitative Comparison with Direct Code Synthesis: With learned skills, the robot opens an iTHOR drawer; direct synthesis fails to complete the articulation while repeatedly reconstructing low-level grasp and pull reasoning.The learned path composes reusable localization, pull-grasp planning, and handle-motion helpers.
  • C.3 Qualitative Comparison with Direct Code Synthesis: Across three matched MolmoSpaces objectives, direct-synthesis runs fail while RATS succeeds using learned skills.The objectives are picking up a vintage yellow track shoe, closing the table, and placing a spanner in or on a white bowl.
  • C.3 Qualitative Comparison with Direct Code Synthesis: For closing and pick-and-place, direct synthesis performs ad hoc affordance probing and manually rebuilds perception, grasp, centroid, and release steps, whereas RATS invokes reusable helpers.The closing example uses push_object_closed; the pick-and-place comparison contrasts manual reconstruction with composition of verified skills.

C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation · D Details about the Evaluation Benchmark

RATs’ frozen LIBERO-derived skill library enables RoboSuite code policies to reuse structured grasp-and-lift behaviors implicitly, whereas direct synthesis reaches lifting but fails to complete the objective. The benchmark section evaluates externally specified tasks with the play-time library frozen and documents the simulated-domain composition and replay protocol.

  • C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation: Direct synthesis reaches the lift stage but does not complete the RoboSuite two_arm_lift task.The run samples Contact-GraspNet handle candidates and scores them with axis and direction heuristics.
  • C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation: RATs selects relevant skills from a frozen library learned from LIBERO play-time tasks.The retrieved skills support the generated RoboSuite policy without being invoked as explicit API calls.
  • C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation: The reused skills are embedded implicitly in the generated RoboSuite code rather than called explicitly.This distinguishes skill reuse from direct API invocation during final-policy execution.
  • C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation: The generated policy localizes task-relevant parts and computes 3D handle and object centers.These operations are part of the structure reused from the LIBERO-derived skill library.
  • C.4 Usage of LIBERO-Derived Skills in RoboSuite Evaluation: The reused policy constructs a side-pinch grasp frame from handle-to-body geometry and executes a coordinated close-and-lift motion.The passage presents this sequence as the concrete RoboSuite manifestation of the retrieved skill structure.
  • D Details about the Evaluation Benchmark: During evaluation, the play-time skill library remains frozen while the system solves externally specified tasks.The benchmark section covers two simulated evaluation domains and documents their composition and replay protocol.

D.1 MolmoSpaces Evaluation Benchmark · D.2 LIBERO-PRO Evaluation Benchmark · E Additional Studies

The evaluations use compact held-out MolmoSpaces and LIBERO-PRO benchmarks with controlled task and trial budgets. Across matched comparisons, RATS succeeds where direct code synthesis fails by reusing verified or learned skills.

  • D.1 MolmoSpaces Evaluation Benchmark: MolmoSpaces uses 40 held-out episodes, with ten each for opening, closing, picking, and pick-and-place across four benchmark components.The components are Open-v1, Close-v1, Pick-v2-classic, and PnP-v2.
  • D.1 MolmoSpaces Evaluation Benchmark: MolmoSpaces success is judged by judge_success(), checking joint state for articulated tasks, object target state for picking, and receptacle support for placement.The selected episodes span 37 houses and store scene, initialization, object, task, referral, and camera information.
  • D.1 MolmoSpaces Evaluation Benchmark: RATS succeeds while CaP-Agent0 fails on matched MolmoSpaces tasks by reusing verified perceptual and manipulation abstractions instead of reconstructing low-level logic inline.Qualitative comparisons cover shoe picking, table closing, and pick-and-place, with learned skills for localization, grasping, closing, transport, and release.
  • D.1 MolmoSpaces Evaluation Benchmark: The MolmoSpaces main sweep repeats each selected task ten times, producing 400 rollouts.The benchmark definition is independent of the number of repeated trials.
  • D.2 LIBERO-PRO Evaluation Benchmark: LIBERO-PRO evaluates object, goal, and spatial categories under Pos position-swap and Task task-specification perturbations, with ten language-conditioned manipulation tasks per suite.The Pos setting is the swap suite, while Task is the task suite.
  • D.2 LIBERO-PRO Evaluation Benchmark: RATS transfers LIBERO-derived skills to RoboSuite two-arm lifting, succeeding where direct code synthesis fails on the same evaluation seed.The transferred skills are selected from a LIBERO-derived library.
  • D.2 LIBERO-PRO Evaluation Benchmark: LIBERO-PRO comparisons use 600 rollouts per evaluated setup, capping trials at ten initial states per task while keeping baseline and learned-library budgets identical.All available states would yield 3,000 rollouts from 50 initial states per task.

E.1 Ablation Results on MolmoSpaces

On MolmoSpaces, RATs play-learned skills are evaluated across open, close, pick, and pick-and-place tasks using 50 play iterations. Curious Play improves both standard CAP-AGENT0 and the full RATS execution system, with larger gains for the latter.

  • Experimental setup: MolmoSpaces evaluates whether RATs play-time skill acquisition transfers to open, close, pick, and pick-and-place tasks, using 50 play iterations.All play-time skills are learned by the proposed RATS system.
  • Play strategy ablation: 25.8% average success with Curious Play under CAP-AGENT0, up from 21.0%, with the largest gain on closing tasks.This shows play-learned skills improve performance even when plugged into the standard CAP-AGENT0 test-time agent.
  • Test-time system ablation: 38.0% average success with Curious Play in the full RATS execution system, up from 32.8% without play.The full RATS system benefits more substantially than the standard CAP-AGENT0 setup.
  • Combined interpretation: The results indicate complementary benefits from play-time skill learning and structured RATS execution: the learned library helps CAP-AGENT0, while full RATS performs better.Learned skills can improve a standard Code-as-Policy agent, whereas the full execution system provides a larger improvement.

E.2 Token Cost Analysis … F Agent I/O and Prompts

RATS’s analysis shows that proactive play converts comparable token expenditure into larger downstream gains than simply extending test-time retries, while producing reusable skills and explicit agent interfaces and prompts. The appendices illustrate learned manipulation, localization, and placement capabilities and document how agents propose tasks and construct executable environments.

  • E.2 Token Cost Analysis: Play-time token consumption is dominated by failed iterations exhausting the five-attempt retry budget, with the Write-Execute-Verify-Diagnose loop as the primary cost driver.Intrinsic task proposal is relatively token-efficient, whereas diagnostics and retries account for much of the consumption.
  • E.2 Token Cost Analysis: 30M tokens over 50 play iterations are amortized across 60 held-out LIBERO-PRO tasks for compute-matched evaluation.This accounts for RATS’s upfront autonomous-play cost before comparing test-time performance.
  • E.2 Token Cost Analysis: 32.3% average success follows play-time skill reuse, versus 26.0% for CAP-AGENT0 with 15 turns and 23.2% for standard CAP-AGENT0.The comparison uses comparable compute, but the skill-augmented system retains the standard 10-turn CAP-AGENT0 test-time budget.
  • E.3 MolmoSpaces Learned Skills: MolmoSpaces examples include skills for axis-aligned pull-direction computation, closed-object pushing, and top-down wrist-camera grasp planning.The snippets expose callable interfaces and implementations that combine geometric reasoning, gripper actions, Molmo, and SAM3 segmentation.
  • E.4 LIBERO Learned Skills: LIBERO examples include verified object localization with pose aliases and segmentation fallback, plus placement on segmented targets with hover correction.The localization skill reports 18 uses and 16 successes, and the placement skill records residual target centers and corrected hover positions.
  • F Agent I/O and Prompts: RATS’s agent specifications enumerate component inputs and outputs, while fixed LIBERO prompts omit runtime-specific content behind placeholders.Omitted runtime content includes scene descriptions, API documentation, retrieved memories, visual evidence, and task-specific code.
  • F.2 Agent Prompts: The curiosity-driven proposer is prompted to suggest a simple, single-step, child-feasible manipulation task using the skill library and task history.Its output schema includes novelty score, difficulty estimate, and affordance hints describing gripper-target features.
  • F.2 Agent Prompts: The Environment Creator prompt converts a high-level task proposal into valid BDDL for a MuJoCo environment, enforcing placement regions, bounded coordinates, and fixture initialization.The documented example places butter in a microwave heating region and requires the microwave to be closed.
Loading 2606.19419v1…