Source-linked AI summary
RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks
Ruiying Li, Yunlang Zhou, YuYao Zhu, Kylin Chen, Jingyuan Wang, Sukai Wang, Kongtao Hu, Minhui Yu, Bowen Jiang, Zhan Su, Jiayao Ma, Xin He, Yongjian Shen, Yang Yang, Guanghui Ren, Maoqing Yao, Wenhao Wang, Yao Mu
TL;DR
Scaling language-driven robotic manipulation to long-horizon tasks requires substantial human involvement and coordination across data collection, policy learning, and execution. RoboClaw unifies these stages in a VLM-driven agent loop using self-resetting action pairs and context-driven skill orchestration, and it reports reduced human time and improved long-horizon task performance.
Problem
Scaling VLA systems to complex, long-horizon manipulation remains challenging because data collection and deployment require substantial human involvement across multiple interdependent subtasks.
Method
RoboClaw unifies data collection, policy learning, and long-horizon task execution within a VLM-driven agent using self-resetting Entangled Action Pairs and context-driven skill orchestration.
Results
RoboClaw significantly outperforms both baselines on long-horizon tasks, while reducing human data-collection time to approximately 1/2.16 of the manual baseline.
Takeaways & Limitations
Closed-loop collected trajectories improve individual policy robustness, while progress monitoring and automatic recovery-policy invocation support long-horizon task execution.
Takeaways & Limitations
The framework may incur latency from cloud-based large models and assumes practical inverse reset behaviors for constructing reusable environment states.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) systems have shown strong potential for language-driven robotic manipulation. However, scaling them to long-horizon tasks remains challenging. Existing pipelines typically separate data collection, policy learning, and deployment, resulting in heavy reliance on manual environment resets and brittle multi-policy execution. We present RoboClaw, an agentic robotics framework that unifies data collection, policy learning, and task execution under a single VLM-driven controller. At the policy level, RoboClaw introduces Entangled Action Pairs (EAP), which couple forward manipulation behaviors with inverse recovery actions to form self-resetting loops for autonomous data collection. This mechanism enables continuous on-policy data acquisition and iterative policy refinement with minimal human intervention. During deployment, the same agent performs high-level reasoning and dynamically orchestrates learned policy primitives to accomplish long-horizon tasks. By maintaining consistent contextual semantics across collection and execution, RoboClaw reduces mismatch between the two phases and improves multi-policy robustness. Experiments in real-world manipulation tasks demonstrate improved stability and scalability compared to conventional open-loop pipelines, while significantly reducing human effort throughout the robot lifecycle, achieving a 25% improvement in success rate over baseline methods on long-horizon tasks and reducing human time investment by 53.7%.
1 Introduction
RoboClaw addresses the difficulty of scaling VLA manipulation to long-horizon tasks by unifying data collection, policy learning, and execution under one VLM-driven agent. EAP supports autonomous data collection, while contextual skill orchestration supports robust task execution.
- Motivation: Long-horizon robotic tasks require many interdependent subtasks, but conventional data collection and deployment demand substantial human supervision.Operators must demonstrate, reset environments, monitor failures, filter trajectories, evaluate policies, and supervise execution.
- Motivation: Separating data collection, learning, and execution creates semantic and state-distribution mismatches that can make errors propagate during long-horizon tasks.The paper identifies a need for a unified semantic representation and decision mechanism across the robot lifecycle.
- Framework: RoboClaw unifies data collection, policy learning, and task execution within a single VLM-controlled agent loop.The VLM reasons over environmental observations and structured memory to maintain consistent task semantics and decision logic.
- Lifecycle learning: RoboClaw closes the lifecycle loop by reintegrating execution trajectories into training to continually improve policies and expand the policy pool.The workflow includes streaming updates from online rollout and later activation of learned policies for complex tasks.
- Data collection: Entangled Action Pairs couple forward manipulation policies with inverse recovery behaviors to form self-resetting loops for continuous online data collection.The paired actions return the robot to reusable precondition regions, reducing frequent manual resets while keeping collection aligned with execution conditions.
- Task execution: The agent dynamically selects skills, monitors subtask states, validates execution conditions, and triggers recovery behaviors during long-horizon tasks.This context-driven orchestration replaces static skill sequences and constant human monitoring.
2 Related Works
Prior work reduces human effort through closed-loop feedback, simulation, planning, or hierarchical task abstraction, but real-world deployment often remains insufficiently adaptive. RoboClaw adds an inference-time supervisory agent that monitors execution and dynamically selects recovery or intervention strategies.
- Closed-loop learning: Closed-loop and semi-automated systems reduce human burden through residual correction, failure detection, or rewind-and-refine mechanisms.These approaches still include human-assisted correction or intervention in the loop.
- Simulation and planning: Simulation-based systems automate demonstration synthesis, while LLM-based methods generate plans, code, or spatial constraints for task execution.Examples include simulation-in-the-loop refinement and language-model-generated manipulation constraints.
- RoboClaw distinction: RoboClaw proposes a fully learning-driven data collection framework with autonomous process monitoring and skill scheduling during inference.The design targets real-time error recovery and robust execution without relying on manual intervention or predefined fields.
- Embodied foundation models: VLA models unify perception, language, and action, but long-horizon execution remains vulnerable to error accumulation and limited execution-time supervision.Hierarchical approaches introduce subtask abstractions and planning structures, but the related-work discussion motivates additional runtime oversight.
- RoboClaw distinction: Its context-aware supervisory agent continuously monitors subtasks and dynamically chooses retry, recovery, or human intervention strategies.The design is described as decoupled from specific task structures or skill libraries.
3 Method
RoboClaw unifies perception, reasoning, memory, tool use, policy execution, and data collection in a closed-loop agent architecture. It combines a VLM-driven controller with modular skills and forward–inverse policy pairs to support autonomous collection and long-horizon execution.
- Agent architecture: RoboClaw uses a VLM as a high-level controller that reasons over visual observations and structured memory to select and execute skills.The closed-loop process repeatedly updates memory, invokes tools through MCP, and interacts with the environment until task completion.
- Agent architecture: The framework organizes capabilities hierarchically into policies, tools, and skills, allowing reusable procedures to invoke low-level manipulation policies and environment interfaces.Policies generate motor actions, tools expose operations such as starting policies or querying the environment, and skills orchestrate tool calls.
- Policy execution: RoboClaw generates structured language instructions for selected subtasks and conditions VLA policies on visual observations, instructions, and robot joint states.The low-level policy predicts a short-horizon action chunk of length H.
- Self-resetting data collection: Entangled Action Pairs couple forward manipulation trajectories with inverse reset trajectories, enabling the environment to return to its initial state without human intervention.After successful forward execution, the reset policy is triggered and the resulting trajectories are stored for subsequent policy learning.
- Deployment and lifecycle learning: During deployment, the agent selects subtasks, invokes corresponding forward policies, and uses shared decision loops and skill interfaces to support recovery and continued skill improvement.The same framework serves task execution and data collection, creating a unified lifecycle in which execution contributes additional experience.
4 Experiments
RoboClaw is evaluated on real-world manipulation tasks spanning data collection, subtask policy learning, long-horizon execution, and failure recovery. Across these experiments, it reduces human effort, improves iterative policy success, outperforms baselines on long-horizon tasks, and expands recovery capabilities.
- Experimental setup: RoboClaw is evaluated on four real-world organization and retrieval scenarios using a dual-arm mobile manipulation robot.The scenarios cover a bedroom vanity table, kitchen shelf, study desk, and convenience-store shelf.
- Data collection efficiency: The manual data-collection baseline requires approximately 2.16× more human time than RoboClaw for the same amount of data.Human effort is normalized to RoboClaw as Ours = 1.
- Data collection efficiency: The manual baseline requires approximately 8.04× more human intervention during rollouts than RoboClaw.RoboClaw performs most data collection autonomously while monitoring the environment and generating trajectories through Entangled Action Pairs.
- Subtask policy success: Iterative rollout improves all four forward policies, including Body Lotion from 21/50 to 43/50 and Lipstick insertion from 2/50 to 23/50.Primer improves from 23/50 to 40/50, while Tissue Wipe improves from 11/50 to 26/50 across five iterations.
- Long-horizon execution: RoboClaw significantly outperforms both baselines on the vanity-table long-horizon task by monitoring progress and invoking recovery policies when necessary.The task combines multiple manipulation subtasks with distinct difficulty characteristics, including constrained insertion and sustained surface contact.
- Failure recovery: Iterative rollout improves existing policies and expands the behavioral repertoire by learning recovery strategies.A growing policy library containing nominal and recovery behaviors increases the range of environment states RoboClaw can handle reliably.
5 Conclusion
RoboClaw unifies data acquisition, policy learning, and long-horizon task execution within a single VLM-driven agent loop. The framework’s main limitations are cloud-model latency and reliance on practical inverse reset behaviors.
- RoboClaw integrates data acquisition, policy learning, and long-horizon task execution within one VLM-driven agent loop.
- Cloud-based large models may introduce latency into the framework.
- The framework assumes practical inverse reset behaviors to construct reusable environment states.
- The authors position RoboClaw as a foundation for scalable embodied AI despite these challenges.