Source-linked AI summary
Zetta $ζ$: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence
Xin Ding, Liang Mi, Mingzhe Huang, Zixuan Wang, Chao Zhang, Zixu Hao, Fu Chen, Xiangyu Li, Yikai Zheng, Yaoyu Guo, Weijun Wang, Kun Li, Hao Wu, Yunxin Liu, Ting Cao
TL;DR
Embodied-agent harnesses have not achieved closed-loop learning because physical execution requires state tracking faster than large agentic models can provide. Zetta evolves code-based critics and recovery skills online through coordinated loops while freezing the base policy, with Z-Infra supporting scalable rollouts. It reaches 90.8% on LIBERO-Pro and 93.6% on RoboCasa, while success scales with self-exploration and learned skills transfer zero-shot.
Problem
Existing embodied-agent harnesses are largely open-loop, so they follow fixed skills during rollout and reflect only after episodes complete despite rapidly changing physical states.
Method
Zetta evolves code-based runtime critics and recovery skills online through action-frequency, rollout-batch, and validation-gated loops while keeping base policy models frozen.
Results
90.8% on LIBERO-Pro and 93.6% on RoboCasa are achieved under the current rollout budget, with success increasing across evolution iterations and learned skills transferring zero-shot.
Takeaways & Limitations
Closed-loop harness self-evolution scales task success toward the frozen policy’s capability ceiling and transfers across tasks.
Takeaways & Limitations
Real-robot rollout collection and self-evolution remain future work, with Zetta and Z-Infra currently needing extension to bridge the sim-to-real gap.
Abstract
from arXiv · showhide
Embodied agents are increasingly used to close the gap left by end-to-end policy models. Yet the agentic path has not realized closed-loop learning in physical execution: existing harnesses remain largely open-loop, following fixed skills during rollout and reflecting only after an episode completes. Such post-hoc reflection cannot govern execution as it unfolds, because physical interaction requires decisions to track rapidly changing robot-environment states at a frequency beyond today's large agentic models. We present Zetta, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online while keeping the base policy frozen. Through three timescale-separated loops, Zetta provides action-frequency governance, rollout-level critic-recovery proposal, and validation-gated skill updates. Together with Z-Infra, a rollout infrastructure decoupling agent logic from heterogeneous execution resources, Zetta achieves state-of-the-art success on LIBERO-Pro and RoboCasa under our current rollout budget, reaching 90.8% and 93.6%, with an 11.1x inference speedup; success continues to scale with self-exploration experience; learned skills transfer zero-shot, and clear robotic "Aha Moments" emerge. These results show that closed-loop harness self-evolution opens a scaling path for reliable physical intelligence.
1. Introduction
Zetta addresses the gap between open-loop embodied-agent execution and the high-frequency state tracking required by physical interaction. It evolves code-based critics and recovery skills online, supported by three coordinated loops and infrastructure for scalable rollouts.
- Motivation: Existing embodied-agent harnesses remain largely open-loop, following fixed skills during execution and reflecting only after episodes end.They do not continuously condition decisions on evolving robot-environment states.
- Motivation: Millisecond-level physical interaction exceeds the decision frequency of large agentic models, limiting post-hoc reflection to completed-episode diagnosis.Episode-level reflection cannot govern execution while it unfolds.
- Zetta: Zetta evolves code-based runtime critics and recovery skills online while keeping base policy models frozen.The harness governs execution and triggers intervention according to changing action and environment dynamics.
- Zetta: Three loops provide action-frequency critic governance, rollout-batch candidate proposal, and validation-gated skill updates.The loops operate at action, rollout-batch, and iteration timescales.
- Infrastructure: Z-Infra decouples agent logic from heterogeneous execution resources through worker pools, batched inference, model partitioning, and asynchronous scheduling.This allows the same agent to scale across machines, accelerators, models, and environments without hardware-specific logic.
- Results: 90.8% on LIBERO-Pro and 93.6% on RoboCasa are achieved under the current rollout budget, with success continuing to improve across evolution rounds.The reported results include state-of-the-art task success and scaling with additional rollout experience.
2. Enabling Deployment-Time Evolution of Embodied Agents
Zetta enables deployment-time evolution by keeping the action policy and orchestrator fixed while evolving a harness that monitors execution, diagnoses failures, repairs runtime behavior, and generalizes validated fixes. Its closed-loop design addresses open-loop execution, brittle repairs, and the scalability limits of manual debugging.
- Offline evolution: The evolutionary pipeline analyzes failed rollouts, extracts causal mechanisms, generates repair patches, and abstracts local fixes into versioned harness updates without changing policy parameters.Offline Evolutionary Agents restructure the orchestrator’s available sensing and action space.
- Framework target: Zetta keeps the action policy and Orchestrator Agent invariant while evolving harness components that govern execution and improve expected task success.The harness exposes sensing and intervention through runtime critics, recovery playbooks, and heterogeneous tools.
- Closed-loop governance: High-frequency runtime critics monitor physical state and trigger interventions when execution deviates from the nominal distribution, converting frozen policies into a closed-loop governance system.This design targets errors such as object slippage and minor collisions that open-loop policies cannot correct in real time.
- Minimal intervention: Hierarchical causal diagnosis applies repairs at the minimal effective layer, avoiding low-level parameter changes that can overfit individual failures and degrade held-out generalization.The diagnosis order proceeds from evaluation through critic, state, planning, recovery, and parameter layers.
- Scalable validation: Automated evolutionary generalization replaces expert-by-expert debugging and validates consolidated harnesses on isolated held-out data to support scalable governance across varied manipulation conditions.The framework is designed to address the cost and limited scalability of manually patching long-tail failures.
- Failure diagnosis: Earliest Observable Divergence and the First Missing Milestone localize failures by identifying when state distributions depart from healthy references and which semantic milestone first went unobserved.These signals support mechanism-level clustering and focus diagnosis on the transition surrounding the first missing milestone.
3. Z-Infra: Embodied Agent Rollout Infrastructure
Z-Infra provides a unified rollout interface that decouples embodied-agent logic from heterogeneous execution resources, supporting scalable and dynamic parallel rollouts. Its layered architecture manages sessions, environments, model inference, routing, scheduling, and resource sharing across backend types.
- Purpose: Z-Infra executes large-scale parallel rollouts across heterogeneous compute resources while exposing a unified interface to upper-layer agent logic.It serves as the execution backbone for agent-environment interaction and self-evolution.
- Infrastructure Challenges: Unpredictable tool usage and irregular session lifecycles make agentic rollouts dynamically heterogeneous, so fixed resource pools or scheduling policies are insufficient.A timestep may involve policy inference alone or a chain of perception, planning, and primitive operations.
- Decoupled Abstraction Layers: A virtual rollout interface decouples agent logic from hardware management by independently handling worker allocation, request routing, batch formation, and hardware mapping.Agents specify what to execute without managing where or how execution occurs.
- System Architecture: The three-layer architecture assigns coordination to the Control Plane, simulation lifecycle and stepping to Environment Workers, and batched model serving to Rollout Workers.The Control Plane routes requests according to operation type and resource requirements, while worker health monitoring supports session recovery after failures.
- Session Management: Session-based lifecycle management isolates long-lived agent-environment interactions and supports admission control through resource accounting, leases, and backpressure.Sessions progress through CREATE → RUNNING → TERMINATED, and saturated workers reject new requests to prevent cascading overload.
- Environment Support: Environment onboarding uses four declarative primitives—init, reset, step, and obs_schema—plus normalization into a canonical observation format.This accommodates environment families with different APIs and execution models behind the same session-level interaction model.
- Resource Sharing: MuJoCo resource-sharing groups amortize model compilation and rendering setup across compatible sessions while assigning isolated slots and reusable rendering resources.Each group compiles the environment once into a reusable ModelTemplate and manages per-slot execution resources.
4. Experiments
Zetta’s cumulative critic–recovery evolution scales execution reliability without changing the frozen policy, while identifying physical bottlenecks that support transfer across tasks and benchmarks.
- Aha Moments: “Aha” moments arise when evolution identifies a decisive physical state variable rather than applying symptom-level repairs.Examples include grasp retention, approach geometry, EEF re-alignment, and centered contact; resolving them produces discontinuous gains.
- Cumulative Scaling: 31.0%→92.5% on Goal-T and 38.0%→89.0% on Goal-S show cumulative scaling without increasing model capacity or fine-tuning weights.Successive reflection rounds repair recurring approach, grasp-retention, and task-relation failures.
- Zero-Shot Transfer: Zero-shot transfer improves Goal-S3 from 9/20 to 20/20 using mechanisms discovered on Goal-T8.The transferred capabilities are pre-grasp staging, grasp-retention criticism, and failure-gated retry, operating on task-independent physical variables.
- Cumulative Scaling: 20.00 percentage points of RoboCasa macro-average improvement accumulate across four global reflection-and-repair rounds with the VLA frozen.The resulting macro-average increases from 73.56% to 93.56% across 18 tasks.
- Zero-Shot Transfer: 64%→84% macro-average success transfers PnP-Stove mechanisms to PnP-Sink, PnP-Cabinet, and PnP-Toaster without additional training or evolution loops.The learned stack includes pre-grasp alignment, re-grasp after grasp loss, and stable placement.
- Case Studies: A runtime critic can interrupt a nominal trajectory after grasp loss, trigger controlled re-approach and re-grasp, and resume execution within one episode.This case demonstrates repeated critic–recovery intervention during physical execution.
5. Related Work
Related work follows two paths—scaling policy models and using language-model agents around policies—but embodied systems still struggle to convert execution feedback into closed-loop improvement. Physical interaction and heterogeneous rollout workloads impose requirements that existing episodic agents and standard rollout infrastructure do not fully address.
- Two paths: Embodied learning pursues end-to-end policy scaling and language-model orchestration around policies, while the deployment gap remains.Limited embodied data and shifting physical distributions challenge reliable real-world execution.
- Closed-loop learning: Existing embodied agents are largely episodic, rarely converting execution traces into governed long-term improvements.The proposed direction turns deployment feedback into reusable critics, recoveries, and future rollouts.
- Embodied difficulty: Physical interaction requires decisions coupled to changing robot and world states, but current embodied agents need frequent external critics, tools, verifiers, and memories.Continuous-world operation makes early errors harder to recover from than in digital domains.
- Design gap: Existing systems usually improve individual stack layers, whereas this design jointly targets runtime critics, recovery traces, reusable skills, and high-throughput execution.Reflection is grounded in runtime critics and recovery actions, then repeatedly evaluated through rollout infrastructure.
- Infrastructure gap: Embodied rollout throughput is constrained by heterogeneous model serving and coordination across CPU, GPU, robot, and cloud resources.Workloads include policy inference, perception, tools, memory, simulators, resets, safety checks, and critics.
6. Conclusion
Zetta closes the gap between static embodied-agent harnesses and the high-frequency governance required by physical execution through online harness self-evolution. Experiments show scalable task success and cross-task transfer, while extending the system to real robots remains future work.
- Zetta: Zetta evolves code-based runtime critics and recovery skills online while keeping the base policy model frozen.Three coordinated loops operate at action, rollout-batch, and iteration timescales.
- Z-Infra: Z-Infra decouples agent logic from heterogeneous execution resources to support scalable rollout generation and execution.It is presented as infrastructure designed specifically for self-evolving embodied agents.
- Results: Experiments on LIBERO and RoboCasa show self-evolution scaling task success toward the frozen policy’s capability ceiling and transferring across tasks.These results support harness self-evolution as a path toward reliable embodied intelligence.
- Scope: Real-robot deployment is future work, including massively parallel rollout collection and integration of real-robot environments as first-class Z-Infra workers.The planned extension explicitly targets bridging the sim-to-real gap.
A. RoboCasa Atomic Task Mapping
This appendix maps compact RoboCasa identifiers to official task names for the complete Atomic-Seen split.
- Mapping: The RoboCasa mapping covers 18 tasks in the complete Atomic-Seen split.Table 4 provides the correspondence between compact identifiers and official task names.
B. LIBERO-Pro Task Mapping
This appendix documents the task identifiers and perturbation settings used for LIBERO-Pro evaluation. The mappings distinguish instruction redirection from object-position swaps while retaining shared task indices.
- Task names: The listed identifiers include T10–T18 tasks such as OpenCabinet, CloseFridge, and TurnOffStove.These names are part of the appendix’s compact-to-official task mapping.
- Benchmark suites: LIBERO-Pro evaluates LIBERO-Goal and LIBERO-10, with each suite containing 10 tasks indexed as Task 0–9.LIBERO-10 is denoted as “Long” for brevity.
- Perturbations: “T” denotes instruction redirection to another valid target or goal, while “S” denotes swapping relevant object positions with the instruction fixed.The two settings test different distribution shifts.
- Identifier mapping: Table 5 maps compact task identifiers to underlying LIBERO instructions, using the same task indices for both T and S settings.For T, the evaluated instruction can differ because of instruction redirection.
C. Case Study: Critic-Guided Recovery for PnP-Stove
The PnP-Stove case study shows how a frozen VLA is governed by a runtime critic that proposes bounded recoveries during execution. The resulting skill combines reusable object-relative recovery behaviors with orchestrator-approved interventions and failure records.
- Runtime critic: The frozen VLA executes the nominal task while a lightweight critic monitors progress and grasp state, producing structured proposals rather than actions or success declarations.Monitored signals include grasp predicates, finger contact, object–gripper drift, transport progress, placement, and post-release separation.
- Recovery skills: Three reusable capabilities are added: object-relative pregrasp, bounded regrasp after grasp failure, and placement recovery with support detection and safe retreat.Failed grasps are released, restaged, and replaced by proposals different from previously failed grasps.
- Execution loop: The critic-guided loop executes VLA or recovery action chunks, collects state windows, and checks the official task predicate during the active episode.The critic output is a proposal, while the algorithm preserves nominal VLA execution when no intervention is required.
- Execution loop: The Orchestrator approves every intervention and any return to nominal execution, with unsuccessful episodes ending in a bounded failure record.This approval gate constrains both recovery actions and transitions back to the frozen policy.
- Transfer: The evolved skill transfers across related pick-and-place tasks by using object-relative geometry and generic physical predicates instead of stove-specific templates or replayed trajectories.Recovery logic is reused by rebinding the live object and target receptacle.
D. Case Study: Critic-Guided Recovery for Libero-Pro Goal-T2
The Libero-Pro Goal-T2 case study applies critic-guided recovery to a frozen VLA while monitoring realized motion, contact, grasp retention, and target-relative transport. Its reusable recoveries support transfer across related pick-and-place tasks through object-relative and generic physical predicates.
- Runtime critic: The frozen VLA executes the nominal policy while a lightweight critic monitors task progress and physical interaction state, producing structured proposals rather than directly acting.The monitored state includes end-effector motion, gripper aperture, contact, object–gripper drift, grasp retention, transport, and the official task predicate.
- Recovery skills: The evolved skill adds collision-safe pregrasp, retained-object monitoring with bounded regrasp, and placement recovery that aligns, lowers, releases, and retreats.The retained-object critic distinguishes successful transport from motion with an empty or slipping gripper.
- Execution loop: The active-episode algorithm executes VLA or recovery chunks, collects state windows, and checks the official task predicate while preserving nominal VLA execution when appropriate.Unsafe motion can be frozen before opening the gripper and restaging.
- Execution loop: The Orchestrator approves every intervention and return to nominal execution, while unsuccessful attempts produce a bounded failure record.The same approval structure governs recovery decisions throughout the skill.
- Transfer: The skill transfers across related Libero-Pro pick-and-place tasks because its recoveries use object-relative geometry, realized motion, and generic contact predicates rather than task-specific templates or replayed trajectories.Transfer occurs by rebinding the live object and target receptacle.