Source-linked AI summary
ENPIRE: Agentic Robot Policy Self-Improvement in the Real World
Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian "Max" Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi "Jim" Fan, Yuke Zhu, Guanya Shi
TL;DR
Real-world dexterous manipulation still depends heavily on human supervision, while coding-agent research has largely remained in digital environments. ENPIRE creates a two-stage physical autoresearch loop and achieves high success rates across multiple dexterous manipulation tasks.
Problem
Real-world dexterous policy learning remains dependent on human supervision, while coding agents lack interfaces for closed-loop physical hypothesis testing and scalable robot-fleet experimentation.
Method
ENPIRE uses human-guided environment construction followed by autonomous policy improvement through reusable interfaces for safety, reset, verification, and real-world feedback.
Results
ENPIRE hill-climbs success rates to high levels across multiple dexterous manipulation tasks.
Takeaways & Limitations
ENPIRE provides a framework for scalable physical autoresearch, including asynchronous hypothesis testing across robot fleets and knowledge transfer to similar novel tasks.
Abstract
from arXiv · showhide
Achieving dexterous robotic manipulation in the real world heavily relies on human supervision and algorithm engineering, which becomes a central bottleneck in the pursuit of general physical intelligence. Although emerging coding agents can generate code to automate algorithm search, their successes remain largely confined in digital environments. We conjecture that the missing abstraction to automate robotics research is a repeatable feedback loop for real-world policy improvement: reset the scene, execute a policy, verify the outcome, and refine the next iteration. To bridge this gap, we introduce ENPIRE, a harness framework for coding agents that instantiates this physical feedback routine with four core modules: an Environment module (EN) for automatic reset and verification, a Policy Improvement module (PI) that launches policy refinement, a Rollout module (R) to evaluate policies with one or multiple physical robots operating in parallel, and an Evolution module (E) in which coding agents analyze logs, consult literature, improve training infrastructure and algorithm code to address failure modes. This closed-loop system transforms real-world manipulation learning into a controllable optimization procedure, minimizing human effort while allowing fair ablations across training recipe and agent variants. Powered by ENPIRE, frontier coding agents can autonomously train a policy to achieve a 99% success rate on challenging, dexterous manipulation tasks, such as organizing a pin box, fastening a zip tie, and tool use, a process that further accelerates when we dispatch an agent team on a robot fleet. Our results suggest a practical and scalable path toward deploying coding agents to autonomously advancing robotics in the physical world.
1. Introduction
ENPIRE addresses the human-supervision bottleneck in dexterous manipulation by formalizing physical autoresearch as a closed-loop process for constructing environment feedback and autonomously improving policies. It also scales policy improvement across parallel robots through asynchronous hypothesis selection based on success rates.
- Human involvement in data collection, reset-based evaluation, and algorithm adjustment limits the scalability and speed of real-world dexterous policy learning.
- Coding agents lack physical-world interfaces for automated policy deployment, evaluation, scene resetting, and closed-loop hypothesis testing.
- ENPIRE formalizes physical autoresearch through a two-step process: human-guided construction and offline verification of environment feedback, followed by fully autonomous online policy improvement.
- ENPIRE uses autonomous environment interfaces to make dexterous manipulation skill acquisition a scalable physical autoresearch procedure.
- A decentralized agent team tests training recipes asynchronously across multiple robots, sharing or abandoning hypotheses according to average success rates.The accumulated knowledge can transfer to similar novel tasks.
2. Method
ENPIRE structures physical autoresearch as environment construction from human feedback followed by automatic policy improvement from real-world feedback. Its immutable interfaces combine safe, resettable environments, automated verification, robot rollouts, and coding-agent-driven training evolution.
- Environment (EN): Environment construction provides task-specific safety constraints, real-time automated verification, and automatic reset for reliable physical interaction and rapid iteration.Safety-limit violations trigger immediate task failure and an automated reset.
- Environment (EN): Automated verification synthesizes binary rewards from brief success and failure demonstrations using video and proprioception recordings.Inference latency is optimized to under 150ms.
- Rollout (R): The Rollout module combines visual-proprioception inputs, robot-controller actions, and rewards through immutable Gym APIs that expose feedback and debugging information.These interfaces support automated policy improvement after the environment modules are constructed.
- Policy Improvement (PI): The Policy Improvement module lets coding agents review literature, formulate hypotheses, and modify behavior-cloning or reinforcement-learning code using real-world verification results.Agents optimize task success rates through autonomous experimentation in a streamlined training codebase.
- Evolution (E): Parallel Evolution deploys N agents on N robots to test hypotheses asynchronously and share successful training recipes through autonomous Git collaboration.Each agent branches from the same baseline policy-training codebase.
- Resource utilization: Mean Robot Utilization measures the fraction of research wall-clock time spent executing experiments, while GPU utilization measures the fraction spent using the GPU.A resource-saturated agent would drive both metrics toward 1.
3. Experiment
ENPIRE enables autonomous policy improvement across heuristic, end-to-end, multi-robot, transferable, and VLA-based manipulation experiments. Results show successful learning in simulation and real-world tasks, faster improvement with robot scaling, robustness through domain randomization, and transfer of learned strategies.
- Experimental Setup: ENPIRE supports autonomous policy learning after environment construction, with success-rate convergence scaling with robot and token resources.Experiments use diverse policy paradigms on a bimanual 6-DoF YAM robot.
- Experimental Setup: The experiments cover Push-T, pin insertion, GPU insertion, and ziptie-cutting, requiring precise reactive control from perceptual feedback.Success measures one-rollout completion with eight retries, capturing precision and in-context recovery after observed failures.
- Heuristic Learning: 95% success rate is achieved by Claude Code and Codex on Gym-PushT simulation within approximately 2 hours, while Kimi Code takes twice the time.All coding agents solve Push-T in simulation, but three agents fail in the real-world environment because physical conditions are nondeterministic and time-varying.
- Resource Scaling: Scaling from one to eight agents reduces Push-T time to reach a 1.0 normalized score from roughly five to two hours.For pin insertion, the same scaling reduces time to reach a near-perfect success rate from more than 1.5 hours to approximately 40 minutes.
- Robustness and Transfer: Multiagent code-based policies apply domain randomization during reset, producing broader spatial-configuration variation for GPU insertion than prior work.The broader variation is intended to enforce stronger policy robustness.
- Robustness and Transfer: Agents transfer documented training-recipe insights from pin insertion to GPU insertion and integrate VLAs with procedural tools for long-horizon manipulation.Motion planning and detection tools improve GR00T VLA performance in RoboCasa365, and the transferred hover-grasp-cut strategy succeeds on real-world zip-tie cutting.
4. Limitations
ENPIRE’s coding agents underutilize robot resources during language-model, coding, debugging, and coordination activities. Larger fleets accelerate policy improvement but reduce token efficiency because token usage and successful-policy budgets grow faster than fleet size while wall-clock time decreases less rapidly.
- Resource utilization: Coding agents underutilize robots while reading logs, writing code, debugging, or waiting for the language-model backbone.These activities leave robot resources unused during substantial portions of the improvement loop.
- Resource utilization: As fleet size increases, MRU decreases while GPU active utilization increases, and agent teams spend more time summarizing peer branches than operating robots.The changing activity mix makes parallel fleets less focused on direct robot operation.
- Scaling costs: Token cost grows super-linearly with fleet size, while the total budget for a successful policy rises faster than the corresponding reduction in wall-clock time.MTU stays near the linear projection through four agents but rises sharply at eight agents.
- Scaling costs: Larger fleets reach successful policies sooner but trade away token efficiency for faster policy improvement.The tradeoff reflects rapidly increasing token usage as fleet size expands.
5. Related Work
Related work uses executable code as a verifiable action and increasingly improves agents through execution-feedback loops. ENPIRE distinguishes itself by closing this loop directly on real robots, whereas prior systems generally iterate in simulation, digital environments, or human-mediated laboratory settings.
- Code as an action abstraction: Executable code became a robotics action abstraction that composes perception and skill APIs into task plans while enabling runtime-verifiable steps.Code-as-Policies and ProgPrompt introduced this formulation for robotics, and Wang et al. argue that code is stronger than language because each step is runtime-verifiable.
- Feedback-driven improvement: Subsequent systems extended single-shot code generation into multi-turn improvement through execution traces, self-critique, iterative repair, inference-time search, and learned tool invocation.Within robotics, early systems typically exposed high-level, human-engineered APIs and left agents to decompose tasks rather than synthesize low-level control.
- Trial medium and cost: Prior self-improvement loops usually rely on cheap execution substrates, adding skills, revising rewards, or generating simulation tasks before real-robot deployment or data collection.Eureka, for example, closes reward revision through thousands of Isaac Gym rollouts per minute, while other systems iterate in simulation and deploy only after revision stops.
- Hardware-in-the-loop iteration: ENPIRE instead runs the improvement loop directly on hardware, where robot-access budget—not compute—is the binding resource.The related work describes real-robot execution in prior systems as a sim-to-real or data target rather than the iteration medium.
- Automating the research loop: Research-loop automation spans autonomous real laboratory hardware, end-to-end digital environments, lab automation, and human-executed wet-lab validation.The paper frames this final body of work by the medium in which experiments are launched, with evaluation still emphasizing digital settings.
Appendices … B. Robot System Setup
The appendices detail ENPIRE’s autonomous reset and reward-design mechanisms, showing how coding agents compose perception, planning, manipulation, verification, and retry capabilities for real-world tasks. They also provide implementation examples and links to supplementary videos.
- Appendices: The appendices include supplementary videos documenting the ENPIRE system and experiments.The videos are available at research.nvidia.com/labs/gear/enpire.
- A.1. Autonomous Reset: Given a task specification, coding agents synthesize long-horizon reset functions by composing perception and planning APIs with real-time RGBD and proprioceptive inputs.The APIs include SAM3, BundleSDF, and cuRobo for detection, pose tracking, and collision-free trajectory optimization.
- A.1. Autonomous Reset: The same research workflow instructs agents to leverage knowledge from origin branches, contribute changes, and collaborate with other agents.The workflow requires actively pushing the agent’s branch and pulling from origin.
- A.1. Autonomous Reset: The reset pipeline combines semantic segmentation, 3D bounding boxes, manipulation skills, failure detection, and retries to handle real-world perturbations.Supported skills include pick-and-place, object reorientation, localization, and bimanual handover.
- A.2. Autoresearch-driven Reward Design: Autoresearch automates reward-function design after humans provide representative success and failure demonstrations plus requirements for precision, recall, and inference latency.The procedure is illustrated by designing a visual reward that detects whether a zip tie is fastened.
- A.2. Autoresearch-driven Reward Design: For pin insertion, the environment awards 1 when the pin is fully inserted and held in place, otherwise 0, after a prescribed pickup, placement, and hovering sequence.The reward design can combine visual alignment, insertion depth, and contact-force signals; dual camera views help verify zip-tie fastening geometrically.
B.1. Fleet Architecture · B.2. Station Hardware · B.3. Low-Level Control
ENPIRE uses eight decentralized, hardware-identical bimanual robot stations coordinated through Git, with each station independently running its agent, hardware, and computation. Each station combines camera-based policy inference with 100 Hz low-level control, including force-limited compliant grippers for robust and safe grasping.
- B.1. Fleet Architecture: Eight decentralized stations each own their robot arms, cameras, compute, and coding agent, while Git synchronizes code, configurations, tools, and results across the fleet.Stations run, fail, and recover independently, and shared Git history records what each station has tried and learned.
- B.1. Fleet Architecture: Each station’s local FastAPI server exposes endpoints for starting rollouts, allocating fresh rollout buffers, and returning the arms home.Separate rollout directories let agents attribute outcomes to specific hypotheses without mixing experimental data.
- B.2. Station Hardware: All 8 stations are hardware-identical, each pairing two YAM arms, cameras, and one workstation running the server, policy inference, and coding agent.The workstation provides the station-local execution environment rather than relying on shared cluster or off-station compute.
- B.2. Station Hardware: Each station uses two 6-DoF arms with 1-DoF parallel-jaw grippers, totaling fourteen actuated joints driven by brushless actuators over CAN.The arm joints use PD control with gravity compensation, while grippers use force-limited actuation.
- B.2. Station Hardware: Perception normally uses one top-down and two wrist-mounted Intel RealSense D405 cameras, with an additional side-mounted D435i for one task.The wrist cameras provide close-range views of grasped objects and contact regions, while the side camera supplies a wider third-person view.
- B.3. Low-Level Control: Policy actions are inferred at 30 Hz and tracked by low-level joint controllers running at 100 Hz over the CAN bus.The six arm joints use PD control with gravity compensation to track targets while offloading static loads through feedforward gravity terms.
- B.3. Low-Level Control: The 1-DoF gripper uses torque-limiting compliant grasping, applying a bounded commanded grip force instead of closing to a rigid target width.This accommodates object-pose and size variation and limits exerted force, supporting robust and safe grasping under perception and placement error.
B.4. Per-Task Configuration · B.5. Real-World RL System Integration · B.6. Idea Tree for Pin Insertion
These sections specify shared camera and control configurations, a distributed RL infrastructure for agentic algorithm research, and an idea-tree analysis of pin-insertion improvements. The pin-insertion run shows large gains from BC regularization followed by smaller refinements as success approaches 100%.
- B.4. Per-Task Configuration: All four tasks share station hardware and a 30 Hz policy loop, differing only in minor camera and control details.Each task uses one top-down camera and two wrist cameras.
- B.4. Per-Task Configuration: The four tasks use three RealSense D405 cameras, while GPU insertion adds a side-mounted RealSense D435i for a wider slot view.The added camera provides a third-person view; other tasks retain the three-camera setup.
- B.5. Real-World RL System Integration: PLD-RL provides a controlled sandbox for coding-agent algorithm autoresearch with online data collection, using an asynchronous SERL-inspired design.The infrastructure is implemented as a three-tier distributed system separating robot interaction, policy learning, and inference.
- B.5. Real-World RL System Integration: A disk-based contract transfers finalized episodes as per-step observations, synchronized camera streams, and action-source labels between deployment and learning.The deployment layer writes rollout data, while DiskBufferIngestor periodically polls and parses newly finalized episodes.
- B.5. Real-World RL System Integration: RL-generated transitions enter the online replay buffer, whereas human and manual transitions enter a demonstration buffer mixed into each training batch.This RLPD-style routing supports flexible mixing of data from different sources.
- B.6. Idea Tree for Pin Insertion: The pin-insertion idea tree pairs team-explored ideas with a hill-climbing curve, connecting related ideas horizontally and assigning each new lane to an iteration.The analysis visualizes how successive ideas contributed to policy improvement.
- B.6. Idea Tree for Pin Insertion: BC regularization contributed +10.8 pp, while batch-size tuning and controller compensation added +0.9 pp and +1.3 pp as success approached 100%.The later ideas are described as smaller, incremental refinements compared with BC regularization.
C. Physical Autoresearch: Ablation Studies … C.4. Model and Harness Comparison
ENPIRE’s physical autoresearch ablations use a simplified Push-T environment to compare token usage, visual-access configurations, and model–harness combinations. Native vision reaches success first, while Codex solves the task fastest and Codex with the Opus API is least efficient.
- C.1. Experimental Setup: The pusht-simple environment preserves Push-T’s visual-servoing and contact-planning structure while reducing episode length and setup variability for repeated ablations.Its relaxed success boundary further supports controlled comparisons across model, harness, and visual-grounding conditions.
- C.1. Experimental Setup: The simplified task requires rotating the T block approximately 180∘ and placing it inside an enlarged success region.This operationalizes the controlled Push-T ablation task.
- C.2. Token Utilization: Token usage is broken down into input, output, and cached categories to distinguish context ingestion, code and plan generation, and interaction-history reuse.The categories differ in cost, latency, and the agent activity they represent.
- C.3. Native Vision Capability: The vision ablation tests native image inspection against callable image understanding and text-only analysis for auto-search hill-climbing on simplified Push-T.The Codex harness supports native visual understanding, whereas many open-source harnesses expose only text input.
- C.3. Native Vision Capability: Codex with native vision reaches success first, while the no-vision baseline succeeds before the function-call vision baseline.The results suggest logging signals can provide useful task-state information, whereas repeated image calls add solving overhead.
- C.4. Model and Harness Comparison: The model–harness comparison evaluates Codex with GPT-5.5, Claude with Opus 4.7, and Codex with Opus 4.7.These configurations measure end-to-end autoresearch performance across both model and harness choices.
- C.4. Model and Harness Comparison: Codex solves the task fastest, while the Codex harness using the Opus API is least efficient among the tested configurations.The comparison therefore separates the effects of the coding-agent harness and the underlying model API.
D. Simulation Benchmark Result · D.1. RoboCasa Simulation Interface and API Surface
RoboCasa365 provides the simulation evaluation setting for generated vision-language control scripts through a modular API namespace. The canonical runtime supports perception and motion while restricting privileged-state access, retries, resets, and oracle queries.
- D.1. RoboCasa Simulation Interface and API Surface: RoboCasa365 evaluates generated vision-language control scripts through modular and end-to-end tools exposed in a single Python namespace.The stack is built on RoboCasa simulation and encapsulates its tools as APIs.
- D.1. RoboCasa Simulation Interface and API Surface: Canonical scripts remove APIs that leak privileged state or enable repeated retries, while reset and task-information APIs remain gated.The interface distinguishes canonical script execution from non-canonical CLI exposure.
- D.1. RoboCasa Simulation Interface and API Surface: Oracle target queries are exposed only when the Oracle interface is explicitly enabled, preventing routine access to ground-truth target poses.The API surface marks get_oracle_targets as O-only rather than canonical-runtime available.
- D.1. RoboCasa Simulation Interface and API Surface: Camera APIs provide aligned RGB-D observations with intrinsics and extrinsics, while debug markers are reserved for development visualization.Resolution-specific calls preserve alignment among RGB, depth, and intrinsics; unsuffixed aliases remain for backward compatibility.
- D.1. RoboCasa Simulation Interface and API Surface: Motion APIs support cuRobo and Pyroki: cuRobo is the default collision-aware free-space planner, whereas Pyroki provides lightweight SE(3) interpolation with inverse kinematics.Joint-execution variants are used when following planned joint paths matters near joint limits or tight obstacles.
- D.1. RoboCasa Simulation Interface and API Surface: Perception APIs provide text-conditioned detection, segmentation, one-shot detection, grasp-pose sampling, and vision-language queries for grounding instructions visually.These calls avoid simulator ground-truth access while connecting language instructions to visual observations.
- D.1. RoboCasa Simulation Interface and API Surface: Collision avoidance is enabled by default for ordinary free-space motion, while collision disabling is reserved for intentional contact-rich actions or collision-model artifacts.Planner APIs maintain cuRobo collision geometry from the simulated scene.
D.2. RoboCasa Evaluation Protocol
RoboCasa evaluation executes each generated script once per episode in a canonical harness that records logs and videos before querying the native success function. Evaluations use fixed matched episodes for fair comparison, while diagnostics expose perception failures and improvements from resolution and prompt changes.
- Evaluation harness: Each generated script runs once per RoboCasa episode through a canonical harness that injects restricted tools, records videos and logs, and queries native success.The reported score is binary task success from the environment’s final success function.
- Evaluation harness: 40-episode evaluations use a static list of seed, layout, and style triplets generated deterministically with generator seed 42.Smaller diagnostic subsets draw rows from the same saved evaluation file rather than regenerating scenes.
- Fair comparison: GR00T and generated vision scripts are scored on matched episodes with identical tasks, seeds, initial states, cameras, language prompts, and success predicates.Neither method receives oracle target poses or simulator object states; GR00T rolls out a learned policy, whereas scripts compose public perception, planning, and manipulation APIs.
- Discovered policy: The discovered RoboCasa strategy combines detection and motion planning to move the gripper above the target before grasping, and transfers to real-world scissors and zip-tie tasks.This strategy is shown qualitatively in project-website demonstrations.
- Result analysis: 17/20 candidate-prompt detections are correct at higher resolutions, versus 14/20 for the original prompt; candidate prompts improve target detection but then plateau.The diagnostic varies top-camera resolution and prompt wording for counter-to-cabinet scenes, identifying SAM3 masking as a perception bottleneck.