PaperlayerView exact revision on arXiv

Source-linked AI summary

Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

arXiv:2610.12369v1cs.ROcs.CV

TL;DR

Robot policies often keep a model in the control loop, while existing code-based approaches may generate code per episode or rely on externally supplied state. COAP measures state and makes decisions in reusable code; on RoboDojo it reaches 70.24% success across 42 bimanual tasks. The authors present its explicit, extensible code as suitable for recursive self-improvement and trajectory generation for other policies.

  • Problem

    Robot policies commonly keep state representation or action selection inside model weights or model context, while existing code-based policies may regenerate code per episode or use externally supplied state.

    Source: §1 · pp. 2–3

  • Method

    COAP uses code to measure and track robot and environment state, compute actions from explicit state, and reuse a shared library across episodes without a runtime model.

    Source: §1 · pp. 2–3

  • Results

    COAP leads all five RoboDojo dimensions and reaches 70.24% success across 42 bimanual tasks.

    Source: §6 · p. 18

  • Takeaways & Limitations

    The authors present explicit, controllable code as suitable for recursive self-improvement and as a source of successful trajectories for training VLAs and Agent Harnesses.

    Source: §1 · p. 3, §5.5 · p. 18

    All results are in simulation, and COAP relies on benchmark configuration and assets that would need to be measured or retuned on a real robot.

    Source: §5.4 · p. 17

Abstract

from arXiv · show

Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo's 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.

1. Introduction

The paper frames a robot and its environment as an Embodied Turing Machine and proposes COAP, which measures state and makes decisions in reusable code. It reports 70.24% success on RoboDojo’s 42 bimanual tasks and presents COAP as a medium for recursive self-improvement.

1. Introduction

  • COAP models robot and environment state as an Embodied Turing Machine’s tape, with the policy as fixed rules that read and write it.Robot state comes from proprioception, while object poses can be recovered from RGB images using measured object geometry.Source: §1 · pp. 2–3
  • Unlike VLAs and Agent Harnesses, COAP stores state in code variables and computes actions through explicit code logic, with no model required at runtime.Existing code-based policies either generate code online per episode or obtain state from a learned model or simulator oracle.Source: §1 · pp. 2–3, Figure 2 · p. 3
  • One COAP program runs across a task’s episodes, and 83% of the code a new task runs comes from the shared library.The remaining logic is task-specific; sufficiently robust code does not need a VLM or VLA in the loop.Source: §1 · pp. 3–4
  • COAP’s explicit state, controllable execution, and extensible shared library are presented as advantages that make code suitable for recursive self-improvement.Coding agents develop the library offline, and each code change is explicit and controllable.Source: §1 · pp. 3–4
  • 70.24% success on RoboDojo’s 42 bimanual tasks, 38.9% above the state-of-the-art method, was achieved by the resulting COAP policy.The coding agent developed the library in a closed loop without human intervention or benchmark test-set use.Source: §1 · p. 3
  • Because the policy applies across episodes, COAP can also generate successful trajectories as training data for VLAs and Agent Harnesses.The paper describes this as an efficient data-engine use of COAP.Source: §1 · pp. 3–4

2. Related Work

Related work is organized around where robot policies store state and how they derive actions, contrasting implicit model representations and runtime model calls with explicit code. The paper also distinguishes COAP’s sensor-based state measurement from code-based approaches that rely on external state sources.

2. Related Work

  • VLAs encode state implicitly in weights, while Agent Harnesses place text memory in a model’s context; neither makes the full action derivation inspectable and controllable.The paper notes that VLA improvements require new data and training, while harness models are queried repeatedly during execution.Source: §2 · p. 4
  • In runtime VLM harnesses, repeated model calls add randomness and cost, whereas COAP computes actions from explicit state without online model calls.The paper characterizes COAP’s online decisions as reproducible and cheap because they do not require model calls.Source: §2 · p. 4
  • Earlier code-as-policy systems generate programs at runtime or use state from learned perception models, while RHO and EmbodiedSWE use external perception or simulator-oracle state.COAP instead tracks state with its own code, allowing coding agents to revise measurement code when measurements are wrong.Source: §2 · pp. 4–5
  • Unlike software-agent settings with explicit environment state, robot policies must measure state from sensors; unlike RoboRSI, COAP uses code for decisions during execution.The paper says COAP’s tested behavior is its runtime behavior, with changes tested offline before merging.Source: §2 · p. 5
  • RoboDojo combines memory, precision, open instructions, generalization, and long-horizon manipulation across one shared protocol of 42 tasks.Its memory and precision tasks directly test state retention and measurement.Source: §2 · p. 5

3. Embodied Turing Machine: Code-Only-as-Policy

COAP treats robot and environment state as an embodied tape and code as fixed rules, implementing policy through a shared library. The library measures and updates state, runs task programs across episodes, and is improved offline through coding-agent branches.

3. Embodied Turing Machine: Code-Only-as-Policy

  • COAP implements the Embodied Turing Machine as a shared library whose variables hold state and whose source code supplies the rules.Development improves code offline; at runtime, a fixed task program runs without an agent or VLA.Source: §3 · pp. 5–6, Figure 3 · p. 6

3.1. Reading and writing the state

  • The code maintains environment, robot, and task state; at runtime, the program is fixed and only the state changes across episode layouts.Environment state is measured from images, robot state comes from proprioception, and task code updates task state during execution.Source: §3.1 · pp. 6–7

3.2. Measuring the environment state

  • Perception code measures object position and pose from images using known shapes, and retains occluded objects’ last measured poses.Tasks needing millimetre accuracy re-measure with a wrist camera before the final approach.Source: §3.2 · pp. 7–8

3.3. Sharing one library across tasks that read the same state

  • A shared, layered library lets tasks reuse modules, inherit task programs, or extend functions with default-off options.The library’s lower layers handle manipulation, perception, and motion, while task code specifies subgoals and recovery conditions; default-off extensions preserve behavior for tasks that do not adopt them.Source: §3.3 · pp. 8–9

3.4. Improving the code by backtracking failures to the state

  • Coding agents inspect recorded failures, propose runnable code branches, test them in parallel, and accept changes only when validation success clears a noise margin.Accepted changes become the base for subsequent branches; the acceptance rule targets improvement while a compatibility condition protects tasks not targeted by a diff.Source: §3.4 · p. 9

4. Experiments

On RoboDojo, COAP leads the reported leaderboard comparison and is evaluated across state representation, execution, and library extensibility. Its offline code-development process also supports parallel branching for recursive self-improvement.

4.1. Setup

  • RoboDojo evaluates 42 bimanual tasks across Memory, Precision, Open, Generalization, and Long-Horizon dimensions, using three seeds of 50 layouts per task.COAP receives RGB streams, proprioception, and language instructions, with no depth, segmentation, or model service at test time.Source: §4.1 · p. 9

4.2. Main results

  • 70.24% success rate, 38.9 percentage points above the 31.38% current state-of-the-art, with COAP highest in all five dimensions.It reaches 65.7% on Generalization and 64.1% on Open; its margin is smallest on Long-Horizon tasks.Source: §4.2 · pp. 10–11, Table 2 · p. 10

4.3. Advantages of Code-Only-as-Policy

  • COAP’s comparison with the strongest leaderboard entry in each baseline category is organized around Explicit State, Execution, and Extensibility.Where RoboDojo provides no measurement, the comparison uses numbers reported in prior work.Source: §4.3 · p. 11, Figure 4 · p. 11

4.3.1. Explicit State

  • On Memory tasks, COAP reaches 90%, versus 47% for the Agent Harness and 33% for the VLA, using persistent state variables for measured values and task memory.Its five explicit state categories are robot, object, environment, relation, and task.Source: Figure 5 · p. 11, §4.3.1 · p. 12
  • On Precision tasks, COAP reaches 75%, versus 33% for the Agent Harness and 27% for the VLA; measured-state success is 63%, compared with 83% using oracle state.The program estimates scene geometry before deployment and makes object position, size, and pose explicit measurements.Source: Figure 5 · p. 11, §4.3.1 · p. 12

4.3.2. Execution

  • COAP can backtrack after a failed grasp, remeasure the object, and retry with a new grasp method; it detects problems in 76% of such episodes and retries in 34%.A control step takes a median 0.3 ms on a CPU; an episode uses 0.8–2.4% of a VLA’s compute and less than 0.1% of an Agent Harness’s.Source: Figure 6 · p. 12, §4.3.2 · p. 13

4.3.3. Extensibility

  • 83% of code new tasks run comes from the shared library; COAP reaches 47% average success within 15 hours on new tasks without data or training.The strongest VLA reaches 4.7% on the same tasks after training on demonstrations; default-off additions and inherited functions are designed to avoid changing earlier tasks’ logic.Source: §4.3.3 · pp. 13–14, Figure 7 · p. 14

4.4. Why Code-Only-as-Policy Suits RSI

  • Parallel branching treats alternative implementations of a decision as runnable programs evaluated on the same layouts, then selects the best before moving to the next decision.Development records show that decisions tried with two or more variants found an accepted change in four of five cases; more branches trade increased GPU time for earlier, larger accepted gains.Source: §4.4 · pp. 15–16, Figure 10 · p. 15

5. Discussion

The discussion specifies COAP’s state, analyzes its failure modes and limits, and describes its potential as a data engine for model-based policies. Its scope remains bounded by simulation, offline development costs, and generalization limits.

5.1. What Should the State Contain?

  • The state stores robot, environment, and task progress in physical units, including object relations, hand contents, and current subgoals.Robot state comes from proprioception, environment state from perception, and task code updates task state during execution.Source: §5.1 · p. 16

5.2. Error Analysis of COAP

  • Failed episodes are classified by the smallest change that would have prevented them, spanning perception, contact, incomplete logic, and poor code design.Perception and contact errors reflect state measurement; incomplete logic and poor design reflect code limitations.Source: §5.2 · pp. 16–17

5.3. The Upper Bound of COAP

  • 62% of failures are state-measurement limits and 37% are logic limits, setting COAP’s upper bound through state quality and code robustness.RGB-only estimates and complex-object contact constrain state measurement; robust failure detection, retries, and state-derived values address logic limits.Source: §5.3 · p. 17

5.4. Limitations

  • All results are in simulation, using benchmark assets and configuration; real deployment would require measuring these and retuning contact parameters.Examples include grip widths and carry speeds.Source: §5.4 · p. 17
  • COAP’s online execution is fast, but each new task incurs offline code-development costs for evaluation and revision.The paper contrasts this with using a model such as GPT directly as a policy.Source: §5.4 · p. 17
  • Generalization is limited: success is 54.1% on randomized layouts versus 77.3% on standard layouts, while standard test-layout success is 74.9% versus 76.7% on validation layouts.Programs were developed on validation layouts, not test layouts; randomized layouts add new objects, rooms, and distractors.Source: §5.4 · p. 18

5.5. COAP as an efficient Data Engine for VLAs, Agent-as-Policy and Harness VLA

  • Once a task program works, CPU execution across layouts can produce successful trajectories quickly, with explicit states and decisions for training VLAs or supervising Agent Harness models.The paper presents this as a data-engine use for VLAs, Agent-as-Policy, and Harness VLA.Source: §5.5 · p. 18

6. Conclusion

The paper proposes COAP as code-based policy for an embodied world modeled as a Turing machine, and reports 70.24% success on RoboDojo. It presents explicit state, execution, extensibility, and recursive self-improvement as advantages, while identifying state representation and code logic as limits.

6. Conclusion

  • COAP treats the robot-environment state as an Embodied Turing Machine’s tape and the robot policy as its rules.It uses explicit state and code, improved by coding agents, rather than keeping a model in the control loop.Source: §6 · p. 18
  • On RoboDojo, COAP succeeds in 70.24% of episodes, 38.9% above the state-of-the-art method, and leads all five dimensions.Source: §6 · p. 18
  • The paper identifies COAP’s upper bound as depending on state representation and code logic, and proposes it as a data engine for VLAs and Agent Harnesses.Source: §6 · p. 18

A.1. Execution

This appendix describes COAP’s execution outcomes and decision-time profile, and compares its success across task-length groups.

A.1. Execution

  • The failure-outcome table separates episodes where COAP reattempted, noticed and reported a problem, or hit the step limit.For reattempted episodes, the table also breaks down the failed steps.Source: §A.1 · p. 24, Table 3 · p. 24
  • The policy makes a median of 352 decisions per episode; median calls take 0.3 ms, while about 25 calls account for 99.4% of decision time.Most calls hand over an action already planned; perception and planning at segment starts account for the longer calls.Source: §A.1 · p. 24
  • COAP leads in every task-length group, although its success falls on the longest tasks, as does every policy’s.The strongest Agent Harness loses most on the longest tasks.Source: §A.1 · p. 24

A.2. Extensibility

The library extends to new tasks while preserving existing code paths, and the reported old-task results show no significant change when code was unchanged. New tasks reuse shared modules, while task-specific files and logic remain limited.

Grasp methods

  • A grasp method is distinguished by its approach and contact; for example, a top grasp descends from above even if it leans.The listed methods also include side, handle, rim, edge, and cloth-pinches, with their example objects and task-program counts shown in the table.Source: §A.2 · p. 25, Table 5 · p. 25

New tasks

  • For four tasks without programs when the library was frozen, the plotted baselines are schematic: the VLA line ramps toward 4.7% average success, while the harness improves slowly.Table 6 records first evaluations, first successes, merges, and validation success at each merge; store_tools_in_toolbox counts as zero because it never got a successful program.Source: §A.2 · p. 25, Table 6 · p. 25

Old tasks

  • 78.4% before and 79.9% after for old tasks whose code did not change, with no task changing significantly (p = 0.16).Including tasks whose code was later cleaned up, success was 80.0% before and 81.4% after; the test rules out a large regression, not every change.Source: §A.2 · p. 25

Extension without breaking

  • New tasks can extend the library through calls to existing functions, inheritance, or default-off parameters that leave other callers on their previous code path.sweep_blocks reuses the sliding function added for push_T.Source: §A.2 · p. 26, Figure 14 · p. 26

Layers

  • The shared manipulation, perception, and motion layers comprise about 14k lines, while family and task-program modules serve fewer tasks.The most-used perception and motion modules each serve 100% of the 40 task programs; manipulation serves 78%.Source: §A.2 · p. 26, Table 7 · p. 26

B. Parallel Branching: Details

The frozen library’s overall success rose through sequential merges, with gains also reported on test; branch selection uses validation success and a margin above rerun noise.

Branches and the estimate

  • A branch changes one decision and is compared with its parent on the same seeds and layouts; acceptance uses a four-episode margin, about one to two rerun-noise widths.The gain estimate bootstraps measured effects, treats branches of one decision as independent, and pools effects across tasks; because gains use layouts that selected them, the true gain is somewhat lower.Source: B. Parallel Branching: Details · p. 26

All tasks at once

Merges and test

  • 66.5% to 70.24% overall success across merges; on 40 tasks with programs, standard success was 76.7% on validation and 74.9% on test, and randomized success was 55.2% and 52.8%.Task updates were merged one at a time through default-off parameters, and changes were selected on validation.Source: B. Parallel Branching: Details · p. 27

C. Error Analysis: Details

Failures are classified by the first unrecovered error and the smallest change that would have prevented it; the classifications are judgment-based and checked against official videos.

How a failure is classified

  • Each failed episode is read from the start, and its class is the smallest change that would have prevented the first error that was never recovered.Each failure receives exactly one class.Source: C. Error Analysis: Details · p. 27

Classification caveat

  • Class labels are judgments because traces show what the program measured rather than what happened, so examples were checked against official videos.Fixes are judged relative to the available robot and assume no new sensor or actuator.Source: C. Error Analysis: Details · p. 27

Error classes

  • The error classes are perception error, contact error, incomplete logic, poor code design, and unknown.Examples include an unrecognized randomized look, a slip or drop, an untried action order, a fixed threshold, and no lead in the trace, respectively.Source: Table 8 · p. 27

D. Per-Task Results

Table 9 reports per-task success rates and progress scores for COAP.

Per-task results

E. Case Studies

The case studies illustrate stored state and recovery across episodes and scenes. Execution trajectories show shared library code recurring across tasks.

E. Case Studies

  • In cover_blocks, the program stores cube colours by position; in stack_bowls, it retries a failed grasp at a new wrist angle; one fold_clothes program works in standard and randomized scenes.Source: E. Case Studies · p. 27
  • Table 9 presents COAP per-task results by dimension and marks tasks without a program.Source: Table 9 · p. 28

F. Execution Trajectories

  • Across successful episodes, the same library code grasps, carries, and places objects, while each task program determines their order and targets.Source: F. Execution Trajectories · p. 29

Related papers

Loading 2610.12369v1…