PaperlayerView exact revision on arXiv

Source-linked AI summary

PhysEvo: Astra Can Act, Let It

Wenqing Tian, Zeyu Zhang, Zhaocheng Liu, Fengwei Liu, Qiang Liu, Liang Wang

arXiv:2610.08995v1cs.ROcs.AI

TL;DR

Reliable robot execution is not guaranteed by a model’s demonstrated ability to act. PhysEvo uses a frozen Astra to act, diagnose outcomes, and test revisions to action and diagnostic resources. Retained versions improve benchmark performance, and continued skill revision also yields strong results on real hardware.

  • Problem

    Astra’s demonstrated robot competence does not guarantee reliable execution, raising the question of how already-present capabilities can be used reliably.

    Source: §1 · p. 2

  • Method

    PhysEvo uses one frozen Astra as task agent and meta-agent, revising tools and skills from execution evidence and retaining tested changes.

    Source: §1 · p. 2

  • Results

    Across 42 RoboDojo tasks, retained versions achieve 68.14/100 score and 62.00% success, versus RoboDawn one-shot Astra’s 47.17% success (Guo et al., 2026a).

    Source: §1 · p. 2

  • Takeaways & Limitations

    Physical experience can improve not only the next action but also the interfaces and diagnostic resources available to future actions.

    Source: §1 · p. 2, §6 · p. 11

Abstract

from arXiv · show

Astra can act, yet reliable manipulation depends on the system through which it observes and controls the world. We introduce PhysEvo, a framework for physical recursive self-improvement (RSI) around a single frozen model. A task agent executes robot tasks; a meta-agent uses the resulting trajectories to diagnose failures, revise tools and skills, and test corrections. The meta-agent can also improve its own diagnostic tools, so retained revisions support both later action and later self-improvement. This process develops joint-level control, evidence-seeking observation, and reusable manipulation skills without model-weight updates or a separately trained action policy. Across 42 RoboDojo tasks, held-out-layout evaluation of retained task-specific deployment versions yields a five-dimension average score of 68.14/100 and 62.00% success, compared with 47.17% for RoboDawn's one-shot Astra agent, the strongest published reference in our comparison. On eight manipulation tasks challenging direct Astra, PhysEvo achieves 55.00% success, compared with 1.25% for the direct-Astra reference. Deploying the simulation-evolved harness on AgileX PiPER and continuing skill revision yields 90.60/100 average score and 84.00% success across 25 trials on five real-world tasks. PhysEvo turns the consequences of action into persistent, testable changes to how a frozen model acts and improves.

1 INTRODUCTION

PhysEvo addresses unreliable robot execution by letting a frozen Astra use physical outcomes to revise its action and diagnostic resources. Across simulation and real hardware, retained revisions support improved task performance.

1 INTRODUCTION

  • Direct Astra’s demonstrated competence does not ensure reliable execution: it succeeds on 26% of RoboDojo attempts overall, while π0.5 raises success to 48%.Direct Astra reaches 100% success on object classification in the ten-task evaluation.Source: §1 · p. 2
  • Trajectory review, aligned images, robot states, and code help Astra diagnose failures and test interventions using evidence unavailable during action selection.In the pouring case, revised skill guidance changes the observation strategy and the subsequent episode succeeds.Source: §1 · p. 2
  • PhysEvo’s physical RSI retains revisions to action and diagnostic resources so later episodes inherit changes that can support both execution and further improvement.In the packing case, a previously built review tool helps trace box displacement to preceding contact and revise clearance guidance.Source: §1 · p. 2
  • One frozen Astra acts as task agent and meta-agent, revising tools and skills while developing joint-level control, evidence-seeking observation, and reusable manipulation skills.The introduction frames these as the paper’s framework, interaction-architecture, and evaluation contributions.Source: §1 · pp. 2–3
  • Across 42 RoboDojo tasks, retained versions achieve 68.14/100 score and 62.00% success, exceeding RoboDawn’s one-shot Astra agent’s 47.17% (Guo et al., 2026a).On eight manipulation tasks challenging direct Astra, PhysEvo reaches 55.00% success versus 1.25%.Source: §1 · p. 2
  • On AgileX PiPER, continued adaptation across five real-world tasks yields a 90.60/100 mean score and 84.00% success across 25 trials.Skill revisions compensate for liquid-flow effects and replace geometry-first localization with visually guided localization followed by close-range geometric refinement.Source: §1 · p. 2

2 RELATED WORK

Prior work studies robot-facing interfaces, self-improving agents, and experience-driven embodied systems. PhysEvo is positioned around joint evolution of action and observation interfaces, skills, and diagnostic resources.

Foundation models for robot control

  • Robot-control systems establish the importance of interfaces around models, while differing in learned policies, API composition, feedback, tools, persistent state, and execution memory.The section surveys VLA policies, Code as Policies, Inner Monologue, tool- and harness-design work, and systems that retain state or execution outcomes.Source: §2 · p. 3

Self-improving agents

  • Self-improving-agent work ranges from retaining feedback and evolving prompts to modifying agent implementations, harnesses, or improvement procedures.PhysEvo differs from Hyperagents’ robotics experiment, which evolves rewards for PPO controllers, by using the same frozen model for robot decisions and software revision.Source: §2 · p. 3

Experience-driven embodied agents

  • Experience-driven embodied systems retain skills, debugging knowledge, or execution memory, while PhysEvo focuses on jointly evolving robot-facing interfaces and diagnostic resources.The comparison notes that SHAPER fixes the action interface and executor, whereas PhysEvo reuses self-built tools to diagnose later failures.Source: §2 · p. 3

3 PHYSEVO: PHYSICAL RECURSIVE SELF-IMPROVEMENT

PhysEvo alternates physical execution with revision of the system used to act. One frozen model performs both roles, and tested changes persist across episodes and improvement rounds.

3 PHYSEVO: PHYSICAL RECURSIVE SELF-IMPROVEMENT

  • PhysEvo uses physical outcomes as evidence for harness revisions, then preserves tested changes for later action and improvement rounds.The system contains task and meta harnesses around one frozen model; the meta-agent can revise either harness, including its own tools.Source: §3 · p. 4, Figure 2 · p. 4, §3.1 · p. 4

3.1 ONE FROZEN MODEL, TWO EDITABLE HARNESSES

  • The initial system starts with TCP targets and local IK plus a code-editing workspace, then develops joint control, observation tools, and reusable skills.The task harness supplies observations, action tools, execution feedback, and skills, while the meta harness supports inspection and editing.Source: §3.1 · p. 4

3.2 TASK EXECUTION AND PHYSICAL FEEDBACK

  • Task execution returns observations and physical outcomes that support both next-action decisions and diagnosis of the command sequence behind a failure.The task agent can select observation tools that do not advance the environment or motion tools that execute commands and return state and images.Source: §3.2 · pp. 4–5

3.3 REVISION, TESTING, AND RECURSIVE REUSE

  • The meta-agent reviews execution and implementation, forms a failure hypothesis, and tests revisions to tools, logic, skills, or diagnostics.Behavioral candidates are compared on the same task and initial scene, while held-out layouts are reserved for final scoring; accepted changes take effect between episodes and persist in later deployment versions.Source: §3.3 · p. 5

4 THE EVOLVED EMBODIED FRAMEWORK

The evolved framework changes what Astra can control, observe, and retain as guidance. It combines joint-level action, active and retrospective observation, and reusable skills with diagnostic tools.

4 THE EVOLVED EMBODIED FRAMEWORK

  • PhysEvo’s evolved framework reshapes control, evidence gathering, and retained guidance, with task-specific deployment versions.The framework described here develops joint control, observation, and reusable skills.Source: §4 · p. 5, §4.3 · p. 6

4.1 JOINT-LEVEL CONTROL

  • Joint control makes posture part of Astra’s policy decision, unlike TCP control, which delegates posture selection to inverse kinematics.The interface accepts absolute arm-joint targets and gripper commands; Astra chooses the approach, configuration, and reconfiguration.Source: §4.1 · p. 5

4.2 ACTIVE AND RETROSPECTIVE OBSERVATION

  • Astra can obtain a useful wrist-camera view with its free arm and retrieve earlier images and aligned robot states without advancing physics.Active viewing seeks new evidence; historical retrieval recovers evidence already observed.Source: §4.2 · p. 5

4.3 REUSABLE SKILLS AND DIAGNOSTIC TOOLS

  • Reusable skills connect tools to the physical conditions for using them, while retained trajectory-review tools help diagnose why actions failed.Skills guide operations such as grasp verification, obtaining another view, and checking release; the task agent uses runtime tools and selected skills.Source: §4.3 · p. 6

5 EXPERIMENTS

Experiments evaluate retained deployment versions on held-out simulation layouts, examine the changes produced by physical RSI, and continue skill adaptation on real hardware. Results span overall RoboDojo performance, challenging manipulation tasks, and five AgileX PiPER tasks.

5.1 EXPERIMENTAL SETUP

  • Each of 42 RoboDojo tasks was attempted on five held-out layouts; scored layouts were excluded from revision and deployment-version selection.Task-specific deployment versions stayed fixed across their five scored layouts.Source: §5.1 · p. 6

5.2 PERFORMANCE ACROSS ROBODOJO

  • Under five-dimension weighting, PhysEvo achieved 68.14/100 score and 62.00% SR, exceeding RoboDawn’s 47.17% SR by 14.83 points.The average weights the five capability dimensions equally.Source: Table 1 · p. 7, §5.2 · p. 7

5.3 COMPARISON WITH DIRECT AND HYBRID ASTRA POLICIES

  • On ten GPT-as-Policy tasks, PhysEvo reached 68.00% SR versus Direct’s 26.00% and Hybrid’s 48.00%, beating Direct on eight tasks and tying two.Hybrid combines Astra with π0.5; Direct uses Astra alone.Source: §5.3 · p. 7, Table 2 · p. 8

5.4 MANIPULATION TASKS CHALLENGING DIRECT ASTRA

  • On eight manipulation challenge tasks, PhysEvo achieved 55.00% SR and 62.88 score versus official Astra’s 1.25% SR and 5.96 score, a 53.75-point SR gain.The task set is defined by VLA SR ≥20% and Astra SR <5%.Source: §5.4 · p. 8, Table 3 · p. 9

5.5 WHAT CHANGES THROUGH PHYSICAL RSI?

  • In focused tests, Joint scored 100 versus TCP’s 40 and Both’s 55; revised pouring guidance also maintained a free-arm side view through recovery.The joint-control comparison held model, skill, initialization, and budget fixed; the revised pouring skill enabled task completion.Source: Figure 3 · p. 9, §5.5 · p. 9, Figure 4 · p. 10
  • An inherited review tool helped trace box displacement to contact with raised flaps, prompting clearance guidance before turning.In the revised execution, the task agent followed the guidance and avoided pushing the box away.Source: Figure 5 · p. 10, §5.5 · p. 10

5.6 REAL-WORLD ADAPTATION THROUGH SKILL REVISION

  • On AgileX PiPER, PhysEvo achieved a 90.60/100 mean score and 84.00% SR across 25 trials on five tasks.Post-deployment cycles revised skills; examples include compensating for liquid flow and using visual guidance before close-range geometric localization.Source: §5.6 · pp. 10–11, Table 4 · p. 11

6 DISCUSSION AND CONCLUSION

PhysEvo uses action outcomes to revise a frozen Astra’s tools, skills, and diagnostic resources, retaining changes for later action and self-improvement. The procedure separates task execution from revision and testing, with updates adopted between episodes.

6 DISCUSSION AND CONCLUSION

  • Across 42 RoboDojo tasks, retained deployment versions achieve 62.00% success; on AgileX PiPER they achieve 90.60/100 mean score and 84.00% success across five tasks.The discussion attributes hardware adaptation to continued skill revision for liquid motion and localization discrepancies.Source: §6 · p. 11
  • Experience changes the system through posture control, useful observations, and diagnostic tools inherited for later failure analysis.The discussion presents these as three ways experience changes the system, not as exhaustive categories.Source: §6 · p. 11

A PHYSICAL RSI PROCEDURE

  • The task harness supplies action context, while each improvement round proposes a revision and tests it before the selection step adopts it or preserves the current harness.The same frozen model is used in both roles; robot dynamics, actuator restrictions, model configuration, and evaluation rules remain outside the editable system.Source: A PHYSICAL RSI PROCEDURE · p. 16
  • Retained tools and their usage guidance are loaded together, updates take effect between episodes, and each task keeps its own deployment version before scoring.Development resources are shared across tasks, but task-specific skills remain task-local.Source: A PHYSICAL RSI PROCEDURE · p. 16

B.1 METRICS AND AGGREGATION

The aggregation combines two overlapping task sets without counting shared tasks twice, while keeping deployment versions fixed for held-out scoring and resetting episode state.

B.1 METRICS AND AGGREGATION

  • The 10-task and 8-task sets overlap on tower building and making kong; their 16-task union contains 80 attempts, with 75.06 score and 65.00% SR.Each task is counted once in the union rather than averaging the two subset averages.Source: §B.1 · p. 17
  • Each task’s retained deployment version is fixed across five scored layouts excluded from harness revision and version selection.Development resources may carry experience across tasks, but task-specific skills remain task-local.Source: §B.1 · p. 17
  • Scored episodes reset session context, historical-frame indices, and writable memory, while allowing within-episode observation review and online correction.This feedback does not change persistent tools, skills, or the deployment version.Source: §B.1 · p. 17

B.2 COMPARISON SOURCES AND AGGREGATION

The comparison section documents source records and aggregation choices for the five-dimension benchmark, ten-task policy comparison, and eight-task manipulation subset.

B.2 COMPARISON SOURCES AND AGGREGATION

  • RoboDawn’s five-dimension weighted means are 54.63 score and 47.17% SR, agreeing with its paper.The figures are computed from public evaluation records for GPT-6 Astra, 1-shot, with five runs per task.Source: §B.2 · p. 17
  • For Direct on the ten-task set, task-mean aggregation gives 38.50 rather than the pooled-case score of 37.81.PhysEvo is evaluated on five layouts per task, one attempt per layout; the three displayed VLA methods have the source report’s highest ten-task average scores among its VLA references.Source: §B.2 · p. 17
  • The eight-task manipulation subset is selected for VLA SR ≥20% and Astra SR <5%, with methods averaged using equal task weights.The subset corresponds to Table F2 of the RoboDojo Astra report.Source: §B.2 · p. 17

B.3 REAL-WORLD SCORES AND SUCCESS CRITERIA

Hardware scores capture partial and full completion, while success rate counts trials reaching each task’s maximum score.

B.3 REAL-WORLD SCORES AND SUCCESS CRITERIA

  • 90.60/100 mean Score and 84.00% SR summarize the five hardware task-level metrics, each based on five trials.Score ranges from 0 to 100; SR is the percentage of trials reaching maximum score.Source: §B.3 · p. 18
  • Stacking three blocks or bowls earns full credit only when all are stacked and the arm returns home; bowl stacking also requires a stable bottom bowl.For either task, a score of 15 is assigned when two objects are stacked.Source: §B.3 · p. 18
  • Pen placement scores 50 for one pen and 100 for both; pouring requires the cup’s required liquid and an upright bottle for full credit.Pouring scores 0 if liquid has not been successfully poured into the cup.Source: §B.3 · p. 18
  • Writing “PhysEvo” scores incrementally by correct letters; its five trials average 93/100 Score and 60% SR.The reported scores are 100, 100, 100, 75, and 90.Source: §B.3 · p. 18

C ADDITIONAL FRAMEWORK ANALYSIS

The section tests control interfaces, validates observation tools, and traces how skill-guided observation and diagnosis alter manipulation episodes.

C.1 CONTROL-INTERFACE COMPARISON

  • Joint control scores 100, versus 40 for TCP and 55 for Both, in the six-episode object-classification comparison.The model, skill, layout initialization, and budget were held fixed; Joint was retained as the default interface.Source: §C.1 · p. 18

C.2 OBSERVATION TOOLS

  • The reconstruction test accepted 8 of 12 points, with 0.615 mm median error and 1.437 mm 95th-percentile error; temporal triangulation also requires matching a stationary point across views.A separate wrist-camera translation of approximately 57.525 mm yielded about 2.14 mm marker error.Source: §C.2 · p. 19

C.3 SKILL-GUIDED OBSERVATION AND DIAGNOSIS

  • A revised pouring skill maintains a free-arm side view through pouring, draining, and recovery, and the revised episode succeeds.The other loaded skills remain fixed; the observation arm gathers views while the pouring arm makes corrections.Source: §C.3 · p. 19
  • Reusing a trajectory-review tool to inspect box displacement leads the meta-agent to add clearance-above-flaps guidance, after which the task agent avoids pushing the box away.The review presents RGB images, robot states, and policy decisions.Source: §C.3 · p. 19

D ROBOT AND AGENT AVERAGES ON THE MANIPULATION SUBSET

On the eight-task manipulation subset, PhysEvo exceeds the displayed robot-policy references; the remaining sections document simulation executions and the task-agent interface.

D ROBOT AND AGENT AVERAGES ON THE MANIPULATION SUBSET

  • 55.00% SR for PhysEvo exceeds Simate-beta’s 50.83% in the extended comparison; G0.5 leads the original 40-policy set at 43.09%.The means weight all eight tasks equally; Table 6 presents the original references and extended comparison.Source: D ROBOT AND AGENT AVERAGES ON THE… · p. 20, Table 6 · p. 20

E ADDITIONAL SIMULATION EXECUTIONS

  • Figures 7–12 present all 42 simulation executions in S01–S42 order, with six left-to-right frames per strip; Align Blocks is one listed task.Task names and instructions appear above each strip.Source: E ADDITIONAL SIMULATION EXECUTIONS · pp. 21, 26

F TASK-AGENT INPUT, OUTPUT, AND TOOL FORMATS

F.1 INPUT CONTEXT

  • Input context separates the task instruction, native-step index, reported state, same-episode memory, prior feedback, and remaining step and model-call budgets.Source: §F.1 · pp. 27–28

F.2 MODEL OUTPUT AND FUNCTION CALLS

  • The model emits one enabled function call with assessment, plan, phase, and updated memory, alongside tool-specific arm targets and segment settings.Source: §F.2 · pp. 28–29

F.3 TOOL ARGUMENTS AND EXECUTION FEEDBACK

  • Motion calls use six absolute joint targets per arm and return execution status and tracking; observation and geometry calls return evidence without executing native actions.Motion segments use phase-dependent step limits, and execution checks may stop a segment early.Source: §F.3 · p. 30
Loading 2610.08995v1…