PaperlayerView exact revision on arXiv

Source-linked AI summary

Has GPT-6-Astra Achieved Embodied Intelligence?

SenseNova, Joint Research Team, :, Zhongang Cai, Ziang Cao, Tailai Chen, Han Chong Chua, Linjun Dai, Hanming Deng, Hokin Deng, Haiwen Diao, Chenjing Ding, Weichen Fan, Xiao Fu, Zerui Gong, Chenyang Gu, Zexin He, Zhi Hou, Bo Li, Qinyue Li, Quanzhou Li, Xiaowan Li, Zhichao Liao, Dahua Lin, Zilong Lin, Bo Liu, Chunxiao Liu, Pengfei Liu, Tianqi Liu, Ziwei Liu, Chen Change Loy, Lewei Lu, Jiapeng Luo, Yuexiao Ma, Liang Pan, Xingang Pan, Oscar Qian, Ziyi Shang, Yuyao Shao, Dawei Su, Dacheng Tao, Qingyi Tao, Quan Wang, Ruisi Wang, Xiaogang Wang, Ziyi Wang, Beichen Wen, Donghao Wu, Jiaxu Wu, Silei Wu, Xiaoming Wu, Zhonghua Wu, Hongcan Xiao, Weiye Xu, Atsushi Yamashita, Lei Yang, Zhe Yang, Baiqiao Yin, Wanqi Yin, Hanfei Zhao, Tianbo Zhong, Long Zhuo, Jing Zuo

arXiv:2610.22471v1cs.ROcs.CVcs.LGcs.MM

TL;DR

The report asks how far a general-purpose model’s embodied capabilities extend beyond specialized task-specific systems, and what those capabilities mean for research. It evaluates GPT-6-Astra with in-context demonstrations and larger execution budgets, studies control harnesses and real-robot deployment, and finds strong benchmark performance alongside limits to practical embodied intelligence.

  • Problem

    The report asks how capable a general-purpose model is at robotic manipulation and navigation, and what its capabilities imply for the field.

    Source: §1 · p. 2

  • Method

    The study evaluates GPT-6-Astra across embodied benchmarks using in-context learning and expanded interaction budgets, and examines harness design and real-robot deployment.

    Source: §1 · p. 2

  • Results

    Across 12 representative benchmarks, GPT-6-Astra outperforms existing methods by substantial margins, approaches saturation on LIBERO and SIMPLE, and scores 81.9% on RoboDojo and 87.3% on DexJoCo.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Strong harness design, in-context learning, and sufficient execution budgets can push many isolated benchmarks close to saturation, but practical embodied intelligence remains constrained by transfer, long-horizon reasoning, and real-world conditions.

    Source: §6 · p. 18

    Even on nearly solved benchmarks, GPT-6-Astra can require substantial tokens and wall-clock time, including about a million tokens and nearly an hour per LIBERO-Long++ episode.

    Source: §2.4 · p. 8

Abstract

from arXiv · show

GPT-6-Astra demonstrates a level of competence in robotic manipulation and navigation that challenges the conventional paradigm of building specialized models for individual tasks and embodiments. In this report, we ask not only how capable GPT-6-Astra is, but what its capabilities imply for the field. First, we probe the limits of standalone GPT-6-Astra. We conduct a large-scale evaluation across 12 representative benchmarks spanning tabletop manipulation, humanoids, mobile platforms, and dexterous hands. Under a unified protocol that allows in-context learning (ICL) from training-set demonstrations and substantially increases the number of execution steps, GPT-6-Astra achieves unprecedented performance (Fig. 1). Future research may focus on improving 1) the generality of ICL 2) cost and execution time, 3) physical-state verification, 4) recovery from irreversible failures, and 5) long-horizon preservation of progress. Second, we study the agentic harness for embodied control. We identify that close-loop coordination between a high-level "System 2" reasoner (i.e., GPT-6-Astra) and a low-level "System 1" controller (e.g., π0.5) is costly but effective. Third, we examine GPT-6-Astra on real-world deployment. Our experiments on 8 robot configurations reveal substantial gaps: 1) robot demonstrations required for ICL are expensive to collect, and 2) it is implausible to trade longer execution for higher success rates with physical constraints. Fourth, we envision the next generation of embodied benchmarks. We develop a prototype that 1) derives tasks from real-world needs, 2) provides no in-domain demonstrations, and 3) judges agents on a world clock that keeps running while they deliberate. Taken together, our results suggest that GPT-6-Astra has largely crossed the threshold of embodied competence, but not yet that of practical embodied intelligence.

1 Introduction

The introduction evaluates GPT-6-Astra’s embodied capabilities, studies the harness needed to use them, and identifies limits in transfer, cost, and physical deployment. It also motivates benchmarks that reflect human needs and real-time world changes.

1 Introduction

  • Across 12 representative benchmarks, GPT-6-Astra outperforms existing methods by substantial margins, approaches saturation on LIBERO and SIMPLE, and achieves 81.9% on RoboDojo and 87.3% on DexJoCo.The evaluation covers tabletop arms, humanoids, mobile platforms, and dexterous hands.Source: §1 · p. 2
  • In-context learning builds a frozen package of task guidance, motion and visual references, helpers, recovery procedures, and metadata, but its benefits weaken as test tasks diverge from demonstration data.The package is constructed from available training demonstrations and adapted to the test scene.Source: §1 · p. 2
  • Allowing up to 10× more interaction steps enables correction-heavy tasks to benefit from detecting failed actions, revising strategies, and recovering through further interaction.The report says standard step limits can significantly underestimate GPT-6-Astra’s capabilities.Source: §1 · p. 2
  • Closed-loop System 2–System 1 collaboration improves performance over standalone or open-loop execution, but frequent GPT supervision incurs significant token cost.Reviewing longer action sequences improves the trade-off; combining closed-loop collaboration with ICL gives the strongest performance at the highest cost.Source: §1 · p. 2
  • On real robots, performance varies by task: tabletop grasping works particularly well, while deformable-object manipulation, precise contact, and long-horizon tasks remain challenging.The evaluation spans single and dual arms, grippers, dexterous hands, a wheeled mobile manipulator, and a humanoid.Source: §1 · pp. 2–3
  • Physical deployment limits longer execution through safety, mechanical, and real-time constraints, while successful trajectory demonstrations for ICL are often expensive or unavailable.The proposed benchmark prototype uses human-needs-based tasks, no in-domain demonstrations, and a world clock that continues during deliberation.Source: §1 · p. 3

2 Large-Scale Study under A Unified Protocol

The study evaluates GPT-6-Astra across 12 embodied settings with fixed weights, task-specific in-context preparation, and extended interaction, finding broad performance alongside unresolved cost and reliability limits.

2.1 Protocol

  • The protocol freezes a task-specific package built from training demonstrations, then tests in isolated episodes at standard or extended step budgets without updating weights or transferring information between episodes.Source: §2.1 · p. 4

2.2 In-context Learning

  • At 1×, GPT-6-Astra matches or exceeds published bests on most of 12 settings, nearly doubling success on LIBERO-Ultra-Long and DexJoCo.Source: §2.2 · p. 4
  • Across benchmarks, six recurring package components support retrieving and adapting references through helpers under feedback rather than replaying demonstrations; RoboLab lacks training data for a task-level package.Source: §2.2 · pp. 5, 7

2.3 Extending Interaction Budget

  • Extending interaction to 10× yields large gains on RoboDojo and Bench2Dex, but only modest gains on LHPR-VLN, while DexJoCo and VLN-CE reach around 90% and LIBERO and SIMPLE exceed 95%.Source: §2.3 · p. 7

2.4 Limitations and Failure Modes

  • ICL packages are largely task-specific, weakening transfer when test tasks depart from their source data; RoboDojo Open and RoboLab illustrate limited package coverage.Source: §2.4 · p. 8
  • Costs remain high even on strong-performing benchmarks: LIBERO-Long++ exceeds 97% success yet uses about a million tokens and nearly an hour per episode, while LIBERO-Ultra-Long costs roughly nine million tokens and nearly two hours.Source: §2.4 · p. 8
  • Physical verification can mistake visual plausibility for completion, while irreversible spills or dropped parts can eliminate recovery paths and stall later subtasks.Source: §2.4 · pp. 8–9
  • Long-horizon execution can lose earlier progress through later corrections or accumulated deviations, so completed conditions need continued tracking and rechecking.Source: §2.4 · p. 9

3 Harness Ablation

The harness ablations show that embodied performance depends on how GPT-6-Astra coordinates with a VLA and how actions are represented and chunked; higher performance can carry substantial token costs.

3 Harness Ablation

  • The harness ablations test whether collaboration and action representations improve task performance while managing inference cost.The study compares System 1–System 2 collaboration and action representations and chunking.Source: §3 · p. 9

3.1 System 1–System 2 Collaboration

  • 54.17% success with ICL plus closed-loop VLA exceeded the best non-ICL closed-loop configuration by 28.34 percentage points, at 19.335 million tokens.Reviewing four proposed actions per GPT call reduced token use by 63.5% versus per-action review while improving success from 23.61% to 25.83%.Source: §3.1 · p. 10, Figure 7 · p. 10

3.2 Action Representation and Chunking

  • Adaptive chunking raised average success by 4.44 percentage points using 1.146 million tokens, while fixed L = 4 reduced success by 4.17 points at 0.435 million tokens.Directional motion commands improved average success by 2.50 percentage points, whereas direct joint-angle control caused a 5.84-percentage-point drop.Source: §3.2 · pp. 10–11, Figure 7 · p. 10

4 Real-World Validation

The real-world study evaluates GPT-6-Astra across eight robot platforms under zero-shot, human-context, wall-clock conditions. Success varies by task, and physical constraints limit the benefits of longer execution.

4 Real-World Validation

  • The real-world study revises simulation assumptions by removing task demonstrations and charging elapsed wall-clock time as the budget.Source: §4 · p. 11

4.1 Setting: The Real-World Variant

  • The variant covered eight platforms in a zero-shot setting, replacing demonstrations and knowledge packages with human-written context and using wall-clock trials judged by physical outcomes.Trials were normally budgeted for 60 minutes; a person judged success from the physical outcome.Source: §4.1 · pp. 11–12

4.2 Findings

  • Across eight platforms, rigid pick-and-place succeeded in 17/18 trials, while cloth folding (2/4), plug insertion (0/11), and mobile navigation/search (9/23) remained harder.A second arm extended the range of zero-shot tasks, and GPT-6-Astra replanned from execution feedback when the scene changed.Source: §4.2 · p. 12, Figure 8 · p. 12
  • On Piper, GPT moved the free arm’s camera until it saw the vase mouth and succeeded, unlike the single-arm Franka attempt, making viewpoint part of the action space.The gripper-held flower obscured the vase mouth from the wrist camera in both cases.Source: §4.2 · pp. 12–13, Figure 9 · p. 13
  • On Unitree G1, GPT-issued manipulation targets succeeded 2/3 and vendor-interface walking and turning 3/3, demonstrating slow feedback-driven feasibility rather than reliable real-time visuomotor control.Source: §4.2 · p. 13

4.3 What Changes in the Real World

  • Physical trials cannot freely trade longer execution for success: protective stops, irreversible outcomes, actuator heating, and deliberation time constrain attempts; G1 deliberates about 90% of each run, and manipulation exceeds an hour.Source: §4.3 · p. 13
  • Human-written context can enable tasks without demonstrations, but quality matters: Tianji grasps were 0/3 without primitives versus 3/3 with them, while alternative context sources remain unevaluated.Source: §4.3 · pp. 13–14

5 Towards a Next-Generation Embodied Benchmark

The section proposes benchmarks grounded in human needs, without in-domain demonstrations, and scored while the world continues to change. A household prototype illustrates how GPT-6-Astra’s failures in tracking obligations, discovering unfamiliar fixtures, and acting on stale observations motivate these principles.

5 Towards a Next-Generation Embodied Benchmark

  • Established benchmarks reflect imitation-learning assumptions; the proposed alternative evaluates a general agent through its robot interface in a changing world.The section argues that demonstrations and extra execution time are not free on real robots, so raising benchmark difficulty alone would not address the mismatch.Source: §5 · p. 14, §5.1 · p. 14

5.1 Principles for Next-Generation Benchmarks

  • Tasks should express human needs, with utility judging alternative outcomes and explicit requests evaluated alongside unstated obligations.A fixed goal can contradict an instruction or miss acceptable refinements; obligations such as an unattended stove or running tap may need attention while the robot is busy.Source: §5.1 · pp. 14–15
  • Benchmarks should withhold in-domain demonstrations and assess exploration within a deployment, using judge-private physical predicates and charging exploration for its time and side effects.Experience carries over between requests in a deployment and resets between deployments.Source: §5.1 · p. 15
  • The world should keep running during deliberation, with the judge evaluating the whole trajectory and outcome utility depending on what was achieved and when.This captures deadlines and obligations over time that a final-state goal can miss, such as an egg that is ready and then burns.Source: §5.1 · pp. 15–16

5.2 A Household Proof of Concept

  • The household prototype uses a physics-bound apartment and need-only tasks, then tests GPT-6-Astra zero-shot with onboard sensing and no demonstrations.The five reported failures include disturbing desk objects and leaving the refrigerator open; final-state checks would miss both episodes.Source: §5.2 · pp. 16, 18, Figure 11 · p. 16

Finding 2: Without demonstrations, GPT-6-Astra does not work out how unfamiliar fixtures move or where

  • Without demonstrations, GPT-6-Astra repeats ineffective fixture interactions and fails to account for how its body and an open door constrain reaching.It also decides to turn off a burner based on an egg observation that becomes outdated during deliberation; the authors characterize these episodes as few and qualitative.Source: Finding 2: Without demonstrations,… · p. 18

5.3 Open Challenges

  • A full benchmark still faces challenges in judge evidence, simulation fidelity, latency-dependent scoring, and the expense of repeating long deployments.The authors note that simulation has not been shown to predict real-robot results for long-horizon household activities, and that short robot trials are often too few for reliable policy comparisons.Source: §5.3 · p. 18

6 Conclusion

The report concludes that strong harness design, in-context learning, and adequate execution budgets bring many isolated benchmarks close to saturation, while practical embodied intelligence remains constrained. It situates this question alongside foundation-model agents and learned robot policies, asking how far a general-purpose model can act without benchmark-specific policy training.

6 Conclusion

  • Strong harness design, in-context learning, and sufficient execution budgets push many isolated benchmarks close to saturation, but real-world constraints and transfer challenges remain.The report argues that benchmark success should be interpreted carefully and that reliable, efficient operation under realistic deployment conditions remains the frontier.Source: §6 · p. 18

B.1 Foundation Models as Embodied Agents

  • Earlier foundation-model systems typically specify what to do while a skill set, planner, or separately trained controller determines how; this report tests a frontier model as controller.The related work also describes harnesses that adapt action interfaces, context, verification, and recovery around models.Source: §B.1 · p. 35

B.2 Robot Policy Learning

  • Robot-policy research includes demonstration-trained policies, vision-language-action models, and world-action models, while this report asks how far a general-purpose model can go without benchmark-specific policy training.The related work notes that these approaches learn policies from robot data, despite differences in their training and action-generation methods.Source: §B.2 · pp. 35–36

B.3 Embodied Benchmarks

The section surveys embodied benchmarks and frontier-model studies, then reports GPT-6-Astra + ICL evaluations across six settings. Results vary by benchmark, while extended interaction and demonstration-informed preparation accompany several strong outcomes.

B.3 Embodied Benchmarks

  • Embodied benchmarks span manipulation, navigation, household activities, and other settings; benchmark saturation and imitation-policy-oriented designs motivate different evaluations for general-purpose agents.The survey includes task suites for long-horizon activities, changing environments, and outcome-based assessment.Source: §B.3 · p. 36

B.4 Empirical Studies of Frontier Models in Embodied Settings

  • Earlier frontier-model studies often examined one benchmark family, capability, or harness at a time, with conflicting demonstration findings; this report combines evaluation, harness ablations, robot validation, and benchmark principles.The cited studies disagree on whether demonstrations improve RoboDojo success.Source: §B.4 · p. 37

C.1 LIBERO-PRO

  • 98.25% success on LIBERO-PRO at 1× exceeded the strongest prior result by 5.62 points; cumulative 10× execution reached 99.25%.The 1× result covers 786 of 800 episodes; the 10× result reaches 794 of 800.Source: §C.1 · p. 38, Table 3 · p. 39

C.2 LIBERO-Long++ and LIBERO-Ultra-Long

  • At 1×, GPT-6-Astra + ICL reached 92.4% SR and 98.1% AP across nine long-horizon scenarios; cumulative 10× results were 99.6% and 99.8%.The benchmark evaluates compositional manipulation under visual clutter and reordered skill chains.Source: §C.2 · pp. 40–41, Table 4 · p. 40

C.3 RoboDojo

  • RoboDojo evaluation covered all 42 tasks across five groups, using task-specific demonstrations for 34 tasks and a shared demonstration package for eight Open tasks.The supplied passages describe the evaluation setup but do not provide RoboDojo outcome values.Source: §C.3 · p. 41, Table 5 · p. 42

C.4 RoboLab

  • 57.1% SR on RoboLab exceeded every listed baseline, including FLUX 3 Action at 42.9%, by 14.2 percentage points.The aggregate comparison does not identify which competency axes account for the difference.Source: §C.4 · p. 43, Table 6 · p. 43

C.5 Bench2Dex

  • At 1×, GPT-6-Astra reached 25.6% SR on Bench2Dex, comparable to π0.5’s 25.5%; at 10× it reached 61.5%.The baseline and GPT-6-Astra runs used different episode counts, and test-token counts are lower bounds.Source: §C.5 · p. 44, Table 7 · p. 44

C.6 DexJoCo

  • 78.2% SR on DexJoCo at 1× exceeded the published π0.5 baseline of 40.3% by 37.9 points; 10× reached 87.3%.The two methods used different numbers of test seeds per task.Source: §C.6 · p. 45, Table 8 · p. 45

C.7 LHPR-VLN

On the selected LHPR-VLN episodes, extended evaluation improves several navigation metrics, but full-sequence success remains limited. The discussion notes that irreversible failed submissions make verification and progress tracking consequential.

Evaluation subset

  • The evaluation retained 114 of 205 unseen-test episodes across 25 scenes, requiring every subtask target object to be visible along its reference trajectory.Each retained episode contained two to four goals.Source: §C.7 · pp. 45–46

Protocol and metrics

  • Agents received a global instruction and frozen training-scene ICL workspace; the 1× and 10× budgets allowed 500 and 5,000 actions per sequence.A failed STOP submission advances the stage regardless of success, and environment feedback does not reveal submission success.Source: §C.7 · p. 46

Results

  • 21.05% SR at 1× rose to 28.95% under extended evaluation, increasing completed episodes from 24 to 33 while average token use rose 1.96×.ISR increased from 54.68% to 62.87%; SR still remained below 30%.Source: §C.7 · p. 46, Table 9 · p. 46

Discussion

  • Full-sequence success remained below 30% despite individual-goal success above 60%; failed submissions advance irreversibly, while aggregate scores cannot isolate verification’s contribution to remaining failures.The discussion also cautions that baseline comparisons do not control for inference cost.Source: §C.7 · p. 46

C.8 VLN-CE

On 100-episode R2R and RxR subsets, extended evaluation improved route-completion and path-efficiency metrics. Direct baseline comparisons are limited by differences in episode coverage, observation access, and inference budgets.

Benchmarks and tasks

  • VLN-CE evaluates continuous-environment route following from a natural-language instruction to one goal, requiring visual grounding and progress tracking without a predefined navigation graph.R2R and RxR use different turn increments and RxR also allows camera tilts.Source: §C.8 · p. 47

Evaluation subsets

  • The 100-episode subsets matched R2R coverage across ten scenes; RxR episodes were selected to approximate the full split’s instruction lengths and reference-path lengths.The selection used stratified sampling followed by within-scene replacements.Source: §C.8 · p. 47

Protocol and metrics

  • Each episode used a fresh session and frozen ICL workspace from non-overlapping training scenes; agents received one forward-facing RGB image and up to 5,000 actions.STOP succeeded within 3 m geodesic distance of the goal.Source: §C.8 · p. 47

Results

  • Extended evaluation raised SR from 80% to 88% on R2R and from 82% to 90% on RxR, adding eight successes per subset.SPL rose by 8.51 and 7.91 percentage points, respectively.Source: Table 10 · p. 47, §C.8 · p. 48

Discussion

  • On R2R, OS exceeded SR by three points under extended evaluation, while SPL remained below SR on both datasets, indicating successful outcomes can involve detours.The discussion cautions that episode coverage, observation access, and inference budgets limit direct baseline comparisons.Source: §C.8 · p. 48

C.9 SIMPLE

SIMPLE results show that extended interaction closes the zero-shot versus ICL gap on the shared complete subset, while ICL substantially improves standard-budget success. These comparisons are limited by incomplete coverage and unmatched system resources.

Results on the shared complete subset

  • Matched-8 covers only eight complete task–level cells across three tasks, and system compute, sensing, tool access, latency, and hardware are not matched.Six-task totals are descriptive references with differing denominators; efficiency figures use the matched-success set, not Matched-8.Source: §C.9 · pp. 49–50
  • 98.8% (79/80) success at 10× for both zero-shot and ICL, up from 65/80 (81.3%) for zero-shot at 1× on Matched-8.The 10× results are compared on the same 80 complete episodes; published Ψ0 and DreamZero each achieve 74/80 (92.5%) at 1×.Source: Table 11 · p. 49, §C.9 · p. 49

Standard-budget ICL and efficiency

  • On 81 episodes solved in both settings, ICL reduces mean action steps from 376.9 to 210.4 (44.2%), while token consumption rises from 2.64M to 6.40M.Mean test wall-clock time falls from 47.0 to 31.4 minutes; ICL preparation additionally costs 8.34M tokens and 90.4 minutes per task, paid once.Source: §C.9 · pp. 49–50
  • 90.8% (109/120) ICL success versus 70.8% (85/120) zero-shot at 1× on Common-4, a 20.0-percentage-point gain.ICL is comparable to the published Ψ0 result of 108/120 (90.0%); the one-case difference does not establish statistical superiority.Source: §C.9 · p. 49, Table 12 · p. 50

C.10 BEHAVIOR

The BEHAVIOR-1K evaluation measures GPT-6-Astra on long-horizon household activities using an end-effector-pose interface. It completes 37 of 100 tasks, while the interface choice and contact-rich failures qualify the result.

Results

  • Across 100 evaluation rollouts on the BEHAVIOR-1K 2026 Challenge, our method successfully completes 37 tasks, a 37% overall task success rate; each task instance receives one evaluation rollout.Source: §C.10 · p. 50

Other Discussion

  • Task success rises from 1% with the 23-dimensional joint-space interface to 37% with the 21-dimensional end-effector-pose interface.The pose interface expresses end-effector position and orientation directly, leaving joint-level kinematic computation to the official controller.Source: §C.10 · pp. 50–51
  • Failures include imprecise grasp alignment, arm collisions, navigation obstructions, and ineffective recovery, so the results do not establish reliable manipulation from high-level reasoning alone.Qualitative trajectory inspection suggests procedural knowledge extraction, task-stage identification, and visual-feedback adaptation.Source: §C.10 · p. 51

D Harness Ablation Study

The harness ablations test whether added agent components improve GPT-6-Astra’s embodied performance. Feedback and history compression provide the clearest gains, while an execution subagent reduces performance.

D.1 Agent Components

  • Feedback evaluates outcomes and proposes continuation, adjustment, or retry; history compression retains four recent turns and summarizes earlier interactions in at most 2,000 tokens.The study also tests planning and execution subagents individually.Source: §D.1 · p. 51
  • Across 42 RoboDojo tasks, feedback adds 7.22 average score points and 9.16 success-rate points for 7.03% more tokens; history compression adds 5.25 and 5.55 points while cutting tokens 66.41%.The ablations report task scores and success rates across five dimensions, with average computed as the unweighted mean; token use includes GPT-6-Astra and auxiliary agents.Source: D Harness Ablation Study · p. 51, Table 13 · p. 51, §D.1 · p. 52
  • The execution subagent reduces average score by 1.22 points at approximately the baseline token cost, while planning adds 3.25 points at a 15.63% token increase.Planning’s substantial gain in the Memory category indicates that its benefits vary across task categories.Source: §D.1 · p. 52

E Full Real Robot Results

  • The appendix documents real-robot experiments across platforms, presenting setting, supplied context, tasks and results, findings, and visualizations.It summarizes all platforms before detailing them from the simplest to the most complex embodiment.Source: E Full Real Robot Results · p. 52

E.1 Platforms and Results

The real-robot appendix reports outcomes across eight platforms, with tasks ranging from successful block stacking and cloth folding to repeated plug-insertion failures. Demonstrations are absent, and success criteria and operating constraints vary by platform.

E.1 Platforms and Results

  • Table 14 summarizes contexts and outcomes across eight platforms without demonstrations; Figure 13 groups task rollouts into all-success, moderate-completion, and no-success outcomes.The gallery orders moderate-completion tasks by success rate and notes that one task’s result is imperfect despite all counted trials succeeding.Source: §E.1 · p. 52, Table 14 · p. 53, Figure 13 · p. 54

B. Moderate completion

  • Moderate-completion examples include cloth folding and navigation tasks, while the gallery’s no-success group includes plug insertion on three platforms and other task failures.The moderate group includes Pika cloth folding, Tracer + xArm7 navigation, a Unitree G1 pick-and-drop, and Franka folding tasks.Source: Figure 13 · p. 54, B. Moderate completion · p. 55, C. Always fails · p. 55

C. Always fails

  • The no-success group also includes toy grasping, placing flowers in a basket, and finding a vacuum machine.These examples are listed alongside plug insertion failures in the outcome gallery.Source: Figure 13 · p. 54, C. Always fails · p. 55

Setting

  • Franka Research 3 tasks used RGB-only third-person and wrist cameras, short end-effector trajectory segments at 15 Hz, and a one-hour trial limit.The model received task instructions, two RGB views, robot-state feedback, and no demonstrations or semantic manipulation primitives.Source: Setting · p. 55, Context provided · p. 55

Tasks and results

  • On Franka Research 3, six block-stacking trials succeeded across different initial configurations, matching simulation; two folding trials ended in execution errors.The three plug-insertion trials were marked failures by the operator.Source: Tasks and results · p. 55, Findings · p. 56

Visualization

  • The visualizations show Franka 3 block stacking, cloth folding, and a plug reaching—but not entering—the socket; Panda and PiPER panels show task phases and feedback-driven replanning.PiPER examples include replanning after a displaced receiving bowl, cloth folding, and plug insertion.Source: Figure 14 · p. 56, Figure 15 · p. 57, Figure 16 · p. 58

Tasks and results

  • Franka Panda runs had no common time limit, and each physical motion request required operator approval; tape-roll stacking and white-towel folding succeeded in all five trials.The successful dark-blue-cloth folding run took 196 minutes; only one trial of the tape-roll and towel tasks had a model log.Source: Setting · p. 56, Tasks and results · p. 56

Findings

  • Franka Panda findings include isolated subtasks during plug insertion, cautious stopping when insertion remains unverified, and no recovery after a reported hardware collision.In one trial, the model made 117 tool calls, used 10.6 M tokens, withdrew 2.2 mm, and reported insertion as unverified.Source: Findings · pp. 56–57

Tasks and results

The reported rollouts cover tabletop tasks ranging from placement to folding and insertion, with findings highlighting spatial-order following and bimanual coordination. The setup varies by robot and by whether human-written grasp scripts are provided.

Findings

  • The listed rollouts include placement, folding, and flower insertion; the findings report following spatial relations and ordered instructions, plus using both arms to lift or hold objects.The reported bimanual examples include lifting a basket clear of the table and holding a carton.Source: Findings · p. 59, Tasks and results · p. 59

Visualization

  • Figures 17 and 18 depict AgileX Piper/PiperX tabletop tasks and Tianji toy and bottle grasps, respectively.The Tianji figure distinguishes a model-written grasp-code attempt from human-scripted examples.Source: Figure 17 · p. 60, Figure 18 · p. 61

Setting

  • The Tianji setup used two Wuji Gen-1 hands, RGB head-camera input, end-effector pose and grasp-label actions, and a 60-minute attempt budget.A person approved the final state; the human-coding condition used grasp scripts, with no demonstrations in either condition.Source: Setting · p. 60, Context provided · p. 61

Context provided

  • The Tianji tasks supplied task instructions, tool schemas, joint limits, hand profiles, RGB observations, and robot-state feedback; the human-coding condition added human-written grasp scripts.Neither condition used demonstrations.Source: Context provided · p. 61

Tasks and results

  • On the Tianji M6S, model-written grasp code achieved 0/3 toy grasps, while a human-written grasp script achieved 3/3; the bottle task with a human script achieved 1/3.The successful bottle attempt followed human feedback after a timed-out attempt.Source: Tasks and results · p. 61, Findings · p. 61

Tasks and results

Results vary across Galaxea manipulation and Tracer navigation tasks, with hardware faults, lost progress, and collision recovery among the reported constraints. The configurations use different interfaces and receive no demonstrations.

Tasks and results

  • The Galaxea table reports successes ranging from 0/2 for flower collection to 2/2 for washer-button pressing and basket lifting; both flower runs ended with torso over-temperature faults.The washer cycle did not start in either run, and one run showed an unexpected fault code.Source: Tasks and results · p. 62

Findings

  • Long Galaxea runs sometimes hit hardware over-temperature faults, which the authors identify as a hardware-reliability limitation.The robot also used joint-effort changes as a contact signal and cross-checked depth patches against RGB when reflective surfaces corrupted wrist depth.Source: Findings · p. 62

Setting

  • For Tracer, each decision returned three waypoints for a controller, with pose estimated from commanded velocity; Galaxea tabletop and navigation tasks used different exposed controls and observations.Galaxea received no demonstrations, and each task was run zero-shot.Source: Setting · p. 63

Tasks and results

  • Tracer navigation successes varied by task: 2/5 for passing the chair and approaching the box, 2/4 through the tripod–table gap, 3/4 approaching the monitor, 2/5 finding a person, and 0/5 finding the vacuum.The model received RGB input only, without a robot model, SLAM, collision detection, or demonstrations.Source: Context provided · p. 63, Tasks and results · p. 64

Findings

  • On Tracer, the model sometimes lost its heading or forgot it had reached the goal area, repeatedly checked constraints, and almost never planned recovery after collisions.The chair instruction prompted repeated confirmation of which side to pass.Source: Findings · p. 64

Visualization

  • Figure 20 shows one recorded run per Tracer navigation task, with chips indicating successes over all trials; Figure 19 depicts Galaxea washer, flower, and tabletop tasks.The navigation footage is third-person phone video.Source: Figure 19 · p. 63, Figure 20 · p. 64

Tasks and results

On the Unitree G1, GPT-6-Astra completed all three locomotion trials and two of three plush-toy pick-and-drop trials, but the small sample establishes feasibility rather than reliability. The findings also qualify the apparent success with stopping-distance and execution-time limits.

Tasks and results

  • Manipulation succeeded in 2/3 trials and locomotion in 3/3, for 5/6 overall; the failed manipulation run dropped the toy off the table and recovery was not completed.With three trials per task, the authors characterize these results as establishing feasibility rather than reliability.Source: Tasks and results · p. 65, Figure 21 · p. 66

Findings

  • All three locomotion runs stopped at a self-estimated 0.15–0.20 m rather than the requested approximately 0.10 m, which the authors attribute to the no-contact requirement.The approach was observation-conditioned, with successive forward commands shortened after fresh images.Source: Findings · p. 65
  • In one successful pick run, all 26 arm motions returned stalled-short-of-target or timeout statuses, despite a human-labelled successful outcome.The median final tracking error was 0.07065 rad and the maximum was 0.1904 rad; reliable error compensation was not established.Source: Findings · p. 66
  • Across 108 archived API calls, median request latency was 24.84 s and API wall time accounted for 88.8–91.7% of elapsed run time; both successful manipulation runs lasted over an hour.Image capture took 4.5–9.0%; the remaining time includes tool execution and harness overhead, not just physical motion.Source: Findings · p. 66

Related papers

Loading 2610.22471v1…