PaperlayerView exact revision on arXiv

Source-linked AI summary

Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies

Galbot Team, Xuchuan Chen, Xiaoqian Cheng, Yu Deng, Lihe Ding, Shaocong Dong, Xiangjun Gao, Haozhe Jia, Zekai Li, Zhoujian Li, Yunrui Lian, Sikai Liang, Chenghuai Lin, Dairu Liu, Jiahang Liu, Qingtao Liu, Yuxuan Ma, Zekun Qi, Jiayi Su, He Wang, Ruochen Xu, Tianyu Xu, Xudong Xu, Zhe Xu, Mi Yan, Siming Yan, Li Yi, Ruixi Yu, Jinlu Zhang, Yintianrun Zhang, Zhikai Zhang, Zhizheng Zhang, Yixin Zheng, Weiyi Zhu

arXiv:2609.38537v1cs.RO

TL;DR

Research demonstrations motivate a systematic assessment of what GPT-6 Astra can do as an embodied policy, where it remains unreliable, and what resources its decisions require. The paper evaluates direct and hybrid control across six domains, finding substantial but uneven capabilities alongside a gap between useful task decisions and reliable physical control.

  • Problem

    Astra demonstrations motivate a systematic assessment of its task capabilities, reliability boundaries, and inference resource demands as an embodied policy.

    Source: §1 · p. 1

  • Method

    The paper evaluates Astra across six robot domains, comparing its generated commands with cooperation involving learned policies or whole-body controllers.

    Source: §1 · p. 1

  • Results

    Results are substantial but uneven across domains; on RoboLab, Direct succeeds on 49/50 trials (98%), compared with 18/50 (36%) for π0.5.

    Source: §1 · p. 2, §3.4 · p. 5

  • Takeaways & Limitations

    The findings expose a gap between useful task decisions and reliable physical effects, with effective cooperation requiring Astra to know when to intervene and when to return control to a policy.

    Source: §10 · p. 20

    Dexterous manipulation remains limited by grasp acquisition and completion checking, and Astra still relies heavily on capable policies to generate viable action candidates.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

GPT-6 Astra exhibits a remarkable ability to generate numerical robot actions, extending its role beyond high-level planning. To assess Astra's capabilities as general-purpose embodied policies, we conduct comprehensive evaluations across six domains, examining direct control, cooperation with learned policies, and feedback-driven adaptation. In gripper manipulation, Astra can correct task targets and prepare contact conditions for subsequent policy execution; hybrid control with {\pi}0.5 achieves 48% success on the evaluated RoboDojo subset. In dexterous manipulation, hybrid control achieves 50% success in ten experience-guided DexJoCo trials, while direct in-hand control struggles to coordinate finger contacts. In mobile manipulation, hybrid control reaches 38.7% success on the evaluated RoboCasa365. In navigation, Astra leads our local comparisons, reaching 92% success on RxR instruction following and 82% on HM3D object search, although search incurs substantial detours. In locomotion, dense motion-reference generation remains unreliable: none of five sequential attempts on a single obstacle course reaches the goal, despite improvements in stability and forward progress. In humanoid loco-manipulation, Astra exceeds baseline methods on 13 of 30 HumanoidBench tasks with pretrained whole-body controllers. These findings reveal a gap between useful task decisions and reliable physical control. Inference latency further constrains practical control: across 50 RoboDojo instances per condition, policy-assisted and direct control consume 624.8 million and 1.132 billion tokens. A 30-second locomotion run requires 250 model calls averaging 39.86 seconds each, with physics paused during inference.

1 Introduction

The introduction evaluates GPT-6 Astra as an embodied policy across six robot domains, examining task outcomes, capability boundaries, and inference demands. It reports useful but uneven performance and analyzes how control interfaces, learned policies, and feedback shape results.

1 Introduction

  • The assessment is motivated by demonstrations of Astra in scene understanding, real-to-sim workflows, and robot manipulation, and asks what tasks it can perform, where it remains unreliable, and what resources its decisions require.The introduction emphasizes accounting for both successes and failures.Source: §1 · p. 1
  • The evaluation covers gripper, dexterous, and mobile manipulation; navigation; locomotion; and humanoid loco-manipulation, using task outcomes, comparisons, failure analysis, and inference demands.It also examines how interfaces, whole-body controllers, and feedback shape outcomes.Source: §1 · p. 1
  • Across the evaluated domains, capabilities are substantial but uneven: navigation and hybrid manipulation show strengths, while in-hand control, locomotion, and some policy-assisted cases remain unreliable.Token use and inference latency also qualify the performance results.Source: §1 · p. 2
  • Task success alone does not identify which responsibilities Astra assumes or which capabilities interfaces and whole-body controllers supply.In successful manipulation cases, Astra can prepare conditions for a policy’s next action without generating the full trajectory.Source: §1 · p. 2

2 Evaluation Framework

The framework describes how Astra selects robot commands and how the studies compare outcomes, account for system configuration, and measure progress and timing.

2.1 How Astra Produces Robot Actions

  • An action is a command submitted through the robot interface; Astra selects one from permitted observations, receives feedback, and chooses the next action, while the interface checks bounds and the environment determines success.Direct omits learned policies and whole-body controllers; Hybrid adds one of these components.Source: §2.1 · p. 2

2.2 What the Comparisons Establish

  • The studies distinguish fixed-configuration evaluations from development and transfer trials, and shared-start comparisons allow outcomes to be compared on the same task instance.Performance also depends on the complete system configuration, including commands, written guidance, and whole-body controllers, even when Astra’s weights remain fixed.Source: §2.2 · p. 3

2.3 Measuring Progress and Timing

  • Progress metrics are chosen to reflect intended physical changes, while robot-control frequency and Astra’s reasoning speed are reported separately.Resource summaries distinguish control steps, action segments, model requests, and cached input, uncached input, and output tokens; physics may pause during inference.Source: §2.3 · p. 3

3 Gripper Manipulation

The gripper studies compare direct Astra control with hybrid control on RoboDojo and RoboLab. Hybrid leads on the evaluated RoboDojo subset, while Direct leads on RoboLab; the discussion relates this difference to task demands and policy fit.

3.3 RoboLab: zero-shot evaluation

  • Direct succeeds on 49/50 RoboLab trials (98%), versus 46/50 (92%) for Hybrid and 18/50 (36%) for π0.5.Cosmos3-Nano-Policy succeeds on 18/50 (36%) and DreamZero on 17/50 (34%); Hybrid remains three successful trajectories below Direct.Source: Figure 2 · p. 4, §3.3 · p. 5

3.1 Control and evaluation setup

  • The studies use different evaluation settings: RoboDojo compares paired bimanual runs on ten selected tasks, while RoboLab tests zero-shot transfer on ten single-arm tasks without training data for its test tasks.RoboLab policy baselines use DROID-trained weights, unlike RoboDojo’s task-finetuned π0.5.Source: §3.2 · p. 4, §3.3 · p. 5

3.2 RoboDojo: paired bimanual evaluation

  • Hybrid succeeds on 24/50 RoboDojo instances (48%) versus 13/50 (26%) for Direct, with mean Score 62.60 versus 37.81.Direct and Hybrid use paired task instances, scene configurations, and random seeds.Source: §3.2 · p. 4, Table 2 · p. 5
  • Hybrid’s RoboDojo gains cluster in sequence imitation (53 vs 0), tower construction (64 vs 12), clothes folding (100 vs 40), and bottle disposal (100 vs 36).Direct is higher on language-based classification (60 vs 38) and object classification (100 vs 71); 85.6% of Hybrid steps follow π0.5 and 14.4% are generated or corrected by Astra.Source: §3.2 · p. 5
  • In the 50 Hybrid trajectories, 85.6% of executed control steps follow π0.5, while 14.4% are generated or corrected by GPT-6 Astra.The setup selects ten RoboDojo tasks by stratifying published π0.5 success rates and uses task-finetuned π0.5 weights.Source: §3.2 · pp. 4–5

3.4 Discussion: why the benchmark rankings differ

  • RoboDojo’s bimanual, long-horizon, deformable, and contact-sensitive tasks benefit from policy interaction patterns and coordinated motion alongside Astra’s target, geometry, or progress corrections.RoboLab instead consists predominantly of single-arm semantic pick-and-place tasks, where Direct can plan from the current observation and π0.5 alone succeeds on 36% of trials.Source: §3.4 · p. 5

4 Dexterous Manipulation

Dexterous manipulation tests both policy-assisted task execution and direct in-hand control. Astra can use feedback for targeted corrections, but reliable grasping and coordinated object motion remain difficult.

4 Dexterous Manipulation

  • The dexterous studies test acquiring and using a grasp as well as changing finger contacts while maintaining object support.The benchmark uses simulated Sharpa hands and compares Direct, standalone π0.5, and Hybrid on five unseen cases per task.Source: §4 · p. 5, §4.1 · p. 6

4.1 Policy Assistance Across Ten Tasks

  • Hybrid’s mean Score is 61.6, versus 44.2 for π0.5 and 16.6 for Direct; it scores above Direct on all ten tasks and above π0.5 on eight.The 0–100 scores give partial credit for task progress, averaged equally across tasks.Source: §4.1 · p. 6, Table 3 · p. 6

4.2 Feedback-Guided Bimanual Manipulation in DexJoCo

  • 5/10 experience-guided DexJoCo trials succeed; 26.4% of 11,320 executed steps are Astra corrections, with the remainder unmodified policy actions.Astra sometimes delays release until the disk contacts its support, then returns control to π0.5 for release and withdrawal.Source: §4.2 · p. 7

4.3 Stable Grasping, Limited Rotation

  • RL tracks rotation within tolerance for 76.90% of cylinder steps and 63.50% of cuboid steps, versus 0.51% and 4.40% for Astra.Both controllers retain the object through all five cuboid horizons, but Astra rotates it too slowly; periodic orientation error should be read alongside axial speed.Source: §4.3 · p. 8, Figure 6 · p. 9
  • Astra succeeds in 1/5 translation cases versus RL’s 4/5, and in 0/5 combined translation-and-rotation cases versus RL’s 4/5 at matched endpoints.Astra retains the object in every translation run, so missed targets are not explained by object drops alone.Source: §4.3 · p. 8

4.4 Contact Changes and Object Support

  • Astra’s corrections can prepare conditions for a policy’s next action, but preserving a grasp can impede contact changes needed for continued manipulation.The studies distinguish useful task correction from reliable coordination of fingers that must reposition while maintaining support.Source: §4.4 · pp. 8–9

5 Mobile Manipulation

The RoboCasa365 evaluation compares Direct Astra, Hybrid with optional π0.5 assistance, and standalone π0.5 on matched mobile-manipulation starts. Hybrid’s advantage is uneven across task groups, and policy access does not preserve every standalone-policy success.

5.1 Control Conditions and Shared Starts

  • The fixed evaluation includes 75 episodes per condition, with shared initial-state fingerprints, robot, controller, task horizon, and native success check.Astra receives three RGB views and robot state; Hybrid adds a frozen multitask π0.5 checkpoint without task-specific finetuning.Source: §5.1 · p. 9

5.2 Uneven Gains from Policy Assistance

  • Hybrid completes 29/75 episodes, compared with 25/75 for Direct and 17/75 for standalone π0.5.Hybrid exceeds Direct on seen atomic and composite tasks, while Direct is stronger on unseen compositions.Source: §5.2 · p. 10, Table 6 · p. 11
  • Hybrid exceeds Direct on seen compositions, but Direct succeeds on 56% of unseen compositions versus 36% for Hybrid.Hybrid and Direct also have complementary successes: Hybrid alone succeeds on 13 paired starts and Direct alone on nine.Source: §5.2 · p. 10

5.3 Recovery and Lost Policy Successes

  • Only standalone π0.5 succeeds on CoffeeSetupMug/03; across five seeds, both Astra conditions fail CoffeeSetupMug and WashLettuce while π0.5 completes three and two cases, respectively.Visual feedback can reveal some action errors, but recognizing a problem does not ensure a successful correction.Source: §5.3 · p. 10

6 Navigation

Astra navigates from RGB observations without pose or map input, using discrete moves or local waypoints. It performs strongly on the evaluated subsets, though ObjectNav path efficiency reveals costly detours.

6.1 RGB Navigation on Four Fixed Subsets

  • Astra chooses movement and stopping from RGB observations plus an instruction or object category, without GPS, compass, depth, semantic labels, goal coordinates, or reference routes.The adapter executes 0.25 m forward steps and 30° turns under a 500-primitive budget.Source: §6 · p. 11, §6.1 · p. 11

6.2 Success and Search Efficiency

  • Astra completes 39/50 R2R and 46/50 RxR episodes, with nDTW of 72.20% and 84.73%, respectively.The evaluation also includes MP3D and HM3D ObjectNav, with 50 episodes per subset.Source: §6.2 · p. 11, §6.1 · p. 11
  • Astra completes 46/50 RxR episodes and 41/50 HM3D ObjectNav episodes; ObjectNav SPL is 22.43% on MP3D and 43.69% on HM3D.The lower SPL indicates inefficient search despite successful endpoints; ten MP3D and five HM3D episodes exhaust the 500-step budget.Source: §6.2 · p. 11, Figure 10 · p. 13

6.3 Local Comparisons and Failure Modes

  • Under the local protocol, Astra has higher SR and SPL than evaluated systems on all subsets with reported results; its SR gains over LightNav-0 are 20, 20, 36, and 16 percentage points.LightNav-0 has slightly higher R2R nDTW, 72.77% versus 72.20%, while Astra has higher sDTW, 59.35% versus 48.89%.Source: §6.3 · p. 12, Table 7 · p. 13

7 Locomotion

The locomotion study tests Astra-generated dense motion references with a frozen tracker and examines compact-reference interfaces using analytic generators. Astra makes progress over sequential attempts but does not reach the goal on the tested course.

7.1 Dense Motion on a Matched Course

  • Astra replaces PASSAGE’s learned motion generator with 25 × 65 scene-aligned references for a frozen G1 ScaleTrack controller, specifying 1,625 values per call.Each proposal covers 0.5 seconds at 50 Hz and includes root motion and positions and velocities for 29 joints.Source: §7.1 · p. 13

7.2 Persistent Locomotion Failures

  • None of Astra’s five sequential attempts reaches the goal; the fifth remains 6.519 m away after traveling 3.833 m and reducing goal distance by 2.598 m.The fifth attempt stays upright for 30 seconds, but remains before the principal obstacle field; these attempts form one adaptation trajectory, not independent trials.Source: §7.2 · pp. 13–14, Table 9 · p. 15
  • The final 30-second episode uses 250 synchronous calls averaging 39.86 seconds of recorded latency, while physics pauses during inference.PASSAGE planning takes approximately 0.08 seconds; the inference budget permits five attempts.Source: §7.2 · pp. 14–15

7.3 Compact-Reference Diagnostics without Astra

  • Neither compact-reference interface achieves strict success; Five-point reaches and holds the endpoint in 4/12 rollouts and WholeBody-14 in 3/12.The comparison favors the tested Five-point generator–interface pair descriptively, without isolating dimensionality because the generators differ.Source: §7.3 · p. 15, Table 10 · p. 15

8 Humanoid Loco-Manipulation

The section evaluates humanoid command selection, experience-guided task transitions, and conversion of egocentric video into interactive scenes and motion references. Across these studies, useful local decisions do not ensure reliable task completion or repeatable handover.

8.2 Command Selection with Pretrained Whole-Body Controllers

  • 1358.8 mean return lets Maze reach native stage 4, while Reach averages 11430.2 below its 12000 threshold.Walk, Run, and Crawl returns are 848.7, 642.1, and 971.9; Crawl averages about 27.39 m displacement per run.Source: §8.2 · pp. 15–16
  • 705.72 to 746.60 mean return results from raising scripted Walk gait frequency from 1.2 to 1.8 Hz across eight paired seeds.Weights, command speed, and feedback rule stay unchanged; measured forward speed is about 1.48 m/s for a 1.25 m/s command.Source: §8.2 · p. 16

8.3 Contact Failures Between Task Stages

  • 4.89 and 4.97 cm box errors accompany successful evaluation pushes in 277 and 357 controls, while the diagnostic reaches 4.97 cm only when both entry and contact references change.The separate guided configuration succeeds on three development seeds and one prespecified transfer seed with errors below 5 cm.Source: §8.3 · p. 16
  • Door episodes rotate the hatch without opening a usable passage, while Stair runs fall or retreat and Cube and Window lose objects early.Whole-body controllers alone do not resolve failures between task stages when posture or contact changes disrupt the next action.Source: §8.3 · p. 16

8.2 Command Selection with Pretrained Whole-Body Controllers

  • 581.9 is Astra’s highest mean native return across 30 HumanoidBench tasks, exceeding the best of three published references on 13 tasks and tying on Kitchen.The compared means are 338.2 for TD-MPC2, 42.7 for SAC, and −41.9 for DreamerV3; six Astra task means meet native thresholds.Source: §8.2 · p. 16, Figure 12 · p. 17

8.4 Refining Task Guidance through Experience in SIMPLE

  • 83.3% success across 60 trials is achieved with experience-refined guidance in SIMPLE, strongest on tabletop and picking tasks and weaker on mobile pick/place.Astra reviews each trial and revises written instructions and notes while both models’ weights remain fixed.Source: §8.4 · pp. 16–17

8.5 Why Handover Remains Fragile

  • A successful handover is not reliably retained: further reflection can precede failure, a fixed procedure passes the additional hold check on two of three scenes, and all three valid recovery attempts fail.Native success does not require the carton to remain stable after complete release; grasp reliability still depends on Astra’s observations and actions.Source: §8.5 · pp. 17–18

8.6 Scene and Motion from Egocentric Video

  • 12.14 mm scene Chamfer-L1 error follows image-based refinement and CAD dimensions, but only one of three apple-transfer cases completes the interaction.Reusing a bottle position displaced by a few centimeters causes grasp failures; the study identifies preserving interaction structure through reconstruction and retargeting as the central challenge.Source: §8.6 · pp. 18–19

9 Discussion

The discussion interprets Astra’s uneven results through the full control path, intervention and verification, repeatability, and resource demands. Useful decisions and feedback do not consistently yield successful, efficient physical behavior.

9 Discussion

  • Strong navigation and useful manipulation coexist with persistent contact-coordination, locomotion, and completion-verification difficulties.The discussion examines how divided control and feedback relate to the gap between local decisions and successful, efficient task behavior.Source: §9 · p. 19

9.1 Capability Depends on the Complete Control Path

  • A command’s physical meaning depends on its interface and controller, so Astra’s decisions must be evaluated alongside the motion supplied by other components.In-hand grasp retention can impede repositioning, and a recognizable navigation landmark need not establish location or justify stopping.Source: §9.1 · p. 19

9.2 Intervention and Verified Progress

  • Astra can prepare conditions for a policy without constructing its trajectory, but RoboCasa Hybrid both completes some policy failures and fails on some policy successes.Error detection, correction, and verification are distinct demands; the physical outcome, not action-source counts or plausible explanations, determines correction value.Source: §9.2 · pp. 19–20

9.3 Retaining Success across Trials

  • Feedback can improve guidance on a known case without ensuring repeatability or transfer, so changes to prompts, interfaces, and controller calibration require repeated and additional-state tests.RoboDojo Hybrid reduces recorded tokens by 44.8%, while RoboCasa Hybrid makes fewer Astra-authored action segments but more model requests than Direct; review frequency affects inference efficiency.Source: §9.3 · p. 20, §9.4 · p. 20

9.4 Resource Costs and Capability Limits

  • Inference time must be measured separately from motion time because physics pauses during model inference and a physical scene cannot be assumed to wait.Long histories and repeated review increase resource use; cooperation can reduce motion authorship without reducing proposal-review work.Source: §9.4 · p. 20

10 Conclusion

The conclusion presents a six-domain evaluation of Astra as an embodied policy and emphasizes the gap between useful decisions and reliable physical effects. The appendices document interfaces, evaluation records, task coverage, and supporting results.

10 Conclusion

  • Astra shows strengths in navigation and task-oriented manipulation, but direct in-hand control, dense-reference locomotion, and policy assistance on some standalone-policy successes remain unreliable.Learned policies and whole-body controllers help bridge the decision-to-physical-effect gap, while token use and inference latency constrain practical control.Source: §10 · p. 20

A.1 Gripper Interfaces

  • The interfaces specify distinct Direct and Hybrid action paths, permitted observations, command safeguards, and frozen-controller inputs for gripper and humanoid control.RoboDojo Hybrid accepts a one-to-fifteen-step π0.5 prefix or replaces it with a one-to-five-step correction; HumanoidBench commands velocity or sparse body targets to frozen controllers.Source: §A.1 · p. 25, §A.2 · pp. 25–26

B.1 RoboDojo

  • RoboDojo Direct and Hybrid use the same task instances, scenes, and seeds, with five evaluation instances per task; RoboLab permits repeat attempts for Astra under a 500-decision budget on two Direct attempts.RoboLab uses five outcomes per task for each method and final task success, with zero-shot DROID-trained policy weights rather than RoboDojo’s task-finetuned policy.Source: §B.1 · p. 26, Table 14 · p. 26, §B.2 · p. 26

C.1 Focused Push and Walk Records

  • 4.97 cm final box error occurs when the scripted Push diagnostic changes both entry state and contact references; changing either alone performs worse.The four errors are 27.02 cm with neither changed, 36.81 cm with entry only, 23.11 cm with contact references only, and 4.97 cm with both.Source: §C.1 · p. 27

C.2 Independent Reach Records

  • 15675.19 and 15380.91 native returns in two separate Reach records both exceed the 12000 threshold, but these records are outside the task-suite sample.The hand remains within the 5 cm target radius for 11.36 s and 12.78 s, respectively.Source: §C.2 · p. 27

C.4 Source Snapshots and Verification

  • The audit checks 270 hashes across 90 online HumanoidBench episodes plus ten further runs; reproduction requires archived manifests, prompts, traces, and robot runtime.Replay verifies recorded dynamics and rewards.Source: §C.4 · p. 27

D.2 Behavioral Case Index

  • The 21 recorded manipulation cases are grouped into goal alignment, contact preparation, terminal-state checking, Hybrid contact and feedback failures, and Direct strategies and local recovery.The cases illustrate observed trajectories, and interpretations use complete trajectories rather than still images.Source: §D.2 · p. 28

E Humanoid Loco-Manipulation: Supporting Results

  • 13 of 30 HumanoidBench tasks exceed the best of three published references, with Kitchen tied; the comparison uses published mean returns and counts only strictly higher Astra means.The appendix reports means ± sample SD over two seeds for all 30 tasks; six Astra task means meet native thresholds.Source: E Humanoid Loco-Manipulation:… · p. 28, Table 18 · p. 29

E.2 SIMPLE Development and Retention Checks

These appendices distinguish SIMPLE development and retention checks from the official evaluation, then document focused guidance experiments and the dexterous-manipulation task and policy setup. The studies keep model weights fixed while changing or evaluating guidance and control procedures.

E.2 SIMPLE Development and Retention Checks

  • Nine of ten SIMPLE task-development endpoints meet native success, and eight also meet execution, audit, and additional hold requirements; these summarize development, not 58 independent trials.Astra revises Skill and Memory documents after episodes while Astra and ScaleBFM remain frozen; Handover reaches native success but times out before the hold check.Source: §E.2 · p. 30
  • 2/5 and 3/5 stabilization variants pass native success and the additional hold check; a fixed procedure passes both criteria on two of three previously successful scenes.The separate handover conditions include carton recovery and instruction adaptation, and are excluded from the L2 evaluation total.Source: §E.2 · p. 30, Table 19 · p. 30
  • 50 of 60 SIMPLE L2 trials meet official success criteria, for an overall 83.3% success rate across six tasks and ten scenes per task.Separate Handover development and retention studies are not included in this total.Source: §E.2 · p. 30

F.1 Contact-Conditioned Guidance

  • −148.56 to 863.90 return and 29.45 to 4.88 cm final box error follow contact-conditioned Push guidance on development seed 3, succeeding after 428 controls.The prompt falls from 129 to 90 lines; seed 5 fails at 500 controls with 27.02 cm error, and each condition has one online attempt on known development seeds.Source: §F.1 · p. 30

F.2 A Saved Diagnosis and a Failed Repeat

  • 381 controls produce one successful Push repeat after a saved diagnosis, but the next repeat fails after 500 controls.The successful trial has return 803.86 and the failed repeat −259.83; the archive does not identify who authored the later guidance change.Source: §F.2 · p. 31

F.3 Finger Control Does Not Solve Door

  • Five full-closure commands do not complete Door: the episode ends at return 144.38 against a threshold of 600, with no passage opened.Guidance, observations, and action access change together in this condition.Source: §F.3 · p. 31

G.1 Task Suite and Demonstration Data

  • The dexterous suite has ten tasks with specified instructions, simulation budgets, scene configurations, and partial-credit subgoals; one pooled demonstration-trained π0.5 policy serves standalone and Hybrid conditions.The evaluated scene configurations vary object location, applicable yaw, and initial hand posture.Source: G Dexterous Manipulation: Data,… · p. 31, §G.1 · pp. 31–32, §G.2 · p. 32

G.3 Observations, Numerical Actions, and Execution

  • π0.5 receives head and wrist RGB plus a 62-dimensional state, and outputs two wrist poses and 22 joint angles per hand; task limits use simulation time, separate from model-call and inference wall time.Each action dimension is 2 × (3 + 6) + 2 × 22 = 62; the baseline replans after discarding each unexecuted prediction suffix.Source: §G.3 · p. 32

G.4 Evaluation Protocol and Scoring

The appendix specifies RoboCasa scoring and safety checks, then documents the paired in-hand benchmark, controllers, inference protocol, and evaluation metrics.

G.4 Evaluation Protocol and Scoring

  • Five fixed cases per task yield 50 evaluation cases across ten tasks, with all three methods tested on the same case set.Placement and rearrangement completion requires release and stable object positions for 2 s; lifting requires a stable 3 s hold at least 10 cm above initial height.Source: §G.4 · pp. 32–34
  • Each task contributes equally to the 0–100 partial-credit score, which is not a success rate.Full credit requires all subgoals and continuous verification; partial tiers reflect terminal subgoals, with a separate 5 cm lift event recording earlier progress.Source: §G.4 · pp. 33–34

H.1 Benchmark Definition

  • Four paired dexterous tasks test continuous Sharpa rotation and Allegro cylinder translation, with or without a −40° long-axis rotation.Each task uses five paired starts; rotation tasks draw on ConTrack and mjlab_hand, while translation follows tactile in-hand manipulation.Source: Table 23 · p. 35, §H.1 · p. 35, Figure 18 · p. 35

H.2 Task-Specific RL Controllers and Training

  • Frozen task-specific RL policies provide the comparison, including a released translation checkpoint and three policies trained for the study.The evaluated policies receive no adaptation or action tuning; the cuboid policy is initialized from the cylinder policy’s actor and observation normalizer.Source: §H.2 · pp. 35–37, Table 24 · p. 36

H.3 Astra Inference Protocol

  • Astra receives synchronized RGB, joint, object, target, and contact observations, then issues bounded joint-target actions held for one to five control steps.Its decision limit is 100 decisions on Sharpa and 120 on Allegro; because commands can span multiple control steps, the limit can arrive before the task horizon.Source: §H.3 · p. 37

H.4 Pairing and Evaluation Protocol

  • Paired evaluation fixes each controller pair’s physical start and goal while retaining their intended observations, architectures, and inference schedules.Translation-and-rotation results distinguish Astra’s decision-limit endpoint, matched-duration RL, and full-horizon RL.Source: §H.4 · pp. 37–38

H.5 Evaluation Metrics

  • In-hand metrics combine orientation error, At-goal rate, axial-speed error, completion, and drops; translation success uses terminal position and rotation thresholds.Periodic orientation agreement can occur without sustained rotation, so orientation error must be interpreted alongside axial speed; reward is not compared across tasks.Source: §H.5 · p. 38

H.6 Complete Per-Case Results

The appendix reports per-case dexterous results and documents the coverage, interfaces, and execution rules for navigation comparisons.

H.6 Complete Per-Case Results

  • Table 25 covers five held-out cylinder starts over a 20-s rotation window, while Table 26 covers five cuboid starts over 10 s, four overlapping RL initialization.Rotation means pool executed steps, and early-ending trajectories are not padded.Source: Table 25 · p. 38, §H.6 · p. 38, Table 26 · p. 39
  • Translation rollouts had no object drops, and Table 27 evaluates terminal cylinder position at the complete 15-s endpoint.Figure 19 reports per-case rotation metrics; reward comparisons are valid only within a task row.Source: §H.6 · p. 38, Figure 19 · p. 39, Table 27 · p. 40

I.1 Navigation Coverage and Termination

  • Navigation evaluation uses four fixed 50-episode subsets, whose balanced sampling means reported averages characterize those subsets rather than source episode distributions.HM3D’s 1 m results rescore saved trajectories rather than rerunning the policy.Source: §I.1 · p. 39

I.2 Released Navigation Policies: Interfaces and Complete Results

  • Released navigation policies use fixed episodes and checkpoints; Astra and two policies share a front camera, while OmniNav Flow uses three views.The adapters convert model outputs to common navigation primitives, with collisions interrupting action sequences and budget exhaustion not forcing STOP.Source: §I.2 · pp. 40–41
  • Published navigation references cover full validation benchmarks, whereas Astra’s rows use 50 fixed episodes per dataset and differing protocols.Tables 32 and 33 preserve each source’s coverage rather than impose a common denominator.Source: §I.2 · p. 42

I.3 RoboCasa Task Results and Paired Outcomes

This appendix records RoboCasa paired-outcome conditions and audits the information access, harness configurations, and request accounting used in evaluation.

I.3 RoboCasa Task Results and Paired Outcomes

  • Among 16 pairs where both Astra conditions succeed, Hybrid averages 1,110.4 steps versus Direct’s 1,256.6, using fewer steps in 11 pairs.For the Hybrid–Direct atomic-group comparison, Bonferroni correction over three task groups changes the nominal p-value from 0.03125 to 0.09375.Source: §I.3 · p. 42
  • An additional multitask-checkpoint evaluation reports 403/2,500 successes across 50 tasks and 132/750 on the matching 15-task subset.Those rates are 16.12% and 17.6%, respectively.Source: §I.3 · p. 43

I.4 RoboCasa Control Configurations, Information Access, and Resources

  • The two Astra harness conditions differ in context compaction, proposal handling, and action-call gating, and resumed trajectories may span configurations.Request logs cover saved requests, including recovery history, but exclude discarded attempts and do not total all inference calls.Source: §I.4 · p. 44, Table 36 · p. 45, Table 37 · p. 46
  • The 150-session audit found no tool access to hidden object truth, complete physics state, reference trajectories, or action-guiding scores.Proprioception was supplied, pretraining was outside the audit, and both Astra conditions paused physics during reasoning.Source: §I.4 · p. 44
  • Policy-only execution takes a median 121.6 s after loading and scene reconstruction, excluding queueing, initialization, and infrastructure time.Astra timing includes inference and recovery waits.Source: §I.4 · p. 44

Related papers

Loading 2609.38537v1…