PaperlayerView exact revision on arXiv

Source-linked AI summary

$τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation

Xiaowei Cai, Yunuo Cai, Bingao Chen, Jingxiao Chen, Zhi Chen, Siyuan Feng, Tengyu Hou, Jingshun Huang, Han Jiang, Runkun Ju, Dong Li, Mingxiang Li, Shaowei Li, Xinchen Li, Yifan Li, Yi Liu, Zhongyuan Liu, Jianlan Luo, Junwen Miao, Ruiqi Ni, Buqing Nie, Mingjie Pan, Xinlin Ren, Jianheng Song, Jiaxu Wang, Peiqi Wang, Sen Wang, Xiaoyan Wang, Dafeng Wei, Dongming Wu, Pengwei Xie, Pu Yang, Hangjian Ye, Xiangyu Yue, Jinyu Zhang, Qinglin Zhang, Xueyong Zhao, Pengfei Zhou, Yue Zhou

arXiv:2608.16885v1cs.RO

TL;DR

Long-horizon robot manipulation requires coherent subtask sequencing, while fixed-budget hierarchical VLA models cannot allocate computation to difficult choices. τ0-VLA uses execution memory and world-model-guided test-time search to compare alternatives before commitment, with a unified low-level policy executing across embodiments. Additional computation improves next-subtask prediction across in-domain and shifted settings and yields higher long-horizon task success.

  • Problem

    Many hierarchical VLA systems use a fixed inference budget for high-level decisions and do not compare alternative subtasks through their expected physical consequences.

  • Method

    τ0-VLA combines a memory-augmented high-level policy with world-model-guided test-time search and a unified low-level policy trained across robot embodiments.

  • Results

    Additional test-time computation improves next-subtask prediction across evaluation settings and increases closed-loop success on long-horizon real-robot tasks.

  • Takeaways & Limitations

    Allocating computation to consequential high-level decisions provides a practical way to improve long-horizon robot manipulation while retaining stable low-level execution.

Abstract

from arXiv · show

Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks. Most hierarchical vision-language-action (VLA) models make each such decision with a single forward pass, leaving no mechanism to allocate additional computation to difficult or consequential choices. We introduce $τ_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem through world-model-guided test-time computation. At each inference step, the high-level policy uses execution memory to generate a subtask and, when needed, searches over alternatives before committing to its output. A low-level policy then executes the generated subtask across multiple robot embodiments. The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training. Across in-domain and distribution-shifted settings, allocating additional test-time computation substantially improves next-subtask prediction accuracy, and these gains translate into higher closed-loop success on long-horizon robot manipulation tasks.

I. INTRODUCTION

Long-horizon manipulation requires coherent subtask decisions because robots must track progress, verify outcomes, and recover from failures. τ0-VLA makes high-level subtask generation compute-scalable, using memory and world-model-guided search while a unified low-level policy executes across embodiments.

  • Motivation: Long-horizon tasks require consequential subtask sequences, not merely precise motor-command execution.An incorrect subtask may be executed perfectly yet still derail the task.
  • Motivation: Many hierarchical VLA systems use a fixed inference budget and neither compare alternative subtasks nor evaluate their expected physical consequences.This leaves high-level decisions without consequence-grounded alternative evaluation.
  • Core idea: τ0-VLA treats next-subtask generation as an inference-time reasoning problem whose computation scales with decision difficulty.Subtasks provide a temporally extended unit for consequence-aware reasoning between action-level control and language-only planning.
  • Method: The high-level policy uses observation and execution memory, invokes extra reasoning when uncertain, and evaluates candidate subtasks through predicted outcomes.Its propose–predict–evaluate procedure uses candidate generation, world-model prediction, value scoring, beam search, and reflection.
  • Method: The low-level policy uses a unified control interface across robot embodiments and is trained on approximately 40,000 hours of heterogeneous robot data.This supports fixed-base manipulation, bimanual coordination, and mobile whole-body control.
  • Evaluation: Across real-world long-horizon tasks lasting up to 12 minutes, hierarchical test-time computation improves task success and next-subtask prediction accuracy, with larger budgets yielding further gains.The evaluation includes room cleaning, meal preparation, tea making, and laundry collection across multiple embodiments.

C. World Models and Test-Time Computation

τ0-VLA distinguishes which subtask to execute by searching open-ended language-subtask branches and evaluating their predicted terminal images. It combines this search with execution memory and hierarchical control so decisions can be grounded in anticipated physical consequences.

  • Positioning: τ0-VLA searches open-ended language subtasks, evaluates candidates through predicted terminal images, and uses retained branches with correctable execution memory.A reflective model generates the final subtask from the retained branches.
  • Hierarchical control: A VLA policy maps the current multi-view observation, proprioceptive state, language command, and embodiment metadata to an H-step action chunk.The metadata specifies embodiment, control mode, and whole-body configuration.
  • Hierarchical control: Hierarchical control separates high-level subtask generation from low-level action generation, with the low-level policy conditioned on the generated subtask.Execution continues with the current subtask unless the high-level output changes.
  • Hierarchical control: The high-level policy determines the currently appropriate subtask, while the low-level policy determines how that subtask is executed.This division assigns semantic sequencing and continuous control to separate policy levels.

IV. METHOD

τ0-VLA applies a sequential hierarchy in which a memory-maintaining high-level policy routes uncertain decisions to world-model-guided beam search, while a low-level policy executes the selected subtask. Search expands, scores, prunes, and reflects over candidate subtask branches before producing the final command.

  • Pipeline: τ0-VLA maintains execution memory, chooses whether to invoke test-time computation, and generates a subtask before low-level action generation.The hierarchy supports long-horizon progress tracking and consequence-aware planning without changing the low-level control interface.
  • Test-time computation: Unlike fixed-compute inference, uncertain predictions trigger world-model-guided beam search over possible subtask sequences.The procedure expands branches, predicts and scores visual outcomes, then uses a reflective model to generate the final subtask.
  • Routing: The proposal model generates a direct subtask and token confidences route inference either to a fast path or to test-time computation.The router sets g_t to select the fast route or invoke TTC.
  • Search: When invoked, SEARCH performs beam search over candidate subtask sequences using branching factor N, beam width B, and search depth D.Each branch stores proposal context, imagined subtask path, cumulative score, and—after the root—its predicted terminal image.
  • Search: Search repeatedly samples proposals for retained branches, predicts branch outcomes, ranks children globally, and retains the top B until depth D.Retained predicted images and branch-local memories provide context for subsequent expansions.
  • Reflection: The reflective model conditions on retained branch summaries and real observation-aligned context to generate the final subtask passed to the low-level policy.Its output may differ from the retained candidate set and does not update persistent execution memory.

C. Low-Level Policy

The low-level policy combines a vision-language backbone with a Mixture-of-Transformers action expert and supports heterogeneous embodiments through a masked unified action space.

  • The action expert and vision-language backbone interact through joint attention while remaining separately parameterized Transformer streams.
  • The policy conditions on multi-view observations, proprioceptive state, and a language command to generate executable action chunks.
  • Unified state and action space: Heterogeneous embodiments share a 40-dimensional state and action space with per-sample masks selecting valid channels.
  • Masked flow matching: Masked flow matching supervises active channels and projects inactive channels before velocity-field evaluation and final output.

D. System Inference

System inference alternates between direct subtask proposals and confidence-triggered search, then executes the selected subtask and feeds the resulting observation back into memory.

  • At each inference step, the proposal model updates execution memory and generates a subtask for the closed-loop system.
  • When proposal confidence indicates uncertainty, test-time search generates candidate subtasks and a reflective model selects the final subtask.
  • The low-level policy executes an action chunk conditioned on the selected subtask, and the resulting real observation enters memory at the next step.
  • The high-level and low-level policies are pipelined asynchronously, while task-specific routing thresholds are calibrated on held-out data.
  • Search uses proposal, world, and value models with branching factor N, beam width B, and depth D as inference controls.

V. TRAINING

Training combines heterogeneous robot-action data with multimodal co-training and separately trains the proposal, world, value, and reflective components used for inference-time reasoning.

  • Low-level policy data: The low-level policy uses 40,115 hours of heterogeneous real-world data across fixed-base, mobile, and bimanual embodiments.
  • Low-level policy data: The corpus combines human demonstrations, autonomous rollouts, and UMI-style recordings with multimodal vision-language data.
  • Training stages: Knowledge-isolated co-training blocks action-loss gradients at the backbone interface while the action expert learns control-relevant representations.
  • High-level models: Proposal, value, and reflective models are fine-tuned from a shared robot-pretrained VLM checkpoint, while the world model is trained separately.
  • Offline supervision: Simulated rollouts alternate subtask proposal with visual outcome prediction and provide offline data for training value and reflective models.

VI. EXPERIMENTS

Experiments evaluate decision quality, long-horizon manipulation, and short-horizon execution across embodiments and household tasks, including controlled comparisons of hierarchical inference and execution memory.

  • The evaluation separates next-subtask prediction, test-time computation, execution memory, long-horizon performance, and short-horizon execution.
  • Robot platforms: Experiments use AGIBOT G1, ARX AC One, and a bimanual Franka Research 3 setup with multi-view RGB observations and proprioceptive state.
  • Task suite: The task suite also includes Book Organization, Collect Laundry, and three separately scored Tidy Makeup Table groups.
  • Task suite: The primary long-horizon suite includes Clean Room, Prepare Ingredients, Tomato and Egg Stir Fry, and Make Milk Tea.
  • Overall system evaluation: Explicit execution memory benefits Clean Room by retaining progress across room transitions and helps track prerequisite stages in Prepare Ingredients.
  • Overall system evaluation: TTC raises Make Milk Tea success rate from 5/10 to 7/10 and progress from more than 91% to 95.38%.

C. Evaluation Across Embodiments

The evaluation measures low-level control across embodiments and high-level test-time computation for long-horizon manipulation. TTC improves next-subtask prediction across evaluation settings, while additional computation yields rapidly increasing but eventually saturating accuracy gains.

  • Evaluation Across Embodiments: The low-level policy is evaluated separately across embodiments using direct execution without task decomposition, execution memory, or test-time search.Collect Laundry and Tidy Makeup Table assess low-level control and language-conditioned manipulation, with failures involving grasping, navigation, object selection, and drawer motion.
  • Closed-loop Evaluation: Additional test-time computation substantially improves next-subtask prediction and increases closed-loop success on long-horizon real-robot tasks.The evaluation covers Make Milk Tea, Book Organization, and Clean Room, including distribution-shifted Book Organization arrangements.
  • Open-loop Subtask Prediction: TTC achieves the highest next-subtask prediction accuracy across Make Milk Tea, in-domain Book Organization, OOD Book Organization, and Clean Room.The comparison includes Plan Once and Best-of-N as high-level inference baselines.
  • Open-loop Subtask Prediction: 74.0% accuracy for TTC on OOD Book Organization exceeds 50.0% for Plan Once and 57.5% for Best-of-N.The OOD arrangements are absent from the high-level policy’s training data, and TTC evaluates candidate consequences through recursive branch expansion.
  • Compute–Accuracy Trade-off: Accuracy rises rapidly at low computational budgets, then marginal gains decrease as saturation curves approach a plateau.The fitted trend indicates a favorable compute–accuracy trade-off at moderate budgets, with benefits eventually saturating.

APPENDIX A AUTHOR CONTRIBUTIONS

The appendix assigns contributors to high-level and low-level policy research, infrastructure, writing, data collection, and deployment. It also documents robot platforms, unified state-action representation, masking, and flow-matching implementation details.

  • Author Contributions: High-level policy training and research are attributed to Xiaowei Cai, Jingxiao Chen, Xinchen Li, and listed collaborators.The contribution list names Jinyu Zhang and Yi Liu among the high-level policy contributors.
  • Author Contributions: Low-level policy training and research are attributed to Xiaowei Cai, Bingao Chen, Jingxiao Chen, and listed collaborators.The list separately identifies contributors for low-level policy work.
  • Author Contributions: Training infrastructure, robot infrastructure, writing and illustration, data collection and deployment, and data infrastructure have separate contributor lists.These roles are enumerated across the appendix contribution statements.
  • Robot Platform Details: The appendix covers AGIBOT G1, ARX AC One, and Franka Research 3, with multi-view RGB images, proprioceptive state, and a unified 40-dimensional action layout.The platforms span wheeled humanoid, bimanual, and dual-arm configurations.
  • Policy Representation and Training: State and action vectors use embodiment-specific masks, relative end-effector and joint encoding, and a 40-dimensional canonical layout.The implementation also describes masked flow matching, noisy action-chunk initialization, and integration of the learned velocity field.

C. Physical-Robot Evaluation Protocol

The physical-robot evaluation defines prerequisite-aware progress and success criteria for long-horizon tasks, using fixed trial limits and standardized resets. Progress credits milestone completion according to ordering constraints, retries, partial states, and task-specific rules.

  • Success and progress criteria: Success requires every annotated subtask and the terminal condition, with no skipped required subtask or prohibited action.
  • Success and progress criteria: Progress uses a prerequisite graph so milestones receive credit only when their required ancestors reached the required state first.
  • Success and progress criteria: A milestone earns local credit of 1 for first-attempt completion, 0.5 after failed autonomous attempts or task-specific partial completion, and 0 otherwise.
  • Success and progress criteria: Completed retries can receive half credit while enabling later milestones, whereas skipped milestones receive no credit and block only their descendants.
  • Task protocol: The four Table I tasks contain 25, 14, 22, and 13 annotated milestones, respectively.
  • Task protocol: Each physical-robot trial has a fixed wall-clock limit, with maximum durations summarized in Table V, and standardized states restored before every trial.

D. Low-Level Data Details

The low-level policy is trained from heterogeneous robot demonstrations augmented with multimodal supervision, while high-level data are automatically constructed from instructions, demonstrations, annotations, and videos. Training also targets memory misalignment and deployment-time recovery through synthetic perturbations and filtered examples.

  • Low-level corpus: The low-level corpus combines approximately 23.4K internal and 16.7K public robot-data hours across multiple embodiments and manipulation datasets.
  • Low-level corpus: Multimodal instruction-following and robot-centric perception examples are interleaved with action data as auxiliary supervision for preserving vision-language capabilities.
  • High-level data construction: High-level supervision is automatically generated from task instructions, stage descriptions, executable subtask annotations, segmented demonstrations, and videos.
  • High-level data construction: The construction pipeline generates scene-state and execution-memory fields, extracts three synchronized views per subtask, and assembles structured VQA examples.
  • Robustness augmentation: Input-memory perturbations create five instance families targeting deployment-time memory misalignment without additional annotation.
  • Robustness augmentation: Recoverable failures retain memory for retries, progress-undoing failures trigger rollback, and rollback instances are capped at 10–15%.
  • Data filtering: A contamination filter discards 11.74% of episodes and retains 40.4M clean samples, with evaluation frames unseen during training.
  • Deployment: Deployment pipelines asynchronous high-level recomputation with low-level control by publishing generated subtasks to a per-episode cache refreshed about every 1 s.

G. Book Organization Task Settings

Book Organization isolates high-level decision making from physical execution by evaluating next-subtask predictions at annotated boundaries under fixed inputs and output formats. Predictions are judged semantically, distinguishing exact immediate transitions from valid but differently ordered steps and incorrect actions.

  • Task setting: Book Organization uses four books and four fixed slots, requiring the robot to exchange book positions until the books are ordered tallest to shortest.
  • Evaluation protocol: The high-level evaluation supplies identical task inputs and output formats while measuring predicted next subtasks at subtask boundaries.
  • Evaluation protocol: Plan Once, Best-of-N, and TTC are compared by varying only the decision-time inference procedure.
  • Semantic judging: GPT-5.4 labels predictions as equivalent, adjacent, or wrong using the task goal, ground-truth subtask, and predicted subtask.
  • Semantic judging: Equivalent predictions must describe the same immediate physical state transition, matching required arms, actions, objects, destinations, and material state.
  • Semantic judging: Only equivalent predictions count as successful, with disagreements resolved through additional temperature-zero judgments and majority voting.
  • Semantic judging: Adjacent predictions are valid preceding, following, or reorderable steps toward the same goal but differ from the ground-truth immediate transition.

J. Adaptive Routing Details

Adaptive routing decides whether to use direct execution or additional test-time search from token statistics already produced by the proposal-model forward pass. When search is invoked, candidate subtasks are expanded, visually predicted, scored, and passed to a reflective model for final generation.

  • Routing decision: At inference step t, the proposal model produces the direct subtask and memory, while routing reuses its token logits without another model invocation.
  • Routing decision: The router computes mean generated-token probability and mean logit margin over all generated tokens and the memory-field tokens.
  • Routing decision: The binary routing rule uses shared statistics and thresholds, with thresholds calibrated separately for each task on held-out validation data.
  • Test-time computation: When additional computation is selected, the system searches candidate subtasks, predicts and scores their visual outcomes, and conditions reflective generation on retained branches.
Loading 2608.16885v1…