PaperlayerView exact revision on arXiv

Source-linked AI summary

$π_{0.5}$: a Vision-Language-Action Model with Open-World Generalization

Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, Ury Zhilinsky

arXiv:2504.16054v1cs.LGcs.RO

TL;DR

How far vision-language-action models can generalize beyond the lab remains an open question. This paper introduces π0.5, which co-trains on heterogeneous data, and reports successful long-horizon, dexterous tasks in homes unseen during training.

  • Problem

    How far vision-language-action models can generalize in the real world remains open, where robots need breadth of knowledge to generalize across levels of abstraction.

    Source: §I · p. 2

  • Method

    π0.5 builds on π0 with co-training across heterogeneous sources, including data from other robots, semantic subtask prediction, human instructions, and web data.

    Source: §I · p. 2

  • Results

    π0.5 consistently succeeded on varied tasks in each home, including multi-stage tasks lasting about 2 to 5 minutes, in homes unseen during training.

    Source: A. Can π0.5 generalize to real homes? · p. 8

  • Takeaways & Limitations

    The work provides a proof of concept that generalization can emerge in an end-to-end robotic system trained on appropriately diverse data.

    Source: §I · p. 3

    Despite broad generalization, π0.5 still makes mistakes, including with unfamiliar handles, partial observability, and easily distracted high-level subtask inference.

    Source: §VI · p. 11

Abstract

from arXiv · show

In order for robots to be useful, they must perform practically relevant tasks in the real world, outside of the lab. While vision-language-action (VLA) models have demonstrated impressive results for end-to-end robot control, it remains an open question how far such models can generalize in the wild. We describe $π_{0.5}$, a new model based on $π_{0}$ that uses co-training on heterogeneous tasks to enable broad generalization. $π_{0.5}$\ uses data from multiple robots, high-level semantic prediction, web data, and other sources to enable broadly generalizable real-world robotic manipulation. Our system uses a combination of co-training and hybrid multi-modal examples that combine image observations, language commands, object detections, semantic subtask prediction, and low-level actions. Our experiments show that this kind of knowledge transfer is essential for effective generalization, and we demonstrate for the first time that an end-to-end learning-enabled robotic system can perform long-horizon and dexterous manipulation skills, such as cleaning a kitchen or bedroom, in entirely new homes.

I. INTRODUCTION

The paper presents π0.5, a co-trained vision-language-action model that combines heterogeneous knowledge sources for broad robotic generalization. Its hierarchical control enables mobile manipulators to perform long-horizon household tasks in homes unseen during training.

I. INTRODUCTION

  • Real-world generalization requires more than scale: robots must transfer familiar skills, adapt them to new sequences, and use scene semantics to choose appropriate actions.The introduction gives picking up a knife or plate, opening the right drawer, and identifying a likely drying rack as examples.Source: §I · pp. 1–2
  • π0.5’s co-training framework uses VLA sequence modeling to combine direct robot experience with other embodiments, language, web-based perception tasks, and semantic commands.The paper presents the system and an empirical evaluation of its generalization and the relevance of different co-training ingredients.Source: §I · pp. 2–3
  • π0.5 controls mobile manipulators in unseen homes, performing 10–15-minute tasks such as cleaning kitchens or bedrooms, hanging towels, and making beds.The authors describe this as the first end-to-end learning-enabled robotic system to demonstrate long-horizon, dexterous manipulation in entirely new homes.Source: §I · pp. 2–3
  • 97.6% of π0.5’s first-phase training examples come from sources other than mobile manipulators performing household tasks, including other robots and web data.The training mixture also includes about 400 hours collected with mobile manipulators in real homes, lab tasks, semantic prediction, verbal instructions, and multimodal web examples.Source: §I · p. 2
  • The model is pretrained on heterogeneous tasks, then fine-tuned for mobile manipulation to predict a semantic subtask before each low-level action chunk.The two levels draw on different sources: other robots’ action data can support low-level inference, while semantic examples, web data, and supervisor instructions support high-level inference.Source: §I · pp. 2–3

II. RELATED WORK · III. PRELIMINARIES

Related work motivates broad-data co-training and high-level reasoning for generalist robot manipulation, while highlighting the difficulty of long-horizon open-world tasks. The preliminaries describe VLAs as imitation-learned token predictors, including flow-matching action generation.

II. RELATED WORK

  • Training on diverse scenes and tasks can broaden manipulation policies’ task coverage and generalization, but brute-force data collection is infeasible for complex tasks like kitchen cleanup.VLAs leverage semantic knowledge from pre-trained vision-language models and expressive action decoding, yet are typically evaluated in environments close to their training data.Source: §II · p. 3
  • π0.5 co-trains one VLA on robot data, semantic subtask predictions, and verbal instructions, extending beyond prior co-training with VLM data mixtures [23] [86].The paper attributes mobile robots’ complex, long-horizon behavior in new environments to this specific combination of supervision sources.Source: §II · p. 3
  • Unlike prior two-model approaches that separate semantic-step prediction from low-level execution [2] [71] [13] [24] [70] [72] [47], π0.5 uses the same model for both inference stages.It first infers a high-level semantic subtask, then predicts actions based on it; high-level inference runs less frequently than low-level action inference.Source: §II · p. 3
  • π0.5 training first combines diverse robotics, semantic-action, and web data, then specializes for mobile manipulation with flow-matching actions and human verbal instructions.At inference, it predicts a high-level subtask before the corresponding actions; robotic pre-training actions use the FAST tokenizer [64].Source: Figure 3 · p. 4
  • π0.5 performs long, multi-stage tasks in entirely new homes, drawing on target-platform experience and other data sources.Examples include putting dishes in a sink and picking clothing off a new bedroom floor; the additional sources include other robots, semantic subtask prediction, and web data.Source: §II · pp. 3–4

III. PRELIMINARIES

  • VLA imitation learning maximizes action or action-chunk likelihood given an observation and language instruction; observations include images and proprioceptive joint state.The observation may contain one or more images, while proprioception captures the robot’s joint positions.Source: §III · p. 4
  • VLAs encode inputs and outputs as discrete or continuous modality-specific tokens and use an autoregressive transformer initialized from a pre-trained vision-language model.Tokenization casts imitation learning as next-token prediction over observation, instruction, and action tokens.Source: §III · p. 4
  • For post-training, π0 represents continuous action chunks with flow matching, using an action expert to output the flow vector field from partially denoised actions.The action expert uses a distinct set of weights and can be smaller than the language-model backbone; prior work also uses diffusion [55] [84] [52] for action distributions.Source: §III · p. 4

IV. THE π0.5 MODEL AND TRAINING RECIPE · A. The π0.5 architecture

π0.5 starts from a web-trained vision-language model and uses two training stages to adapt it for diverse robotic tasks, then specialize it for mobile manipulation and efficient inference. Its multimodal model produces high-level text predictions and conditioned action chunks.

IV. THE π0.5 MODEL AND TRAINING RECIPE

  • Weights start from a standard web-trained vision-language model, followed by pre-training for diverse robotic tasks and post-training for mobile manipulation and efficient test-time inference.Pre-training represents all tasks, including robot actions, with discrete tokens; post-training adds an action expert for finer-grained actions and more compute-efficient real-time control.Source: §IV · pp. 4–5

A. The π0.5 architecture

  • During inference, π0.5 predicts a high-level subtask first, then uses that prediction to condition low-level action generation.Both stages are represented by the same model, and the action distribution depends on the predicted subtask rather than directly on the overall task prompt.Source: §IV · p. 5, A. The π0.5 architecture · p. 5

IV. THE π0.5 MODEL AND TRAINING RECIPE

  • Pre-training represents robot actions and other tasks with discrete tokens, while post-training adds an action expert for finer-grained action representation [64].The action expert also enables more compute-efficient inference for real-time control.Source: §IV · p. 5

A. The π0.5 architecture

  • The architecture represents both action-chunk distributions and tokenized text, using text for co-training tasks such as question answering and for high-level subtask predictions.Its outputs can also include answers to vision-language prompts in web data.Source: A. The π0.5 architecture · p. 5
  • Inputs comprise camera images, robot configuration, and an overall task prompt; outputs include a tokenized text prediction and a low-level action chunk.The configuration includes joint angles, gripper pose, torso lift pose, and base velocity.Source: A. The π0.5 architecture · p. 5
  • A transformer processes multimodal input tokens and produces multimodal outputs, with token types determining their encoders and, in some cases, separate expert weights.Image patches use a vision encoder, text tokens use an embedding matrix, and action tokens are linearly projected into the transformer embedding space.Source: A. The π0.5 architecture · p. 5
  • The output sequence separates text-token logits for sampling a predicted subtask from action tokens produced by a separate action expert and mapped to continuous actions.The first M outputs are text logits and the later H outputs are action tokens; not all outputs necessarily receive a loss, and proprioceptive state is discretized as text tokens.Source: A. The π0.5 architecture · p. 5

B. Combining discrete & continuous action representations

The model combines discrete action-token training with continuous flow matching to benefit from faster training and enable continuous action generation at inference. Its staged training preserves language-following ability while adding a non-autoregressive action expert for fast inference.

B. Combining discrete & continuous action representations

  • Discrete action tokens can accelerate VLA training, especially with efficient chunk compression such as FAST, but autoregressive decoding makes them less suited to real-time inference [64].The design goal is to train on discretized actions while retaining flow matching for continuous actions at inference.Source: B. Combining discrete & continuous… · p. 5
  • The model predicts actions both by autoregressively sampling FAST tokens and iteratively integrating a flow field, with attention preventing the two representations from attending to each other.The model minimizes a combined loss.Source: B. Combining discrete & continuous… · p. 5
  • With α = 0, pre-training maps actions to text tokens; post-training then adds action-expert weights to predict continuous actions non-autoregressively, yielding stable pre-training and strong language following.The action expert is smaller than the main model.Source: B. Combining discrete & continuous… · p. 5
  • At inference, the model autoregressively decodes text tokens, then uses 10 denoising steps conditioned on those tokens to produce action chunks.Source: B. Combining discrete & continuous… · p. 5

C. Pre-training

π0.5 pre-training uses heterogeneous robot and web data in autoregressive next-token prediction of text, object locations, and FAST-encoded action tokens.

C. Pre-training

  • Robot pre-training combines about 400 hours of mobile-manipulator household data from about 100 homes with multi-environment and cross-embodiment robot data.It also includes the open-source OXE dataset [15], an extension of the dataset used by π0 [8].Source: C. Pre-training · p. 6
  • For multi-subtask robot tasks, π0.5 predicts semantic subtasks and relevant bounding boxes before predicting actions conditioned on the subtask.The annotations cover MM, ME, and CE data, enabling the model to output subtasks as a high-level policy and execute their actions as a low-level policy.Source: C. Pre-training · p. 6
  • Web pre-training includes image captioning, question answering, and object-localization data, with indoor-scene and household-object boxes added for localization.Captioning sources include CapsFusion [87] and COCO [12]; question-answering sources include Cambrian-7M [77], PixMo [19], and VQAv2 [32].Source: C. Pre-training · pp. 6–7
  • Action targets are joint or end-effector poses; prompts specify the control mode, and per-dataset action dimensions are quantile-normalized to [−1, 1] and zero-padded to a fixed size.The fixed action dimensionality accommodates the largest action space among the datasets.Source: C. Pre-training · p. 7

D. Post-training … A. Can π0.5 generalize to real homes?

π0.5 is post-trained for mobile manipulation in homes and evaluated on unseen environments. In three real homes, it consistently succeeded on multistage cleaning tasks, with high-level inference autonomously determining steps from simple commands.

D. Post-training

  • Post-training specializes π0.5 for home mobile manipulation and adds a flow-matching action expert, jointly trained with next-token prediction for 80k steps.Training uses successful MM and ME episodes, web and multi-environment data, and language demonstrations that teach high-level subtask outputs.Source: D. Post-training · p. 7

E. Robot system details

  • Two mobile-manipulator platforms use four cameras, dual 6-DoF arms, a mobile base, and torso lift, yielding 18–19-dimensional state and action spaces.High-level inference uses all four cameras; low-level inference uses the wrist and forward cameras.Source: Figure 5 · p. 7
  • At 50 Hz with action chunking, π0.5 directly commands arm, gripper, torso, and base targets, tracked by simple PD controllers without trajectory planning or collision detection.Manipulation and navigation control are fully end-to-end.Source: E. Robot system details · p. 7

V. EXPERIMENTAL EVALUATION

  • All experiments use environments unseen in training; mock homes provide controlled quantitative comparisons, while three real homes provide the realistic final evaluation.The novel kitchens and bedrooms include unseen objects, backgrounds, and layouts.Source: §V · p. 7, Figure 6 · p. 8
  • The evaluation asks whether generalization scales with training-environment diversity, how co-training ingredients contribute, how π0.5 compares with π0, and how high-level inference compares with flat and oracle baselines.It also examines whether π0.5 generalizes to complex multistage tasks in entirely new homes.Source: §V · p. 7

A. Can π0.5 generalize to real homes?

  • In three unseen real homes, π0.5 consistently succeeded on bedroom and kitchen cleaning tasks, including 2–5-minute multistage trials autonomously decomposed from simple commands.The evaluation used both robot platforms; the reported tasks included putting items in a drawer, moving laundry to a basket, and putting dishes in a sink.Source: A. Can π0.5 generalize to real homes? · pp. 7–8, Figure 7 · p. 8

B. How does generalization scale with the number of scenes?

Generalization on multi-stage household tasks improves as the number of training locations increases, and the 104-location co-trained model reaches performance similar to a model trained on test homes. Language following and placement success also improve with more locations, including for unseen object categories.

B. How does generalization scale with the number of scenes?

  • Models were post-trained with mobile-manipulation data from 3, 12, 22, 53, 82, or 104 locations, using 40k steps and equal numbers of unique samples.They shared pre-training on robot-action prediction data without mobile-manipulation data, controlling for dataset size while varying the number of locations.Source: B. How does generalization scale with… · p. 8
  • Evaluation used unseen mock homes for four multi-stage chores and language-directed object pickup, testing new instances from seen categories and objects from unseen categories.The chores were putting dishes in the sink, packing items into a drawer, putting away laundry, and making a bed.Source: B. How does generalization scale with… · pp. 8–9
  • With 104 training locations, the full co-training model matched test-home-trained performance, while models omitting other co-training tasks performed significantly worse (Figure 8).The weaker baselines performed worse even when trained directly on test-home data, and the full recipe was especially important when test-home data was excluded.Source: B. How does generalization scale with… · p. 9, Figure 8 · p. 9
  • Language-following and success rates rose with more locations; seen categories improved faster than unseen categories, while new household items supported generalization to absent task categories.Language-following rate measures selection of the commanded object; success rate measures placing it in the correct drawer or sink.Source: B. How does generalization scale with… · p. 9, Figure 9 · p. 9

C. How important is each part of our co-training recipe? · D. How does π0.5 compare to other VLAs?

The results show that cross-embodiment data is important for π0.5’s performance, while web data particularly helps language following with out-of-distribution objects. π0.5 significantly outperforms π0 and π0-FAST+Flow in mock-home tests.

C. How important is each part of our co-training recipe?

  • Recipe ablations remove WD, multi-environment non-mobile data (ME), laboratory cross-embodiment data (CE), or both ME and CE; the full recipe also includes mobile-manipulator data (MM), high-level data (HL), and verbal-instruction post-training (VI).The ablations evaluate end-to-end task performance in mock homes and language following.Source: C. How important is each part of our… · p. 9
  • In mock-home tasks, excluding either ME or CE significantly degrades performance, and excluding both harms it even more.These results indicate benefits from cross-embodiment transfer across both environments (ME) and tasks (CE).Source: C. How important is each part of our… · p. 9, Figure 10 · p. 10
  • Removing WD does not significantly change mock-home performance, but it significantly worsens language following for out-of-distribution objects.The authors conjecture that web data’s broad knowledge of physical objects helps the model follow commands involving unseen object categories.Source: C. How important is each part of our… · p. 10, Figure 11 · p. 10
  • In language following, removing ME or CE significantly degrades performance on both in-distribution and out-of-distribution objects.The figure reports evaluation across models trained on different numbers of locations.Source: C. How important is each part of our… · p. 10, Figure 11 · p. 10

D. How does π0.5 compare to other VLAs?

  • With the same cross-embodiment training set and comparable training steps, π0.5 significantly outperforms π0 and π0-FAST+Flow, even when π0 trains for up to 300k steps.The comparison uses Figure 12; π0-FAST+Flow is an enhanced π0 baseline.Source: Figure 12 · p. 10, D. How does π0.5 compare to other VLAs? · p. 10
  • π0.5 adds high-level (HL) and web data (WD) and uses discrete-token pre-training followed by flow-matching post-training; π0 always uses an action expert.π0-FAST+Flow uses the hybrid recipe but only robot-action data, so it cannot perform high-level inference.Source: D. How does π0.5 compare to other VLAs? · p. 10
  • The authors report that FAST-token training is more compute-effective than pure diffusion training, consistent with Pertsch et al. [64].This conclusion follows from π0.5 outperforming π0 even when π0 is trained for up to 300k steps.Source: D. How does π0.5 compare to other VLAs? · p. 10

E. How important is high-level inference? · VI. DISCUSSION AND FUTURE WORK

The full π0.5 model performs best in high-level inference experiments, while implicit high-level inference also benefits from subtask-prediction training. The discussion describes π0.5’s generalization and outlines limitations and future directions.

E. How important is high-level inference?

  • The comparison uses π0.5 for both inference levels, policies without web or verbal-instruction data, implicit or absent high-level inference, GPT-4, and an expert-human oracle.The high-level policy predicts a subtask from a command to contextualize lower-level actions; the experiments also compare direct task-prompt input to low-level inference.Source: E. How important is high-level… · pp. 10–11
  • The full π0.5 model performs best, even outperforming the human-oracle policy; implicit high-level inference is second best, showing that subtask-prediction training provides substantial benefit.Explicitly inferring high-level subtasks adds benefit, but much of it is obtained by including subtask prediction data in training.Source: Figure 13 · p. 11, E. How important is high-level… · p. 11
  • Excluding high-level task data, verbal-instruction data, or web data significantly weakens performance; verbal instructions make up about 11% of high-level mobile-manipulation examples.The authors indicate that much of web data’s benefit may lie in improving the high-level policy.Source: E. How important is high-level… · pp. 10–11, Figure 13 · p. 11
  • Zero-shot GPT-4 as the high-level policy performs worst, indicating the importance of adapting vision-language models with robot data.GPT-4 was prompted with a task description and commonly used labels; the human policy served as an oracle comparison.Source: E. How important is high-level… · p. 11, Figure 13 · p. 11

VI. DISCUSSION AND FUTURE WORK

  • π0.5 builds on π0 through co-training across data sources, enabling mobile manipulators to perform multistage tasks in unseen homes with about 400 hours of mobile-manipulation data.Training also uses more data from other robots, web data, and high-level language-prediction data; demonstrated behaviors include cleaning, making beds, and hanging towels.Source: §VI · p. 11
  • π0.5 still struggles with unfamiliar or difficult-to-open fixtures, occlusion, and distracted subtask inference, while its prompt complexity is limited by training data.The authors propose better co-training, transfer, and larger datasets, as well as more intricate and diverse human or synthetic annotations.Source: §VI · p. 11
  • Future work could add richer context and memory for partial observability, explore broader data sources and ways for people to provide contextual knowledge, and build on π0.5 toward broader real-world generalization.The discussion gives navigating between rooms and remembering object locations as settings that may benefit from richer context and memory.Source: §VI · p. 11

APPENDIX · A. Contributions · B. Task evaluation rubric

The appendix lists contributors by work area and defines the evaluation protocol and scoring rubrics for kitchen and bedroom cleanup tasks.

A. Contributions

  • The contributions are organized by work area, including data collection, annotation, policy training and research, infrastructure, robot hardware, and writing and illustration.The appendix names contributors under each area.Source: A. Contributions · p. 17

B. Task evaluation rubric

  • Evaluations covered four training tasks in new scenes and configurations across 12 locations, using 10 trials per task unless stated otherwise and 40 evaluations per policy for standard evaluations.Policies were interleaved; cancelled episodes were removed, sample sizes were kept close, and plots report two-sided t-tests with variable trial counts.Source: B. Task evaluation rubric · p. 17
  • Dishes-in-sink scoring awards one point per item picked up and placed in the sink, for a maximum of 8 points.The task starts with four dishes near a sink, and the robot must place them all in the sink.Source: B. Task evaluation rubric · p. 17
  • Items-in-drawer scoring awards one point each for pickup, opening the drawer, placing the item inside, and closing it if the item is inside, for a maximum of 4 points.The task starts with an item on a countertop and requires placing it in a drawer beneath the counter.Source: B. Task evaluation rubric · p. 17
  • Under the evaluation protocol, laundry scoring gives up to 3 points for navigating to and picking up clothing, placing it in or on the basket, and getting it fully inside.The task begins with clothing on the ground and aims to place it in a laundry basket.Source: B. Task evaluation rubric · p. 17
  • Under the evaluation protocol, bed-making scoring gives up to 5 points for straightening the blanket and placing two pillows at the head, with extra points for neatness.The bed starts unmade; the rubric separately awards points for a neatly straightened blanket and neatly placed pillows.Source: B. Task evaluation rubric · p. 17

C. Language following experiment setup

The experiments test language following in unseen kitchen scenes and compare π0.5 with π0 and π0-FAST+Flow, finding advantages for π0.5.

C. Language following experiment setup

D. Per-task performance breakdown

Across four representative household tasks, the breakdowns show that diverse co-training and explicit high-level inference support performance, with task-specific differences in data and policy sensitivity.

a) Co-training recipe ablations

  • Cross-embodiment transfer and diverse co-training are critical, but data-source dependence varies across Items in Drawer, Dishes in Sink, Laundry Basket, and Make Bed.Items in Drawer degrades substantially without cross-embodiment or web data, with the largest drop when all are excluded; Dishes in Sink is relatively robust to web-data removal but degrades without cross-embodiment data. Laundry Basket and Make Bed also degrade without cross-embodiment data, but are less sensitive to other mixture changes.Source: D. Per-task performance breakdown · p. 18

b) High-level model analysis

  • Without high-level inference, Items in Drawer and Dishes in Sink decline substantially; π0.5 outperforms GPT-4 HL on both, while Items in Drawer also declines sharply without web data.The passages attribute the benefit to structured subtask prediction, long-horizon planning, in-domain fine-tuning, and semantic knowledge for less-seen objects.Source: D. Per-task performance breakdown · pp. 18–19
  • Across four representative tasks, explicit high-level inference improves performance, and the full π0.5 model achieves the best overall results.The tasks are Items in Drawer, Dishes in Sink, Laundry Basket, and Make Bed.Source: D. Per-task performance breakdown · p. 18, Figure 17 · p. 19
  • Laundry Basket and Dishes in Sink are less sensitive to high-level policy choice, as they involve shorter horizons or require less detailed semantic reasoning.Source: D. Per-task performance breakdown · p. 19

E. Model technical details

π0.5 builds on π0 with a PaliGemma VLM for visual-language understanding and a separate action expert for fast action generation [5]. Its technical design combines flow-matched action prediction with one-way information flow from the VLM to the action expert.

E. Model technical details

  • The model uses a PaliGemma VLM [5] for images, language, proprioceptive state, and autoregressively predicted tokenized actions [64], alongside a smaller transformer action expert.The action expert processes noisy action chunks over a horizon of 50 steps.Source: E. Model technical details · p. 19
  • The action expert uses flow matching, projecting timestep information separately and injecting it into each layer with adaptive RMSNorm before decoding outputs into a target vector field.A separate MLP projects the timestep, while a final linear projection decodes the action tokens.Source: E. Model technical details · p. 19
  • The two transformers retain π0 dimensions: the VLM has 2B parameters, and the action expert has 300M parameters.The VLM uses width 2048, depth 18, MLP dimension 16,384, and 18 attention heads; the action expert uses width 1024 and MLP dimension 4096.Source: E. Model technical details · p. 19
  • VLM and action-expert embeddings interact through self-attention, with information flowing only from the VLM prefix to the action expert and no VLM attention to action-expert embeddings.FAST action tokens attend to the prefix and previous action tokens; action-expert embeddings do not attend to FAST tokens, preventing information leakage between action representations.Source: E. Model technical details · p. 19, Figure 18 · p. 19
  • Timesteps are sampled from a low-timestep-emphasizing Beta distribution, excluding values above s; experiments use s = 0.999 to accommodate up to 1,000 integration steps.The distribution uses α = 1.5 and β = 1, and timesteps above s are unnecessary when the integration step δ > 1 − s.Source: E. Model technical details · p. 19
  • All input images receive random-crop, resize, rotation, and color-jitter augmentations.The listed operations include resizing and rotation from −5° to 5°.Source: E. Model technical details · p. 19

Related papers

Loading 2504.16054v1…