PaperlayerView exact revision on arXiv

Source-linked AI summary

LaLa-World: Scaling Latent World Models with Natural Language Actions

Junha Song, Geonmo Gu, Jeongeun Park, Heejeong Nam, Lucas Maes, Byeongho Heo, Dongyoon Han, Sangdoo Yun

arXiv:2610.27851v2cs.CVcs.LG

TL;DR

Latent world models have largely relied on domain-specific numerical actions, while image and video data often lack those annotations, leaving joint learning across these sources underexplored. LaLa-World uses language to describe both numerical actions and visual changes and builds its predictor on a language model. It achieves higher average MPC planning performance than separately trained LWM baselines and reports competitive downstream policy performance with faster inference than Cosmos Policy.

  • Problem

    Latent world models commonly train on domain-specific numerical actions, while broader image and video data often lack numerical action annotations, leaving joint learning across these sources underexplored.

    Source: §1 · p. 3

  • Method

    LaLa-World describes numerical actions and visual changes in language and uses a language-model predictor to predict future latent states.

    Source: §1 · p. 3

  • Results

    LaLa-World achieves higher average MPC planning performance than separately trained LWM baselines and, against a matched RAEv2 baseline, outperforms in planning with 6× less training cost and 13.9× faster inference.

    Source: §1 · p. 4

  • Takeaways & Limitations

    Fine-tuning pretrained LaLa-World into LaLa-Policy yields competitive performance with Cosmos Policy on LIBERO and RoboCasa, with up to 5.5× faster inference per trajectory.

    Source: §1 · p. 4

    The paper identifies real-robot policy evaluation, larger training data and models, and longer prediction horizons as directions for future work.

    Source: §6 · p. 11

Abstract

from arXiv · show

This paper shows that JEPA-style latent world models (LWMs) can learn across diverse domains and data sources by adopting the language-conditioned interface of video generation models. We establish this approach as an effective and efficient basis for planning with imagination and downstream policy fine-tuning. To this end, we introduce LaLa-World, a Language-conditioned Latent World model that learns from simulation and robot trajectories together with broader image and video data. Two designs make this work. First, we describe both numerical actions and visual changes in language, enabling joint training of a single future predictor across heterogeneous datasets. Second, we build the predictor on a language model, allowing it to interpret language conditions while predicting future latent states. The resulting predictor achieves higher average MPC planning performance than existing LWMs trained separately for each domain or task. Adding image and video data further improves average planning performance and robustness to action phrasing. In a matched comparison with an RAEv2-based video generation model baseline, LaLa-World reaches comparable planning performance with $6\times$ less training cost and $13.9\times$ faster planning. Beyond planning, the pretrained LaLa-World serves as an initialization for LaLa-Policy, which jointly predicts actions and future visual representations. LaLa-Policy performs competitively with Cosmos Policy on LIBERO and RoboCasa, with up to $5.5\times$ faster inference.

1 Introduction

LaLa-World addresses the difficulty of training latent world models across domains with different numerical action spaces and visual data lacking action annotations. It uses language to condition one predictor on both control actions and visual changes, achieving strong planning results and supporting downstream policy fine-tuning.

1 Introduction

  • Numerical actions vary in dimensionality and physical meaning across domains, while image and video data often lack numerical action annotations, complicating joint training.The paper focuses on JEPA-style latent world models that predict changes in visual representations without pixel-level reconstruction.Source: §1 · p. 3
  • Language provides a shared interface for numerical control actions and visual changes, letting one future predictor train across heterogeneous contexts and actions.The predictor is built on a language model that interprets conditions while predicting future latent states.Source: §1 · p. 3, Figure 3 · p. 5
  • LaLa-World’s jointly trained predictor achieves higher average MPC planning performance than LWMs trained separately by domain, and outperforms a matched RAEv2 baseline with 6× less training cost and 13.9× faster inference.The matched comparison uses the same language conditions, training data, and visual encoder.Source: §1 · p. 4
  • LaLa-Policy fine-tuned from pretrained LaLa-World performs competitively with Cosmos Policy on LIBERO and RoboCasa, with up to 5.5× faster inference per trajectory.LaLa-Policy jointly predicts actions and future visual representations.Source: §1 · p. 4
  • The paper’s contributions include joint training across control and visual-transition data, cross-domain MPC planning, matched generative-model comparisons, and policy fine-tuning.The shared interface enables numerical actions and visual changes to be described in language.Source: §1 · pp. 3–4

2 Background and Motivation

Latent world models predict future visual representations under numerical action conditions, making repeated imagined rollouts useful for planning. Joint learning from heterogeneous control data and broader visual data remains underexplored, while language-conditioned generative world models offer a flexible interface and pretrained weights used for policy learning.

2 Background and Motivation

  • LWMs encode observations into latent representations and predict a future latent from a context window and a k-dimensional numerical action condition.Existing action-conditioned predictors are commonly trained on simulation or robotics trajectories and evaluated through domain-specific planning.Source: §2 · p. 5
  • Language-conditioned video generation models learn visual changes across domains and data sources, and pretrained weights have been adapted for robot action prediction.Cosmos Policy fine-tunes Cosmos-Predict2 into a robot policy.Source: §2 · p. 5
  • Joint learning by LWMs from heterogeneous control datasets and broader visual data, and their use in downstream policy learning, remain underexplored.The paper explores retaining LWM efficiency while adopting the flexible language interface of generative world models.Source: §2 · p. 5

3 LaLa-World

LaLa-World replaces numerical action conditions with language and pairs language-conditioned transition data with a predictor that understands those conditions. The design pools heterogeneous control and visual-transition data while predicting future latent representations.

3 LaLa-World

  • LaLa-World replaces the numerical action condition with language and requires both language-described transitions and a predictor that understands them.The section presents data construction and predictor architecture as the two components of the model.Source: §3 · p. 5

3.1 Language-Conditioned Transition Data

  • Fixed templates convert numerical actions from seven simulation and robotics datasets into language while retaining their direction and magnitude, allowing control data to be pooled.The datasets’ actions differ in physical meaning and dimensionality.Source: §3.1 · pp. 5–6
  • Captioned video and image-editing transitions use descriptions of visual change as language conditions, without requiring numerical action annotations.The added sources include OpenVid, OmniEdit, and ShareGPT, broadening the language and visual changes seen during training.Source: §3.1 · p. 6

3.2 Model Architecture and Objective

  • LaLa-World uses a frozen DINOv2-S/14 or DINOv3-L/16 observation encoder and excludes proprioceptive state as input and prediction target.The paper states that domain-specific proprioception limits applicability to a domain-agnostic world model.Source: §3.2 · p. 6
  • A pretrained language model serves as the predictor, reading language conditions and predicting future latent representations in the same network.Visual latents are mapped into the model’s embedding space, and training minimizes squared error between predicted and target future representations.Source: §3.2 · p. 6

4 MPC Planning

LaLa-World uses MPC to plan with imagined future states, jointly evaluating diverse control domains and visual data. It performs strongly across domains and, in matched comparisons, outperforms RAEv2 with lower training and planning costs.

4 MPC Planning

  • MPC samples candidate action sequences, scores autoregressive latent rollouts by distance to the encoded goal, and executes an initial segment of the selected sequence.The evaluation uses seven control datasets and assesses zero-shot transfer on RoboCasa, which is excluded from training.Source: §4 · p. 7

4.1 Results and Analysis

  • LaLa-World achieves the highest average score in both benchmark groups, including 72.1% average success across five simulation tasks and zero-shot RoboCasa success of 40.6% on Reach and 46.9% on Place.Its average simulation success exceeds JEPA-WMs by 7.5 percentage points.Source: §4.1 · p. 8
  • Adding image and video transitions raises average score from 59.3 to 61.6 and improves robustness to paraphrased actions.With control trajectories alone, paraphrasing lowers average score by 6.8 points.Source: §4.1 · p. 8, Figure 4 · p. 8
  • Qualitative language actions can guide planning in Wall and RoboCasa even when exact action magnitudes are unspecified.Numerical action magnitudes are converted to three qualitative levels: slightly, no modifier, or far.Source: §4.1 · p. 8

4.2 Latent vs. Generative World Models

  • The matched comparison uses the same vision encoder, language conditions, and training data for LaLa-World and RAEv2.The full-depth predictors each contain approximately 0.8B parameters.Source: §4.2 · p. 8
  • 59.4% planning performance exceeds RAEv2’s best 48.0%, and full-depth LaLa-World plans 13.9× faster than five-step RAEv2.LaLa-World reaches five-step RAEv2’s 12-hour performance in two hours; Figure 5 compares performance over training time and planning time across predictor depths.Source: Figure 5 · p. 9, §4.2 · p. 9

5 LaLa-Policy: From World Model To Action Predictor

LaLa-World is fine-tuned into LaLa-Policy to jointly predict action chunks and future latent representations. On LIBERO and RoboCasa, it performs competitively with Cosmos Policy while offering faster inference.

5 LaLa-Policy: From World Model To Action Predictor

  • LaLa-Policy jointly predicts an action chunk and future latent representation from an observation and task instruction.It adds learned action queries to the pretrained predictor; an action generator uses the resulting features to generate actions through iterative denoising.Source: §5 · p. 10
  • The policy is fine-tuned on LIBERO and RoboCasa demonstrations, with the observation encoder frozen and the action generator trained from scratch.For RoboCasa, training uses either 50 human demonstrations or 300 MimicGen demonstrations per task.Source: §5 · p. 10
  • Up to 5.5× faster average inference time per trajectory accompanies competitive performance with Cosmos Policy on LIBERO and RoboCasa.The comparison is reported in Table 4.Source: §5 · p. 10, Table 4 · p. 10
  • Initializing the predictor from LaLa-World outperforms random and Qwen3.5 initialization on both LIBERO and RoboCasa.The finding supports world-model pretraining as an initialization for policy fine-tuning.Source: §5 · p. 11
  • During execution, predicted future representations capture robot motion and object-position changes during manipulation, although decoded images may be visually inaccurate.LaLa-Policy is not trained for pixel reconstruction.Source: §5 · p. 11

6 Conclusion

The conclusion presents LaLa-World as a shared-language latent world model for heterogeneous control and visual data, and reports benefits in planning and policy learning. The related work situates it among latent world models, video generators, planning methods, action predictors, and visual representation pretraining.

6 Conclusion

  • LaLa-World outperforms existing LWMs in average MPC planning, and under matched settings surpasses a generative baseline in planning performance and efficiency.The conclusion also reports that LaLa-Policy outperforms evaluated VLA baselines and is substantially faster than Cosmos Policy; future work includes scaling data and model size, longer prediction horizons, and real-robot policy evaluation.Source: §6 · p. 11

A Related Work

  • The comparison frames LaLa-World as using language-conditioned control and visual transitions, while LaLa-Policy predicts actions with future latents.The figure contrasts these designs with representative world models and policy models.Source: A Related Work · p. 17

A.1 Latent World Models

  • Prior latent world models commonly train action-conditioned predictors within one domain; joint training on control trajectories and broader image and video transitions remains underexplored.LaLa-World uses language to unify actions across control datasets and co-trains with diverse visual transitions.Source: §A.1 · p. 17

A.2 Video Generation Models

  • Video generation models can learn from web-scale video and serve as interactive simulators, but repeated synthesis of visual detail makes them costly for MPC planning.Prior faster-planning comparisons differ in action interface, encoder, predictor size, and training data; the paper compares against RAEv2 under matched settings.Source: §A.2 · p. 18

A.3 Planning and Policy Learning with World Models

  • The paper follows reward-free, goal-conditioned MPC planning, where test-time sampling-based optimization seeks a goal image using learned dynamics.The related work distinguishes this protocol from methods that train actor-critic policies on imagined rollouts or use reward and value prediction.Source: §A.3 · p. 18

A.4 Action Prediction Models

  • LaLa-Policy follows joint action-and-future prediction approaches but predicts future representations rather than reconstructing visual details.The related work contrasts this with VLA models that predict actions alone and Cosmos Policy, which predicts actions and future states.Source: §A.4 · p. 18

A.5 Visual Representation Pretraining

  • The paper follows the V-JEPA 2 evaluation protocol: freeze a pretrained encoder, train an action-conditioned predictor on robot trajectories, and plan toward image goals.The cited protocol is also used by DINO-WM and JEPA-WMs.Source: §A.5 · p. 18

B Future Directions

The paper identifies six directions for extending LaLa-World and LaLa-Policy, including scaling data and model capacity, adapting visual representations, and broadening prediction and evaluation. Several boundaries remain open, from longer prediction horizons to real-robot testing.

B Future Directions

  • Training data are smaller than those of video-generation models such as Wan, and it remains unknown whether image/video gains continue beyond the 0.8B predictor.The authors propose scaling dataset size and predictor capacity separately and together, then measuring planning and downstream policy learning.Source: B Future Directions · p. 19
  • Freezing the vision encoder prevents representation collapse but also prevents its representations from adapting to future prediction.The authors identify joint encoder-predictor training without collapse as a future direction.Source: B Future Directions · p. 19
  • LaLa-World is supervised only on the next latent state, leaving autoregressive generation of longer sequences an open question.The proposed extension uses autoregressive rollouts with losses over multiple future states, at additional training cost.Source: B Future Directions · p. 19
  • MPC evaluation uses planning horizons of at most six steps, so performance at longer horizons remains to be assessed.Longer-horizon evaluation would test anticipation of distant consequences, including in real-world tasks requiring such foresight.Source: B Future Directions · p. 19
  • LaLa-Policy’s benefit is established in LIBERO and RoboCasa simulation; evaluation on real robots remains a next step.Source: B Future Directions · p. 19
  • LaLa-Policy predicts actions competitively without value supervision, while extending it to best-of-N planning is left for future work.Cosmos Policy learns from policy rollouts to predict future states and values for candidate action chunks.Source: B Future Directions · p. 19

C.1 ImageNet Training and Evaluation of LaLa-World

Across the two subsections, LaLa-World retrieves language-specified ImageNet transformations and supports LLM planning through language actions and evaluation costs. The reported planning results use far fewer imagined rollouts than CEM, while ImageNet retrieval is strongest with language-action conditioning.

C.1 ImageNet Training and Evaluation of LaLa-World

  • ImageNet training pairs source images with transformed target views and language descriptions of crops, shifts, rotations, or flips, training the predictor to match the encoded target.The predictor is trained jointly on these pairs and other transition data.Source: §C.1 · p. 19
  • 0.856 MRR with language actions, versus 0.024 using the source latent and 0.026 with empty-language prediction.Table A reports mean reciprocal rank over 256 transformed views per source image; its published IWM results use different training and transformation settings.Source: §C.1 · p. 20

C.2 Communicable World Modeling

  • In communicable world modeling, an LLM proposes language action sequences, receives LaLa-World’s goal-distance costs, revises proposals, and executes the lowest-cost sequence.The LLM receives costs but neither imagined states nor privileged simulator states.Source: §C.2 · pp. 20–21
  • 8–96 rollouts per trajectory versus 4,500–54,000 for CEM; Claude Opus 5 reaches 100.0% success on Wall with 90 rollouts and 53.1% on RoboCasa-R with 80.Source: §C.2 · p. 21

C.3 LaLa-World Architecture Ablations

These ablations test whether predictors follow the meaning of language conditions rather than visual regularities or numerical patterns. They examine how training data, language backbones, and predictor architecture affect that reliance.

C.3 LaLa-World Architecture Ablations

  • Language sensitivity compares held-out prediction accuracy with the score from retaining the current state; exact predictions score 1, retaining the current state scores 0, and worse-than-current predictions score negatively.The evaluation keeps the target fixed while changing language conditions, including qualitative descriptions and swapped direction or magnitude words.Source: §C.3 · pp. 21–22
  • Qualitative descriptions preserve direction but coarsen magnitude, so some accuracy loss is expected; reversed direction words can make the score negative.Swapping magnitude words preserves direction and is expected to cause a smaller drop.Source: §C.3 · p. 22
  • Adding image and video transitions to control data narrows the accuracy gap for qualitative magnitudes and makes the direction-swap score more negative.The results suggest broader transition data help the predictor interpret qualitative magnitudes and respond to direction words.Source: §C.3 · p. 22
  • Qwen3.5 most closely follows the expected language-sensitivity pattern, while Gemma shows a similar pattern with lower scores under original actions.Nemotron's validation loss is close to Qwen3.5's, yet its score barely changes when direction words are reversed.Source: §C.3 · p. 22
  • Low prediction loss alone does not establish that a predictor follows action semantics.Pretrained Qwen3.5 reaches lower loss earlier than randomly initialized Qwen3.5, although their final losses are similar.Source: §C.3 · p. 22
  • 3D input positional embeddings with learned queries yield the highest language reliance among the tested predictor designs.The results support the default predictor combining Qwen3.5 with 3D input positional embeddings and learned queries.Source: §C.3 · pp. 22–23

C.4 Predictor Initialization and Joint Training

This ablation compares jointly trained predictors against separate domain-specific predictors and tests pretrained versus random initialization. Joint training with pretrained initialization has the highest reported average score.

C.4 Predictor Initialization and Joint Training

  • Single-domain predictors are pretrained and trained separately on each domain's control transitions, with the DROID-trained predictor used for zero-shot RoboCasa evaluation.The jointly trained predictors instead use the full mixture of control trajectories and image and video transitions.Source: §C.4 · p. 23
  • 60.2 average score with pretrained joint training compares with 54.5 for single-domain training.The comparison uses the same predictor architecture and planning settings; joint training uses control trajectories plus image and video transitions.Source: §C.4 · p. 23
  • 55.6 is the average score for the jointly trained predictor with random initialization, below the 60.2 score with pretrained initialization.The two initializations reach similar final validation losses, suggesting prediction loss alone does not fully capture planning performance.Source: §C.4 · p. 23

C.5 LaLa-World Vision Encoder Ablations

The vision-encoder ablation compares frozen encoder configurations, including one predictor using both DINO representations. Encoder preferences vary by benchmark, and the combined configuration has the highest reported average for LaLa-World.

C.5 LaLa-World Vision Encoder Ablations

  • Each encoder configuration has a separate predictor trained on all training data; the combined DINOv2-S/14 and DINOv3-L/16 configuration predicts both representations as future targets.The comparison reports ten benchmarks and their average score.Source: §C.5 · p. 23
  • 63.0 average score is achieved by one predictor using both DINOv2-S/14 and DINOv3-L/16 representations, though improvements are not uniform across benchmarks.The encoder results suggest the two representations provide complementary information for planning.Source: §C.5 · p. 24
  • LaLa-World performs better with DINOv2-S/14 on Push-T and Granular, and with DINOv3-L/16 on PointMaze and both MetaWorld tasks.The preferred DINO encoder varies across benchmarks for both LaLa-World and JEPA-WMs.Source: §C.5 · p. 24

C.6 Additional Results on Planning with Qualitative Language Actions

The section tests planning with qualitative language actions that omit exact numerical magnitudes. LaLa-World succeeds on several domains, although success rates can decrease when magnitudes are coarsened.

C.6 Additional Results on Planning with Qualitative Language Actions

  • 83.3% success on Wall, 70.8% on MetaWorld-R, and 60.4% on MetaWorld-RW are reported for qualitative language actions.These actions replace numerical magnitudes with slightly, no modifier, or far.Source: §C.6 · pp. 24–25
  • Success rates decrease on some tasks, as expected when distinct numerical magnitudes are mapped to only three qualitative levels.The qualitative descriptions retain direction and relative magnitude information without exact numerical action values.Source: §C.6 · pp. 24–25
  • Only the language conditions supplied to the predictor change; CEM samples numerical actions and the simulator executes the selected actions in their original numerical form.Source: §C.6 · p. 25

C.7 Additional Results on Action Phrasing Robustness

Additional results test robustness to action phrasing and compare latent with generative world models. The appendix also details planning procedures, action conversion, data, and LaLa-Policy fine-tuning.

C.7 Additional Results on Action Phrasing Robustness

  • Paraphrasing lowers scores in all five domains for control-only training, reducing the average from 59.3 to 52.5; with image and video transitions, the average remains more robust to paraphrasing.Model weights and planning settings are held fixed during evaluation; the authors suggest image and video transitions reduce dependence on action-template wording.Source: §C.7 · p. 25, §C.9.1 · p. 26

C.8 Additional Results on Latent vs. Generative World Models

  • LaLa-World scores 59.4 on average, versus 40.4 and 48.0 for one- and five-step RAEv2, outperforming both on six of seven benchmarks and matching five-step RAEv2 on MetaWorld-R.The comparison uses full-depth predictors and follows the matched setup in Section 4.2.Source: §C.8 · p. 25, §C.9.1 · p. 26

C.9.2 Planning with Qualitative Language Actions

  • Qualitative-action examples show MPC rollouts across Wall, PointMaze, Push-T, MetaWorld, and RoboCasa, with action descriptions shown alongside simulator observations.The RoboCasa Reach example conditions zero-shot MPC on descriptions of gripper translation, rotation, and opening or closing.Source: §C.9.2 · p. 26

C.10 Qualitative Policy Results

  • LaLa-Policy rollout visualizations pair simulator observations with decoded future predictions across LIBERO suites and RoboCasa kitchen tasks.The examples cover LIBERO Spatial, Object, Goal, and Long, plus RoboCasa coffee, door and drawer, object-transfer, and stove-control tasks.Source: §C.10 · p. 26

D.4.2 Action Aggregation and Execution

  • For action aggregation, component sums across five substeps are verbalized and optimized, then divided equally across substeps for execution, discarding within-step variation.The authors note that despite this information loss, LaLa-World achieves accurate future prediction and high planning success rates.Source: §D.4.2 · p. 29, §D.4.1 · p. 29

D.4.3 Qualitative Language Action Conversion

  • Qualitative action descriptions preserve direction while mapping magnitudes into three equal-width bins; MPC evaluates descriptions but the simulator executes the original numerical actions.The bin labels are slightly, no modifier, and far; the predictor and other planning settings remain unchanged.Source: §D.4.3 · pp. 29–30

D.7 Policy Model Training Data

  • The policy datasets include 40 LIBERO tasks with 1,700 successful training episodes and RoboCasa tasks under approximately 50- and 300-demonstration settings.LIBERO future-prediction training uses all 2,000 replays, including failures; the approximately 50-demonstration RoboCasa setting includes 1,199 successful training episodes.Source: §D.7 · p. 31

D.8 LaLa-Policy Fine-tuning Details

  • LaLa-Policy fine-tuning initializes the predictor from LaLa-World, jointly trains it with new policy modules, and trains the action generator from scratch.Training combines a flow-matching velocity loss on policy examples with mean squared error on future representations for both example types.Source: §D.8 · p. 32

D.9 Policy Model Evaluation Details

This section details LaLa-Policy evaluation and MPC examples across LIBERO and RoboCasa tasks. It covers simulator interaction, language-described gripper actions, and demonstrations pairing observations with decoded future predictions across camera views.

Evaluation protocol

  • LaLa-Policy is evaluated by direct interaction with LIBERO and RoboCasa, following the Cosmos Policy procedure and conditioning on task instructions, proprioception, and camera observations.LIBERO supplies external and wrist views; RoboCasa supplies left, right, and wrist views.Source: §D.9 · p. 32
  • After 10 no-op steps, the policy predicts action chunks; all 16 LIBERO actions and the first 16 of each 32-action RoboCasa chunk are executed before querying again.Predicted future representations are not used for action selection during evaluation.Source: §D.9 · p. 32
  • Evaluation covers 40 LIBERO tasks and 24 RoboCasa tasks, with 50 episodes per task and success rates averaged across LIBERO suites and RoboCasa tasks.RoboCasa evaluation uses five test scenes and held-out object instances from split B; episode step budgets vary by suite or task.Source: §D.9 · p. 32

Numerical language actions

  • Figures illustrate numerical language-action sequences with decoded future predictions on Wall, PointMaze, Push-T, MetaWorld, DROID, and RoboCasa Reach and Place.RoboCasa is excluded from world-model training.Source: §D.9 · pp. 34–36

Qualitative language actions

  • Qualitative-action MPC examples show the agent passing through an opening on Wall and the ball moving through PointMaze, while Push-T examples depict simulator states under pusher directions and relative displacements.The captions describe actions with directions and relative movement magnitudes on Wall, and qualitative force descriptions in PointMaze.Source: §D.9 · p. 37, Figure V · p. 37

Qualitative Language Actions

  • LaLa-World plans MetaWorld reaching trajectories from qualitative descriptions of gripper motion and grip adjustment, without exact numerical magnitudes.RoboCasa Reach and Place examples show zero-shot MPC conditioned on qualitative gripper translation, rotation, and opening or closing.Source: §D.9 · p. 38, Figure X · p. 38

LaLa-Policy Demonstrations

  • LaLa-Policy demonstrations cover LIBERO spatial, object, goal, and long-horizon tasks, with observed states paired against decoded future predictions.Examples include placing bowls and food items, opening a drawer, and transferring multiple objects.Source: §D.9 · pp. 39–42
  • RoboCasa demonstrations show LaLa-Policy completing button-pressing, serving, door-opening, drawer-opening, object-transfer, and stove-control tasks with decoded future predictions.The examples compare predicted and observed states across external and wrist camera views.Source: §D.9 · pp. 43, 44, 45, 46, 47, 49

Related papers

Loading 2610.27851v2…