PaperlayerView exact revision on arXiv

Source-linked AI summary

Latent Action Pretraining from Videos

Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, Minjoon Seo

arXiv:2410.11758v2cs.ROcs.CLcs.CVcs.LG

TL;DR

LAPA addresses the limited scalability of VLA pretraining that depends on ground-truth robot actions by learning discrete latent actions from unlabeled videos. It predicts those latent actions during VLM pretraining, maps them to robot actions with small-scale labeled finetuning, and achieves stronger reported performance than video-based baselines and current action-labeled VLAs.

  • Problem

    Existing VLA pretraining depends on ground-truth robot action labels, limiting the use and scale of internet video data.

  • Method

    LAPA learns discrete latent actions between video frames, pretrains a VLM to predict them, and finetunes on labeled robot trajectories to map them to robot actions.

  • Results

    LAPA significantly outperforms video-pretraining baselines and surpasses current action-labeled VLAs on reported simulation and real-world manipulation evaluations.

  • Takeaways & Limitations

    LAPA can transfer from actionless robot and human manipulation videos, including pretraining entirely on human videos with an embodiment gap.

  • Takeaways & Limitations

    LAPA underperforms action pretraining on fine-grained motion generation such as grasping, and it faces real-time inference latency.

Abstract

from arXiv · show

We introduce Latent Action Pretraining for general Action models (LAPA), an unsupervised method for pretraining Vision-Language-Action (VLA) models without ground-truth robot action labels. Existing Vision-Language-Action models require action labels typically collected by human teleoperators during pretraining, which significantly limits possible data sources and scale. In this work, we propose a method to learn from internet-scale videos that do not have robot action labels. We first train an action quantization model leveraging VQ-VAE-based objective to learn discrete latent actions between image frames, then pretrain a latent VLA model to predict these latent actions from observations and task descriptions, and finally finetune the VLA on small-scale robot manipulation data to map from latent to robot actions. Experimental results demonstrate that our method significantly outperforms existing techniques that train robot manipulation policies from large-scale videos. Furthermore, it outperforms the state-of-the-art VLA model trained with robotic action labels on real-world manipulation tasks that require language conditioning, generalization to unseen objects, and semantic generalization to unseen instructions. Training only on human manipulation videos also shows positive transfer, opening up the potential for leveraging web-scale data for robotics foundation model.

1 INTRODUCTION

LAPA addresses the difficulty of scaling VLA pretraining from teleoperated robot data by learning from unlabeled human and robot videos. It combines unsupervised latent-action learning, latent-action prediction, and small-scale action finetuning, achieving strong simulation and real-world results.

  • Approach: The method quantizes frame-to-frame behavior into latent actions, pretrains a VLM to predict them, and finetunes on labeled robot trajectories.The final finetuning stage maps latent actions to executable robot actions.
  • Results: LAPA significantly outperforms video-pretrained baselines and transfers across environments, embodiments, and human manipulation videos.The method also outperforms a VLA trained on action-labeled data in real-world manipulation tasks.
  • Motivation: LAPA learns robotic skills from web-scale video without ground-truth action labels.The approach targets human motion videos and other raw video sources that lack robot actions.
  • Results: +6.22% over the state-of-the-art action-labeled VLA, with over 30x greater pretraining efficiency.The comparison is reported for simulation and real-world robot tasks.
  • Additional capability: The latent action model can also serve as a policy and its decoder as a world model for closed-loop neural rollouts.Future frames are predicted from observations and latent actions without physical execution.

2 RELATED WORK

Prior VLA and video-learning methods improve robotic policies but generally depend on labeled actions or use videos only for representation and trajectory modeling. LAPA instead uses latent actions to train a monolithic VLA from actionless real-world human motion.

  • Vision-Language-Action Models: Monolithic VLAs extend vision-language models with robotic action data, but remain dependent on labeled actions.This dependence limits scalability relative to raw internet video.
  • LAPA's distinction: LAPA requires labeled robot actions only during finetuning, reducing reliance on human-teleoperated data.Its pretraining stages operate on videos without ground-truth robot actions.
  • Training Robot Policies From Videos: Video-based robot-learning methods commonly pretrain visual representations, generate future trajectories, or extract interaction features from human videos.These approaches address video learning without necessarily producing the latent action representation used by LAPA.
  • Latent Actions: Earlier latent-action work uses ground-truth actions or targets video-game policies, whereas LAPA learns latent actions from real-world human motion.LAPA applies the representation to actionless robotic video and monolithic VLA training.

3 LAPA: LATENT ACTION PRETRAINING FOR GENERAL ACTION MODELS

LAPA sequentially learns discrete latent actions, trains a VLM to predict them from observations and instructions, and maps them to robot actions through labeled finetuning. The design supports actionless video pretraining while retaining executable robot control.

  • 3.1 LATENT ACTION QUANTIZATION: Latent Action Quantization encodes a current frame and future frame into a discrete latent action that reconstructs the future frame.The encoder processes x_t and x_t+H, while the decoder reconstructs x_t+H from x_t and z_t.
  • 3.1 LATENT ACTION QUANTIZATION: The VQ-VAE objective converts continuous representations into codebook tokens that are easy for VLMs to predict.Each latent action is represented as a sequence from the codebook vocabulary.
  • 3.2 LATENT PRETRAINING: The quantization encoder functions as an inverse dynamics model, while its decoder supports neural closed-loop rollouts.Together, the policy and world-model components produce and evaluate latent-action trajectories.
  • 3.2 LATENT PRETRAINING: Latent pretraining uses the quantization encoder to label video transitions and trains a VLM to predict those latent actions from instructions and current images.A separate latent-action head predicts the codebook vocabulary, without ground-truth robot actions.
  • 3.3 ACTION FINETUNING: Action finetuning maps latent actions to discretized delta end-effector actions using a small set of labeled robot trajectories.Latent actions are not directly executable because they are not actual robot action coordinates.
  • 3.3 ACTION FINETUNING: The implementation excludes previous frames from quantization inputs because of computational constraints.Prepending past observations is left for future work.

4 EXPERIMENTS

LAPA is evaluated across simulation and real-world manipulation settings, including cross-task, cross-environment, cross-embodiment, and human-video pretraining scenarios. Across these evaluations, latent-action pretraining generally improves transfer and real-world performance while requiring substantially less pretraining compute than OPENVLA.

  • Evaluation setup: LAPA is evaluated on 9 task categories, 2 simulation environments, and 3 real-world robotic tasks using multiple pretraining and fine-tuning setups.The real-world experiments use Bridgev2, Open-X, and Something-Something v2 pretraining data with three multi-instruction tabletop tasks.
  • In-Domain Performance: With 181k pretraining trajectories and only 1k action-labeled fine-tuning trajectories, LAPA largely outperforms SCRATCH and narrows the gap with ACTIONVLA in-domain.LAPA also surpasses UNIPI and VPT, while the comparison highlights the effectiveness of latent actions.
  • Cross-Task Performance: Latent pretraining significantly benefits both the separate task and the other four task categories after fine-tuning on 7k trajectories, improving seen and unseen cross-task performance.VPT can surpass ACTIONVLA in the unseen setting, likely because it uses more labeled data for pseudo-label generation.
  • Cross-Environment Performance: LAPA significantly outperforms SCRATCH when pretrained on 440k real-world trajectories and fine-tuned on 1k simulation trajectories, demonstrating positive cross-environment transfer.UNIPI and VPT underperform LAPA in this setting, and VPT shows only minor positive transfer.
  • Real-World Results: LAPA outperforms action-labeled-pretraining VLAs on average Bridgev2 real-world performance and significantly outperforms OPENVLA on 2 out of 3 tasks with Open-X pretraining.The authors attribute this advantage to avoiding overfitting to the WidowX action space and to a shared latent action space across embodiments.
  • Learning from Human Manipulation Videos: Human-video pretraining transfers positively despite the human-to-robot embodiment gap, with LAPA outperforming baselines in simulation and outperforming OPENVLA (Bridge) on average in real-world experiments.The human-video model shows especially good generalization to unseen objects, while LAPA pretraining is approximately 30–40 times more efficient than OPENVLA pretraining.

5 ABLATION AND ANALYSIS

LAPA benefits from scaling model size, data size, and latent action representation, while the best latent action-space configuration depends on dataset complexity. Latent actions correspond to semantic movements and can form a shared representation across embodiments and datasets, supporting coarse-grained rollout planning.

  • Scaling model, data, and latent action size: Scaling model parameters, data size, and latent action representation improves LAPA’s downstream average success rate on SIMPLER tasks.The ablation varies model parameters, Bridgev2 data ratio, latent action sequence length, and vocabulary size.
  • Scaling model, data, and latent action size: The optimal latent action-space scale depends on the action complexity of the pretraining dataset.For Language Table, increasing vocabulary size is more effective than increasing sequence length, whereas the main experiments retain generation space 84 except for Language Table.
  • Latent action analysis: Each quantized latent action corresponds to a distinct semantic movement reconstructed from the current image observation.Examples include [1,1,3,2] mapping to moving down and left, and [3,2,0,1] mapping to moving slightly upward.
  • Latent action analysis: Latent actions can represent camera viewpoint changes in human videos and remain similar across differing embodiments and environments.The shared representation is presented as facilitating positive transfer across diverse datasets.
  • Closed-loop rollout: A pretrained LAPA model can generate closed-loop rollout images from language-conditioned latent actions without action finetuning.The rollout uses the instruction “take the broccoli out of the pot” and qualitatively examines coarse-grained planning.

6 LIMITATIONS AND CONCLUSION

LAPA is a scalable method for building VLAs without ground-truth action labels, demonstrated across simulation and real-world experiments and applicable to human manipulation videos. The authors identify limitations in fine-grained motion generation, inference latency, and scope beyond manipulation videos.

  • Conclusion: LAPA builds VLAs without ground-truth action labels and improves downstream transfer across simulation and real-world benchmarks.The method also surpasses models trained on 970K action-labeled trajectories and applies to human manipulation videos despite the embodiment gap.
  • Limitations: LAPA underperforms action pretraining on fine-grained motion generation such as grasping, faces real-time inference latency, and remains unexplored beyond manipulation videos.The authors suggest increasing the latent action generation space and using a hierarchical architecture, while leaving other video domains for future work.
  • Latent Action Quantization: Latent action quantization uses an encoder-decoder architecture to infer discrete actions between frames and reconstruct future frames.The encoder processes current and future frames, while the decoder reconstructs the future frame from the current frame and latent action.
  • Latent Action Quantization: The quantization model uses cross attention, stop-gradient on p1, and an L2 reconstruction objective to avoid representation collapse and train decoding.The decoder contains a spatial transformer, while the encoder uses spatial and causal transformers.
  • Latent Action Quantization: After training, the encoder serves as an inverse dynamics model and the decoder as a world model for latent-action pretraining and closed-loop rollouts.The encoder supplies latent action labels for frames, while the decoder generates neural-based rollouts.

B DETAILS ON EXPERIMENTAL SETUP

The experiments evaluate LAPA in Language Table, SIMPLER, and real-world tabletop manipulation settings, using randomized object configurations and tasks requiring language-conditioned object selection. Setup details include video and robot datasets, partial-success metrics, and implementation choices for baseline methods.

  • Experimental Setups: Figure 9 covers 440k real-world and 181k simulated Language Table trajectories, four SIMPLER tasks, and three real-world tabletop tasks.The figure summarizes the experimental setups used across the benchmark environments.
  • Language Table: Language Table models generate language directions before actions, and evaluation uses 50 randomized rollouts for each subtask category.Examples include directions such as “move up.”
  • SIMPLER: SIMPLER provides no fine-tuning data, so the underlying VLM is trained on Bridgev2 before zero-shot rollout on four tasks.The evaluation uses held-out trajectories with different object orientations and positions; 25 successful trajectories per task are selected for fine-tuning.
  • Real-world Tabletop Manipulation: Real-world tabletop experiments collect 150 teleoperated trajectories per task, with three objects requiring multi-instruction following to identify the target object.Evaluation tests unseen combinations of seen objects, totally unseen objects, and semantic instruction generalization.
  • Evaluation: Evaluation matches image resolution and initial object positions across models and uses partial-success criteria for fine-grained scoring.The setup controls image and scene variability to support fair comparison.
  • Implementation: UNIPI uses a diffusion model trainable on four A100 GPUs, batch size 128, and periodic replanning after every two executed actions.VPT uses a ResNet18 followed by an MLP for its inverse dynamics model.

D EXPERIMENTAL RESULT ANALYSIS

Downstream performance depends on how well pretraining data covers the evaluated skills, while LAPA shows a favorable real-world comparison against OpenVLA. The analysis links task-specific trajectory availability to observed gains and failures.

  • Pretraining Task Distribution: Table 4 counts trajectories matching Knocking, Covering, and Pick & Place across Bridgev2, Open-X, and Sthv2 pretraining datasets.The counts are determined through lexical matching.
  • Knocking: Almost no knocking trajectories in Bridgev2 may explain its worse performance than Sthv2 despite Sthv2’s larger embodiment gap.The comparison indicates that task coverage can matter alongside embodiment differences.
  • Covering: Bridgev2’s relatively small number of covering trajectories coincides with occasional underperformance against LAPA trained on Sthv2.The passage reports the same direction of association for the covering task.
  • Pick & Place: Sthv2’s relatively small number of pick-and-place trajectories may explain why its LAPA model significantly underperforms models trained on Bridgev2 or Open-X.The authors expect broader skill coverage to support more robust generalist policies and call for deeper analysis of task distribution.
  • LAPA versus OpenVLA: 65.4% of non-tied trials favor LAPA over OpenVLA, while including ties gives LAPA 31.5%, OpenVLA 16.7%, and ties 51.9%.The evaluations alternate models with identical target and distractor initial locations.

E DETAILED LATENT ACTION ANALYSIS

Latent actions form meaningful representations of motion: they cluster in continuous action space, correspond to semantic movement directions, and capture both hand and camera movements in human videos.

  • Language Table Latent Actions: With vocabulary size 8 and sequence length 1, latent actions correspond to semantic directions such as moving left, right, forward, back, or not moving.The Language Table analysis assigns each latent action to a distinct movement pattern.
  • Language Table Latent Actions: Latent actions are well-clustered in the actual 2D action space and strongly related to continuous actions.This supports their interpretation as meaningful action representations.
  • Human Manipulation Videos: In Something-Something V2, latent actions capture both hand movements and camera movements caused by changing egocentric viewpoints.The model therefore represents motion beyond the actor’s hand in human manipulation videos.

F ADDITIONAL ABLATION RESULTS

The ablations examine how latent-action temporal context, vocabulary, sequence length, and fine-tuning data scale affect performance across SIMPLER and Language Table. Larger fine-tuning datasets and richer latent-action representations generally improve results.

  • Quantization window: Latent-action quantization uses video-dependent temporal windows, with the next frame set 0.6 seconds ahead for robot videos and 2.4 seconds ahead for human videos.The longer human-video interval reflects qualitatively less dynamic motion in many human-video frames.
  • Fine-tuning data scale: LAPA consistently outperforms SCRATCH on SIMPLER even with small fine-tuning datasets, while absolute performance increases with more data.The comparison uses Bridge data and SIMPLER evaluation.
  • Human-video data scale: Scaling Something-Something V2 human videos improves SIMPLER performance, including final success across all subtasks.The comparison is between training on 10% and the full human-video dataset.
  • Latent-action representation: Increasing latent-action sequence length and vocabulary size improves Language Table performance.The vocabulary-size effect is larger than the sequence-length effect in absolute performance for this visually simple environment.

G DETAILED EXPERIMENTAL RESULTS

The detailed appendix reports Language Table evaluation procedures and qualitative failure cases. Results are organized across seen and unseen settings, while UNIPI performs well on short-horizon tasks but fails when longer-horizon planning is required.

  • Evaluation protocol: Language Table appendix tables report success rates from 50 evaluation rollouts per task category and 250 total rollouts per model in each table.Best and second-best comparison results are bolded and underlined, respectively.
  • UNIPI qualitative results: UNIPI’s diffusion model generates correct plans for simple, short-horizon tasks but produces incorrect plans for instructions requiring longer-horizon planning.The qualitative example contrasts a successful separation task with a failed placement instruction.
  • Language Table results: The Language Table results are reported for in-domain, cross-task, and cross-environment settings, each with seen and unseen evaluations.The appendix includes tables covering these six evaluation categories.

G.2 SIMPLER

SIMPLER evaluates manipulation policies across Bridgev2 and human-video pretraining, with detailed task, subtask, rollout, and qualitative analyses. LAPA outperforms action-label-free baselines and approaches action-supervised performance as action-space complexity increases.

  • Evaluation setup: SIMPLER evaluates four main tasks with grasping and moving subtasks, using models pretrained on Bridgev2 and fine-tuned on 100 SIMPLER videos.The tasks are stacking a green block, placing a carrot, placing a spoon, and placing an eggplant.
  • Bridgev2 pretraining: LAPA outperforms UNIPI and VPT on SIMPLER after Bridgev2 pretraining without ground-truth action labels.It closes the performance gap with ACTIONVLA, which uses all 60K action-labeled Bridgev2 trajectories.
  • Bridgev2 pretraining: OPENVLA scores 36.4 on SIMPLER after fine-tuning, similar to training from scratch.The passage attributes this result to OPENVLA’s known lack of robustness to real-to-sim transfer for SIMPLER.
  • Human-video pretraining: Human-video pretraining compares UNIPI, VPT, and LAPA on SIMPLER without ground-truth action labels; ACTIONVLA is excluded because it cannot be trained in that setting.The human-video source is Something-Something V2.
  • Real-world comparison: Figure 11 compares OpenVLA and LAPA by pairwise win rate across 54 real-world evaluation rollouts, both excluding and including ties.The comparison is presented in two panels with different tie handling.
  • Latent-action analysis: Latent actions are well clustered in Language Table’s 2D ground-truth action space and can correspond to semantic movements such as directional pushing.In human videos, latent actions can also encode semantic camera movements.
  • Additional ablations: Figure 15 varies the latent-action quantization window and fine-tuning data scale, reporting average SIMPLER success rate.Figure 16 separately varies latent-action vocabulary and sequence length for Language Table.
  • Qualitative planning cases: UNIPI succeeds on a short-horizon instruction to separate objects but fails on a longer-horizon instruction to move one object toward another.These examples illustrate the qualitative planning boundary reported for UNIPI.
Loading 2410.11758v2…