A Survey on Vision-Language-Action Models for Embodied AI

Yueen Ma; Zixing Song; Yuzheng Zhuang; Jianye Hao; Irwin King

Source-linked AI summary

A Survey on Vision-Language-Action Models for Embodied AI

Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, Irwin King

arXiv:2405.14093v8May 23, 2024cs.ROcs.CLcs.CV

TL;DR

VLAs are rapidly proliferating, but their components, control policies, planners, and supporting resources lack a comprehensive synthesis for embodied AI. This survey defines and taxonomizes VLAs, reviews their research lines and resources, and discusses challenges and future directions. It provides the first broad overview of LVLAs and generalized VLAs for embodied AI.

Problem
The rapid proliferation of VLAs creates a need for a comprehensive synthesis of models, resources, and challenges in embodied AI.
Method
The survey defines generalized VLAs and organizes research into components, low-level control policies, and high-level task planners while reviewing datasets, simulators, and benchmarks.
Results
The survey provides the first overview of LVLAs and generalized VLAs, comparing architectures, training strategies, modules, and resources across embodied AI.
Takeaways & Limitations
The taxonomy and resource summary capture the evolving VLA landscape and identify directions for future embodied AI research.
Takeaways & Limitations
Action representations involve trade-offs: SE(2) actions suit many tabletop tasks, whereas more complex tasks such as pouring may require additional degrees of freedom and SE(3) actions.

Abstract

from arXiv · show

Embodied AI is widely recognized as a cornerstone of artificial general intelligence (AGI) because it involves controlling embodied agents to perform tasks in the physical world. Building on the success of large language models (LLMs) and vision-language models (VLMs), a new category of multimodal models -- referred to as vision-language-action (VLA) models -- has emerged to address language-conditioned robotic tasks in embodied AI by leveraging their distinct ability to generate actions. The recent proliferation of VLAs necessitates a comprehensive survey to capture the rapidly evolving landscape. To this end, we present the first survey on VLAs for embodied AI. This work provides a detailed taxonomy of VLAs, organized into three major lines of research. The first line focuses on individual components of VLAs. The second line is dedicated to developing VLA-based control policies adept at predicting low-level actions. The third line comprises high-level task planners capable of decomposing long-horizon tasks into a sequence of subtasks, thereby guiding VLAs to follow more general user instructions. Furthermore, we provide an extensive summary of relevant resources, including datasets, simulators, and benchmarks. Finally, we discuss the challenges facing VLAs and outline promising future directions in embodied AI. A curated repository associated with this survey is available at: https://github.com/yueen-ma/Awesome-VLA.

I. INTRODUCTION … 1) Reinforcement Learning:

The survey defines VLAs as multimodal embodied-AI models that map vision and language to robot actions, and organizes their study around components, low-level control policies, and high-level task planners. It reviews the emerging field’s foundations, resources, challenges, and future directions while emphasizing its first comprehensive VLA-focused coverage.

I. INTRODUCTION: VLAs process vision, language, and action to understand instructions, perceive environments, and generate robot actions for embodied tasks.The term was recently coined by RT-2, and VLA systems can support complex tasks in both factory and household settings.
I. INTRODUCTION: VLA development addresses the limitations of conventional RL policies, which typically handle limited controlled tasks, by enabling more versatile language-conditioned multitask behavior.Language instructions provide a more intuitive user-robot interface but require learning broader skills across diverse environments.
III. VISION-LANGUAGE-ACTION MODELS: The survey organizes VLA research into component methods, low-level control policies, and high-level planners that decompose long-horizon tasks into subtasks.Hierarchical systems let high-capacity planners guide policies while low-level controllers emphasize speed and precision, as in hierarchical RL,,.
I. INTRODUCTION: The survey defines a VLA as any model that processes vision and language to produce robot actions, distinguishing original LLM- or large-VLM-based systems as Large VLAs.This generalized definition extends the original concept of adapting VLMs to robotic tasks.
A. Related Work: The survey is presented as the first comprehensive review focused specifically on emerging VLA models, complementing prior surveys of broader embodied-AI and robotics foundation-model topics.Earlier work covered foundation models, LLMs, vision-language-robotic models, or real-world applications rather than VLA models as its central focus.
B. Contributions: Its contributions include reviewing VLA components, architectures, objectives, and tasks; introducing a hierarchical taxonomy; summarizing datasets and benchmarks; and identifying challenges and future opportunities.The discussion includes data scarcity, inconsistency, safety, foundation models, and real-world deployment.
II. BACKGROUND: Embodied AI concerns agents that actively interact with physical environments, with robots as a prominent embodiment, and robot learning is commonly formulated through RL or imitation learning.Language-conditioned policies model actions from instructions, observed states, and prior actions, including under partial observability.
1) Reinforcement Learning:: RL provides a foundation for VLA research through pixel-based policy learning, trajectory modeling with Transformers, multimodal multitask systems, and emerging RL–LLM synergies.DT, TT, Gato, π∗0.6, SEED, Reflexion, and Eureka illustrate trajectory modeling, RL-based VLA learning, feedback alignment, linguistic RL, and LLM-designed rewards.

2) Pretrained Visual Representations: … 5) World Models:

The survey reviews visual, video, and dynamics representations that improve VLA perception and action prediction, then describes world models for imagining future states and enabling model-based control. It covers methods ranging from self-supervised visual pretraining and temporal modeling to latent dynamics and imagined-trajectory policies.

2) Pretrained Visual Representations:: Pretrained visual representations supply VLAs with object categories, positions, and affordances, motivating methods based on contrastive, masked reconstruction, language-conditioned, and self-distillation objectives.CLIP learns image-text alignment from 400 million WIT image-text pairs; R3M combines temporal contrastive learning with video-language alignment, while MVP, Voltron, and DINOv2 develop masked or self-supervised alternatives.
2) Pretrained Visual Representations:: Theia distills multiple vision foundation models into one model that fuses segmentation, depth, and semantic information while outperforming previous PVRs with less data and a smaller model.VC-1 also systematically explores ViT configurations and compares PVRs across manipulation and navigation datasets.
3) Video Representations:: Video representations extend framewise PVRs with temporal, multiview, 3D, and audio cues, including NeRF, 3D Gaussian Splatting, and audio for robot policies.F3RM and 3D-LLM exemplify extracting NeRF-based 3D information from videos, while 3D-GS, improves visual quality and rendering speed over NeRF.
4) Dynamics Learning:: Dynamics learning models forward transitions, inverse actions, or reordered state sequences to capture how actions alter environments.Forward dynamics predicts a subsequent state from an action, whereas inverse dynamics predicts the action needed to reach a known subsequent state; these approaches are compared in Table II.
4) Dynamics Learning:: Vi-PRoM combines contrastive video discrimination with temporal-dynamics recovery and pseudo-label image classification, improving behavior cloning and PPO over preceding pretraining methods.Its supervised temporal objective recovers shuffled video frames, while image classification uses pseudo labels.
4) Dynamics Learning:: SMART combines forward dynamics, inverse dynamics, and masked hindsight control to model short-term transitions and long-term temporal dependencies.MaskDP masks state and action tokens to learn both dynamics directions and can be applied zero-shot to downstream tasks; PACT autoregressively models state-action transitions for localization, mapping, and navigation.
5) World Models:: World models encode commonsense knowledge and predict future states for actions, enabling agents to search for action sequences in imagination before acting.They can also generate imagined visual experiences, whereas forward dynamics in many VLAs remains pretraining or an auxiliary action-decoder objective rather than a standalone module.
5) World Models:: Dreamer, DreamerV2, DreamerV3, and DayDreamer build latent dynamics models for imagined actor-critic learning, while IRIS trains policies on trajectories unrolled by a GPT-like Transformer world model.Dreamer uses representation, transition, and reward modules; IRIS uses a VQ-VAE vision encoder, and TWM also studies Transformer-based world models.

6) LLM-Induced World Models: … 1) Non-Transformer Control Policies:

The surveyed approaches extend VLAs with LLM- or visually induced world models, reasoning, policy steering, and increasingly structured low-level control policies. Together, these methods span symbolic and visual simulation, planning-time reasoning, test-time action selection, and non-Transformer architectures for language-conditioned robotics.

6) LLM-Induced World Models:: LLM-induced world models use commonsense knowledge for planning, including abstract graphs, PDDL simulators, and MCTS-based policies that improve task guidance or search efficiency.DECKARD alternates subgoal sampling and world-model updates for faster Minecraft crafting; LLM-DM generates PDDL models, while RAP and LLM-MCTS combine language-based world modeling with structured search.
7) Visual World Models:: Visual world models generate future images, videos, 3D scenes, or trajectories that align more closely with physical environments and can guide or train embodied agents.Genie provides an interactive generative environment, 3D-VLA generates goal states for robot control, and UniSim simulates outcomes for high- and low-level actions.
8) Reasoning:: CoT-based reasoning improves embodied planning by recovering missing actions, interleaving thoughts with execution, retrieving information to reduce hallucinations, and organizing plans as trees.ThinkBot, ReAct, RAT, and Tree-Planner apply these strategies to high-level task planning.
8) Reasoning:: Reasoning can also augment low-level policies: ECoT trains OpenVLA to reason about plans, subtasks, motions, and visual features before predicting actions, improving challenging-task generalization without additional robot data.CoT-VLA further introduces visual chain-of-thought reasoning for VLAs.
9) Policy Steering:: Policy steering improves VLA behavior at test time without retraining by re-ranking sampled actions with a learned value function or a VLM-based verifier.V-GPS uses value-based action re-ranking, whereas RoboMonkey selects among sampled actions using visual-language verification.
10) Strengths and Limitations:: Forward dynamics are generally harder than inverse dynamics because future-state prediction is more complex, while inverse dynamics can generate action labels from state-only robot videos.The difficulty of forward dynamics can yield larger performance improvements, as demonstrated in SMART.
10) Strengths and Limitations:: World models and reasoning remain functionally distinct: world models mainly support control policies by predicting immediate next states, whereas textual CoT methods mainly refine high-level task plans.Both paradigms can apply to either control or planning, but their current uses reflect these differing strengths.
B. Low-Level Control Policies: Before Transformers, early language-conditioned control policies varied in architecture, including semantic-spatial fusion in CLIPort, multimodal instruction conditioning in BC-Z, and free-form language with unstructured demonstrations in MCIL.

2) Transformer-Based Control Policies: … (a) FiLM

The survey organizes low-level VLA control policies around Transformer architectures, multimodal instructions, 3D visual representations, and diffusion-based action generation. Across these lines, models improve task specification, generalization, multimodal action modeling, and scalability through specialized representations, training objectives, and modular designs.

2) Transformer-Based Control Policies:: Transformer-based control policies have converged toward similar architectures, with representative systems enabling unified multitask training, adaptation from as few as 100 demonstrations, learning from failed trajectories, trajectory conditioning, and action chunking.Gato unifies inputs and outputs through tokenization, RoboCat rapidly adapts using as few as 100 demonstrations, Q-Transformer learns from successful and failed trajectories, and ACT predicts temporally ensembled action sequences.
3) Control Policies for Multimodal Instructions:: Multimodal prompts enable demonstrations, novel-object references, and pointing-based task specification beyond pure language, while VIMA evaluates generalization across placement, combinatorial, object, and task novelty.MOO extends RT-1 with OWL-ViT localization and additional objects and prompt images to support pointing and graphical-interface clicks.
4) Control Policies with 3D Vision:: 3D control policies use point clouds, voxels, continuous feature fields, or virtual-view rendering to exploit richer spatial information and guide manipulation from multiview observations.PerAct represents observations and actions with voxels, Act3D adapts feature resolution to the task, and RVT and RVT-2, re-render point-cloud scenes from virtual views.
5) Diffusion-Based Control Policies:: Diffusion-based policies formulate action generation with diffusion models, combining visual conditioning, receding-horizon control, and diffusion Transformers to model multimodal actions in high-dimensional spaces with stable training.Diffusion Policy provides the core formulation, while later approaches extend it to language-guided multitask learning, modular cross-robot transfer, and auxiliary-objective training.
(a) FiLM: SUDD composes an LLM with robot utilities to generate and filter language-guided data, then distills it into a language-conditioned visuo-linguo-motor diffusion policy.The framework uses primitives such as grasp samplers and motion planners before extending Diffusion Policy for multitask learning.
(a) FiLM: Octo uses a modular Transformer-based diffusion policy with interchangeable task, observation, and action components, achieving positive transfer and generalizability across robots and tasks through the OXE dataset.Its open-framework design flexibly integrates different encoders and action decoders with the Octo Transformer.
(a) FiLM: RDT-1B [113] addresses data scarcity with a physically interpretable unified action format, pretraining on heterogeneous multirobot datasets containing over 1M trajectories, scaling to 1.2B parameters, and enabling zero-shot generalization.The model is a DiT-based diffusion foundation model for bimanual manipulation.

6) Diffusion-Based Control Policies with 3D Vision: … 10) Strengths and Limitations:

The survey covers diffusion and geometric control policies, point-based actions, large VLAs, and their architectures, training objectives, strengths, and limitations. It emphasizes the tradeoffs between generalization, dexterity, instruction following, efficiency, and scalability.

6) Diffusion-Based Control Policies with 3D Vision:: Diffusion policies incorporate 3D point clouds, Act3D, mixture-of-experts architectures, DiT action diffusion, and rectified flow for robot control.DP3, 3D Diffuser Actor, and 3D-MoE represent these architectural directions.
7) Control Policies for Motion Planning:: Motion planning decomposes movement tasks into discrete waypoints under obstacle-avoidance and kinematic-limit constraints, while language-guided cost maps support natural-language correction and tracked visual stages enable few-shot visual imitation.
8) Control Policies with Point-Based Actions:: VLMs provide cost-effective point-based control by selecting visual keypoints, predicting 2D affordance points projected into 3D, or solving keypoint-based constraints.PIVOT, RoboPoint, and ReKep respectively select proposals, predict spatial affordances, and optimize numerical costs over 3D keypoints.
9) Large VLA:: Large VLAs extend RT-2 through co-fine-tuning, action hierarchies, larger datasets, efficient backbones, diffusion experts, and world-model integration.RT-X retrains RT-1 and RT-2 on OXE, while later systems explore quantized multimodal tokens and autoregressive modeling for actions, text, and images.
9) Large VLA:: Large VLA efficiency and capability advances include flow-matching action experts, linear-complexity Mamba reasoning, 3D position encoding, latent-action pretraining, and dual-rate humanoid control.π0, RoboMamba, SpatialVLA, LAPA, and GR00T N1 exemplify these directions.
10) Strengths and Limitations:: Most low-level policies predict end-effector poses, improving embodiment generalization but limiting dexterity by abstracting away fine-grained motion planning.Discrete, SE(2), SE(3), and point-based actions offer different tradeoffs: discrete actions can cause early grasping, SE(3) supports complex tasks, and points enable zero-shot VLM extraction.
10) Strengths and Limitations:: LVLAs improve instruction following but face training-cost and inference-speed concerns, motivating smaller diffusion systems and early-exit dynamic inference.TinyVLA targets speed and data efficiency, while DeeR-VLA partially activates the model during inference.
10) Strengths and Limitations:: Scaling laws in robotics show that model size, dataset size, and environmental and object diversity matter for robust in-the-wild VLA generalization.The survey identifies scaling laws as a guide for future VLA development.

IV. TASK PLANNERS … 2) End-to-End Task Planners with 3D Vision:

Task planners decompose complex instructions into subtask sequences for low-level policies, enabling VLAs to handle longer-horizon tasks. Monolithic and end-to-end planners extend this framework with embodied reasoning, replanning, and increasingly rich 3D inputs.

IV. TASK PLANNERS: Task planners decompose complex tasks into ordered subtasks that serve as instructions for low-level control policies.This subgoal-decomposition process is related to task and motion planning and embodied decision making.
IV. TASK PLANNERS: Equipped with task planners, VLAs can complete more complex, long-horizon tasks, ideally with optimized subtask scheduling.
A. Monolithic Task Planners: Monolithic task planners use a single LLM or multimodal LLM to generate plans through tailored frameworks or embodied-dataset finetuning.
1) End-to-End Task Planners:: End-to-end planners leverage Internet-scale knowledge in multimodal models to perform embodied task planning.
1) End-to-End Task Planners:: PaLM-E generates image- and language-conditioned text plans, maps them to executable instructions through SayCan, and replans as environments change.It combines ViT and PaLM, while also supporting standard and embodied VQA tasks.
1) End-to-End Task Planners:: EmbodiedGPT produces task-relevant instance-level features by combining vision-encoder embeddings with LLM-provided embodied planning information.These features inform the low-level policy about the immediate next action.
2) End-to-End Task Planners with 3D Vision:: 3D-vision task planners require architectural changes because most multimodal LLMs accept images, and they are consequently usually end-to-end models.
2) End-to-End Task Planners with 3D Vision:: LEO aligns 3D vision and language before 3D vision-language-action instruction tuning, performing across 3D QA, manipulation, navigation, and task planning.Related systems include 3D-LLM for 3D-assisted dialog and navigation, MultiPLY with audio, tactile, and thermal modalities, and ShapeLLM with the ReCon++ 3D encoder.

3) Grounded Task Planners: … C. Strengths and Limitations

The survey organizes grounded task planners into grounded, modular, language-based, and code-based approaches that connect high-level planning with executable low-level actions. It highlights their contrasting trade-offs in training cost, controllability, language alignment, and dependence on underlying model capabilities.

3) Grounded Task Planners:: Grounded task planning generates high-level actions while checking whether low-level control policies can execute them, as exemplified by SayCan.SayCan combines task-grounding from the LLM with world-grounding from the policy’s affordance value function to select skills executable in the current state.
3) Grounded Task Planners:: Translated ⟨LM⟩ iteratively generates language actions, maps them to admissible VirtualHome actions, and uses re-prompting to correct precondition errors.(SL)3 instead alternates segmentation, labeling, and parameter updates to discover reusable skills from sparse natural-language annotations.
B. Modular Task Planners: Modular task planners assemble off-the-shelf LLMs and VLMs, avoiding the expense of finetuning end-to-end models and following a tool-use architecture.This design improves deployability by reusing pretrained components rather than training a large embodied model from scratch.
1) Language-Based Task Planners:: Language-based planners exchange multimodal information through natural language, enabling feedback-driven closed-loop planning without additional LLM training, as in Inner monologue and ReAct.LID uses active data gathering and hindsight relabeling to improve combinatorial generalization.
1) Language-Based Task Planners:: Socratic models compose pretrained multimodal models through multimodal-informed prompting, converting non-language inputs into language descriptions for perception and planning.Their task plans can be expressed in natural language or pseudocode, unifying heterogeneous modalities in a language space without finetuning.
2) Code-Based Task Planners:: Code-based planners generate executable programs that invoke object detectors, VLMs, and control policies through APIs, with ProgPrompt incorporating environmental feedback through assertions.ChatGPT for robotics iteratively writes and executes API-based code using simulation and user feedback, placing the user in the control loop.
2) Code-Based Task Planners:: Code as policies (CaP), COME-robot, DEPS, and ConceptGraphs extend code-based planning with API-driven control, multimodal reasoning, replanning, goal selection, and semantic 3D scene graphs.These methods respectively support spatial-geometric reasoning and generalization, perception-free GPT-4V planning, self-explanation, achievable subgoal selection, and rich spatial-semantic representations for LLM planning.
C. Strengths and Limitations: Monolithic planners can achieve better specialized performance through embodied-data finetuning but incur substantial training costs, whereas modular planners are more readily deployable.Language-based planners integrate LLMs and VLMs naturally but need alignment with admissible low-level instructions; code-based planners offer debugging and controllability but require API wrapping and depend on programming ability.

V. DATASETS AND BENCHMARKS … VII. CONCLUSION

The survey reviews datasets, simulators, automated and human data collection, and specialized benchmarks while identifying persistent data, evaluation, safety, generalization, multimodality, planning, responsiveness, and societal challenges. It concludes by organizing VLA research around components, control policies, and task planners, alongside resources for training and evaluation.

A. Real-World Robot Datasets & Benchmarks: Real-world robot datasets remain scarce because collection is costly, robot configurations are inconsistent, 6D pose capture is difficult, and benchmarks often require human evaluation.These obstacles involve equipment, environment setup, expert demonstrations, sensory and control differences, gripper types, and setup reproduction.
B. Simulators, Simulated Robot Datasets & Benchmarks: Simulators scale data collection and enable reproducible, fair evaluation, but unrealistic rendering, physics errors, domain shifts, and difficult object modeling create a persistent sim-to-real gap.Real-to-sim methods can improve fidelity, recreate failures, and support digital twins.
C. Automated Dataset Collection: Automated collection methods use generative simulation, LLM orchestration, VLM-based instruction augmentation, or procedural scene generation to expand embodied datasets.RoboGen, AutoRT, DIAL, and RoboPoint exemplify these strategies.
D. Human Datasets: Human data offers plentiful, dexterous guidance but is difficult to transfer across robot embodiments because motion capture, viewpoint inconsistency, and filtering remain challenging.UMI addresses some transfer issues with hand-held grippers, while broader human-dataset comparisons are referenced in.
E. Task Planning Benchmarks: Task-planning benchmarks assess human-annotated planning, cost optimality, verification, replanning, and simulator-executed success, while EAI adds finer-grained metrics beyond success rates.EgoPlan-Bench, PlanBench,, and LoTa-Bench target complementary planning capabilities.
F. Embodied Question Answering (EQA) Benchmarks: Embodied question-answering benchmarks evaluate spatial reasoning, physics understanding, world knowledge, 3D perception, and multimodal or temporal reasoning through active exploration or egocentric video.Active exploration requires simulation, whereas EgoVQA, EgoTaskQA, EQA-MX, and OpenEQA broaden evaluation settings and abilities.
VI. CHALLENGES AND FUTURE DIRECTIONS: VLA progress is constrained by insufficiently comprehensive benchmarks, limited foundation-model generalization, multimodal alignment, hierarchical-planning failures, real-time speed-capacity tradeoffs, multiagent coordination, and ethical risks.Safety requires guardrails, risk assessment, human-robot protocols, RLHF, evaluation without execution, and interpretability; future systems also need broader applications, including specialized and healthcare robots requiring stronger privacy and safety measures.
VII. CONCLUSION: The survey organizes VLA research into key components, control policies, and task planners, while comparing architectures and training strategies and summarizing datasets, simulators, and benchmarks.It presents itself as the first survey covering LVLAs alongside generalized VLAs and aims to guide future embodied-AI research.

APPENDIX … • C-ViViT [408]

This appendix surveys foundational unimodal and multimodal components underlying vision-language-action models, covering vision, language, reinforcement learning, graphs, and vision-language architectures. It also includes supplementary related work omitted from the main text, including HLSM, MDETR, and ViT variants.

C. Background (Extended Version): Vision-Language Models: VLMs combine vision and language through BERT-based single- or multi-stream Transformers, contrastive pretraining, and large multimodal models, and can also serve as high-level task planners.The survey compiles representative VLM specifications in Table VIII and reviews early self-supervised, contrastive, and LMM developments.
B. Background (Extended Version): Unimodal Models: VLAs commonly reuse unimodal vision, language, and reinforcement-learning models, whose development has progressed toward Transformer-based architectures and increasingly larger models.These components process multimodal inputs, interpret instructions, generate responses, and support policy optimization; Figures 7 and 8 summarize their timelines and scaling trends.
1) Computer Vision:: Robotic vision focuses on classification, detection, and segmentation, progressing from CNN backbones such as ResNet to Transformer models including ViT, DETR, and SAM.Beyond RGB, visual inputs include depth maps, point clouds, and volumetric data such as voxels or octrees.
2) Natural Language Processing:: NLP evolved from word, syntax, and semantic processing through RNNs such as LSTM and GRU toward Transformer-based models including BERT, GPT, and T5.Scaling produced LLMs such as ChatGPT, GPT-4, PaLM, and LLaMA, while instruction tuning, prompting, and DPO enabled efficient preference- and instruction-following.
3) Reinforcement Learning:: RL provides the policy-learning foundation for VLAs, encompassing value-based methods, policy search, imitation learning, hierarchical RL, and robotic systems such as QT-Opt and Dreamer.Representative methods address overestimation, sparse rewards, instability, convergence, sample complexity, and long-horizon control through techniques including CQL, PPO, GAIL, and hierarchical managers and workers.
4) Graph:: Graph models process structured data across scenes, language, and 3D objects, progressing from spectral and spatial GNNs to graph Transformers and E(n)- or SE(3)-equivariant networks.Scene graphs represent visual object relationships, while dependency, constituency, AMR, sentence, document, and knowledge graphs encode linguistic structure and support downstream tasks.
1) Self-Supervised Pretraining:: Self-supervised VLMs align modalities using single- or multi-stream Transformers and objectives such as masked modeling, matching, prefix language modeling, and modality-specific experts.ViLBERT pioneered multi-stream modeling, while VL-BERT, UNITER, ViLT, SimVLM, VLMo, and BEiT-3 represent subsequent designs.
2) Contrastive Pretraining:: Contrastive VLMs improve transferable multimodal representations through large-scale image-text learning, while LMMs connect frozen language models to vision encoders using modules such as gated cross-attention, Q-Former, or linear projections.Representative systems include CLIP, ALIGN, ALBEF, Flamingo [6], BLIP-2, LLaVA, and MiniGPT-4.
D. Supplementary Related Work: The appendix notes that supplementary related works are listed because page limitations prevented citing all mentioned models in the main text.
• MDETR [402]: The supplementary list includes the HLSM object detector.
• MDETR [402]: The supplementary list also includes ViT, ViT-B, and ViT-L,.

• OSRT [409] … • JARVIS-VLA [477]

The survey’s additional VLA-related work spans component pretraining, world models, imitation learning, subsequent surveys, and newer developments across robot embodiments and environments. It also supplements the main text with a search-based collection of approximately 400 VLA-related papers.

• OSRT [409]; • PaLI-X [365]; • DD-PPO [432]; • DDIM [437]: Additional VLA-related work includes language, memory, vision-language, and diffusion-model components, with DDIM [437] also involving multicontext imitation learning, maximum likelihood estimation, model predictive control, and conditional variational autoencoders.
E. Additional VLA-Related Work:: The survey explicitly includes additional VLA-related work that could not fit in the main text because of page limits.
1) Components of VLA: Pretraining:: Pretraining advances include temporal representation learning, frozen-encoder feature fusion, self-supervised representations, audio dynamics, and tactile policies for dexterous manipulation.VIP models long-range temporal relationships and local smoothness but received conflicting results in later evaluations; SpawnNet uses adapter-based two-stream fusion without training pretrained vision encoders, while Holo-Dex, Dobb·E, AuRL, and T-Dex extend representation learning across modalities.
2) Components of VLA: World Models:: World-model research transfers knowledge from large-scale human videos to robot control through visual affordance maps and supports long-horizon decision-making by independently rolling out noncontrollable transitions.SWIM grounds actions in visual affordance maps, while Iso-Dream decouples controllable and noncontrollable dynamics and rolls out the noncontrollable branch independently of actions.
3) Components of VLA: Imitation Learning:: Imitation-learning variants address noisy, multimodal demonstrations through discretized and corrected actions, goal conditioning, and vector quantization for long-range behavior.BeT combines k-means action discretization with continuous correction, C-BeT adds a target frame or demonstration, and VQ-BeT replaces k-means with vector quantization.
4) Subsequent Surveys on VLA:: The survey notes that embodied AI has generated several subsequent VLA surveys since its initial release,,,,,,,,,,.
5) Latest Developments of VLA:: Recent developments advance robot-video relabeling, dexterous action tokenization, in-the-wild generalization, hierarchical navigation, and work involving humanoid, quadruped, and dexterous-hand embodiments.LLaRA relabels existing robot demonstration videos with rich descriptive language; FAST prioritizes dexterous skills; π0.5 co-trains with diverse data sources. Mobility VLA integrates VLM reasoning with topological-graph-based navigation.
• JARVIS-VLA [477]: Its supplementary search combined automated scripting and manual searching for papers published from January 2020 through December 2025, yielding approximately 400 VLA-related papers after relevance filtering.The authors acknowledge possible automated errors and invite corrections.

• CNRS: CNRS IRL 2010 CROSSING … • NAVER: NAVER AI Lab

This block defines abbreviations for universities, laboratories, companies, and research organizations represented across five affiliation groups. It includes institutions spanning China, Europe, North America, Japan, Korea, and Singapore.

• CNRS: CNRS IRL 2010 CROSSING: The CNRS: CNRS IRL 2010 CROSSING group includes CSU, CUHK, CUHK(SZ), CWRU, Caltech, Cambridge, and Google.It also lists China Mobile, China Telecom, CityU, Cognitive AI, Columbia, DSO, DUT, Didi Chuxing, Drexel, ECNU, EIT, EKUT, ENS, EdUHK, Fudan, GDUT, Gachon Univ., Georgia Tech, HBUT, HIT, HIT(SZ), HKU, HKUST, HKUST(GZ), HUST, Hanyang Univ., Harvard, Horizon, Huawei, HuggingFace, Hyundai, IC, IEIT, IIT, Infinigence, and JHU.
Affiliation glossary: Across these groups, the glossary combines academic institutions, corporate research organizations, laboratories, and technology companies as affiliation abbreviations.Examples include Google, MIT, Meta, Microsoft, Huawei, Hyundai, LiAuto, and Infinigence alongside universities and national laboratories.
• KAIST: KAIST AI: The KAIST: KAIST AI group lists KIT, Kobe Univ., Korea Univ., LZU, Lehigh Univ., LiAuto, MBZUAI, and MEGVII.These abbreviations expand to institutions and companies including Karlsruhe Institute of Technology, Kobe University, Korea University, Lanzhou University, Lehigh University, LiAuto Inc., Mohamed bin Zayed University of Artificial Intelligence, and MEGVII Technology.
• MIPT: IAI MIPT: The MIPT: IAI MIPT group identifies MIT as the Massachusetts Institute of Technology.MIT is the highlighted affiliation listed for this group.
• MMLab: MMLab @ CUHK, MMLab @ HKU: The MMLab: MMLab @ CUHK, MMLab @ HKU group includes Macalester, Meta, Microsoft, Mila, Monash Univ., and Moxin.The entries cover Macalester College, Meta AI and related groups, Microsoft Research units, Mila, Monash University, and Moxin (Huzhou) Technology Co., Ltd.
• NAVER: NAVER AI Lab: The NAVER: NAVER AI Lab group lists NEU, NJU, NJUPT, NPU, NTU, and NUS.These abbreviations refer to Northeastern University, Nanjing University, Nanjing University of Posts and Telecommunications, Northwestern Polytechnical University, Nanyang Technological University, and National University of Singapore.

• NVIDIA: NVIDIA Research

This section defines abbreviations for research institutions, companies, and organizations associated with the survey. It also identifies 3D Diffuser Actor as a VLA appearing in Figure 9.

Abbreviations: The list also includes universities, laboratories, companies, and robotics organizations from multiple regions.Examples include Northwestern University, Shanghai AI Laboratory, Toyota Research Institute, and Xiaomi Robotics.
Abbreviations: The abbreviation list covers institutions and organizations including Stanford, Tsinghua, NVIDIA, and Physical Intelligence.Stanford and Tsinghua are listed as universities, while NVIDIA and Physical Intelligence are listed as research or industry organizations.
Figure 9: Figure 9 includes 3D Diffuser Actor among the listed VLAs.The passage explicitly introduces this entry as part of the VLAs appearing in Figure 9.

• ACG [483] … • ET-VLA [556]

This section lists representative works associated with ConRFT [516], CoT-VLA [79], and DLR [534]. The supplied passages identify Control Your Robot, Counterfactual VLA, and Don’t Blind Your VLA respectively.

• ConRFT [516]: ConRFT [516] is associated with Control Your Robot.
• CoT-VLA [79]: CoT-VLA [79] is associated with Counterfactual VLA.
• DLR [534]: DLR [534] is associated with Don’t Blind Your VLA.

• ExpReS-VLA [559] … • RPD [698]

This block lists works associated with four subsections: HULC++ [85], INSIGHT [600], MemER [623], and RoboOS-NeXT [692]. The listed works are ImaginationPolicy, Interactive Language, Mind to Hand, and Robotic Assistant, respectively.

• HULC++ [85]: HULC++ [85] is associated with ImaginationPolicy.
The cited material names two works, each with its original bibliography marker: ImaginationPolicy and Interactive Language, without stating any relationship between either work and a subsection.
• INSIGHT [600]: INSIGHT [600] is associated with Interactive Language.
• MemER [623]: MemER [623] is associated with Mind to Hand.
The cited material includes entries titled Mind to Hand and Robotic Assistant.
• RoboOS-NeXT [692]: RoboOS-NeXT [692] is associated with Robotic Assistant.

• RS-CL [699] … 6) Beyond VLA:

The survey identifies research directions beyond standard VLA architectures, including world-model integration, expanded perception modalities, and video-action pretraining for efficient low-level control.

• TA-VLA [736]: TA-VLA [736] is listed alongside Transporter Networks as a related research entry.
• UP-VLA [750]: UP-VLA [750] is listed alongside USIM & U0 as a related research entry.
6) Beyond VLA:: Recent research expands beyond standard VLA architectures in several key directions.
6) Beyond VLA:: VLA-plus-world-model architectures unify VLAs with world models, including WorldVLA, UniVLA, NORA-1.5, RynnVLA-002, and Motus.
6) Beyond VLA:: PLA and MLA incorporate perception modalities beyond vision, including Perception-Language-Action and Multisensory Language–Action.
6) Beyond VLA:: Video-Action Models jointly capture semantics and dynamics during video pretraining, enabling more efficient post-training for low-level robot control; mimic-video (VAM) exemplifies this direction.