PaperlayerView exact revision on arXiv

Source-linked AI summary

Pure Vision Language Action (VLA) Models: A Comprehensive Survey

Dapeng Zhang, Jing Sun, Chenghui Hu, Xiaoyan Wu, Zhenlong Yuan, Rui Zhou, Fei Shen, Qingguo Zhou

arXiv:2509.19012v3cs.ROcs.AI

TL;DR

VLA research lacks a focused, consensual account of pure VLA methods and their supporting resources, while existing reviews emphasize foundational models or broad robotics coverage. This survey systematically classifies VLA approaches, analyzes their applications and infrastructure, and synthesizes over three hundred studies. It concludes that progress depends on addressing data, evaluation, embodiment, controllability, trustworthiness, and safety constraints.

  • Problem

    Existing reviews provide limited coverage of pure VLA methods, while VLA systems still face a gap between semantic understanding and reliable physical execution.

  • Method

    The survey develops a taxonomy of VLA methods, reviews applications and supporting datasets, benchmarks, and simulators, and analyzes current challenges and directions.

  • Results

    The survey synthesizes over 300 articles and organizes VLA innovations across autoregressive, diffusion, reinforcement-based, hybrid, and efficiency-oriented approaches.

  • Takeaways & Limitations

    VLA progress requires coordinated advances in methods, data resources, standardized evaluation, and practical deployment considerations.

  • Takeaways & Limitations

    Current VLA evaluation remains concentrated in laboratory or structured simulated tasks, limiting assessment of open-world generalization and real-world applicability.

Abstract

from arXiv · show

The emergence of Vision Language Action (VLA) models marks a paradigm shift from traditional policy-based control to generalized robotics, reframing Vision Language Models (VLMs) from passive sequence generators into active agents for manipulation and decision-making in complex, dynamic environments. This survey delves into advanced VLA methods, aiming to provide a clear taxonomy and a systematic, comprehensive review of existing research. It presents a comprehensive analysis of VLA applications across different scenarios and classifies VLA approaches into several paradigms: autoregression-based, diffusion-based, reinforcement-based, hybrid, and specialized methods; while examining their motivations, core strategies, and implementations in detail. In addition, foundational datasets, benchmarks, and simulation platforms are introduced. Building on the current VLA landscape, the review further proposes perspectives on key challenges and future directions to advance research in VLA models and generalizable robotics. By synthesizing insights from over three hundred recent studies, this survey maps the contours of this rapidly evolving field and highlights the opportunities and challenges that will shape the development of scalable, general-purpose VLA methods.

1 INTRODUCTIONS

VLA models extend multimodal generative modeling into executable robotic control, while this survey addresses the limited coverage and lack of consensus taxonomy for pure VLA methods. It organizes approaches, resources, applications, limitations, and future directions into a focused review.

  • Motivation: VLA foundation models combine vision-language generative paradigms, large-scale datasets, and fine-tuning strategies for fine-grained robotic action control.The cited methods include autoregressive and diffusion models.
  • Research gap: Existing reviews leave a significant gap because they emphasize VLM foundations or broad robotics histories rather than systematically examining pure VLA methods.The field also lacks an established methodological landscape and consensus taxonomy.
  • Survey scope: The survey classifies VLA approaches by action-generation strategy and analyzes their motivations, core strategies, mechanisms, applications, resources, and future directions.Its application coverage includes robotic arms, quadrupeds, humanoids, and wheeled robots.
  • Organization: The paper is organized around background, VLA approaches, datasets and benchmarks, simulation platforms, robotic hardware, and challenges and future directions.The final section summarizes the survey and presents perspectives on future developments.
  • Contributions: The survey contributes a structured taxonomy, methodological analysis, resource overview, and discussion of practical impacts, limitations, and future research avenues.Resources include datasets, benchmarks, and simulation platforms.

2 BACKGROUNDS

VLA models connect visual perception, language understanding, and executable control to extend multimodal intelligence into embodied tasks. Their development depends on foundational models, data resources, and simulators that support training, evaluation, and deployment.

  • Background: Traditional robotic systems generalize poorly beyond constrained environments because they rely on isolated perception, hand-engineered control, or task-specific reinforcement learning.VLA models target more dynamic and unstructured settings.
  • VLA formulation: VLA systems encode images, instructions, robot states, and sensory feedback before autoregressively generating action tokens that close the perception-language-action loop.This design unifies goals, constraints, and intent across modalities.
  • Resources: Datasets and simulators provide the data, standardized formats, and evaluation environments needed to develop and assess VLA systems.Representative resources include Open X-Embodiment, BridgeData, THOR, Habitat, MuJoCo, Isaac Gym, and CARLA.
  • Foundations: VLA models integrate visual encoders, language-model reasoning, and reinforcement-learning or control-based decision making for physical-world task execution.They build on advances in perception, reasoning, and sequential decision-making.

3 VISION-LANGUAGE-ACTION MODELS

VLA research has developed through several methodological paradigms whose innovations progressively expand model capabilities. The survey presents these paradigms chronologically and highlights representative works within each category.

  • Methodological paradigms: The survey reviews autoregressive, diffusion, reinforcement-learning, hybrid, and specialized VLA designs as major methodological paradigms.The taxonomy groups representative works by their architectural or policy strategies.
  • Taxonomy structure: The taxonomy is organized chronologically to show how methodological innovations have progressively expanded VLA capabilities.A tree diagram presents the progression and representative works in each branch.

3.1 Autoregression-Based Models in Vision-Language-Action Research

Autoregressive VLA models unify multimodal perception, language reasoning, and sequential action generation through token-based sequence modeling. The field has progressed toward generalist, cross-platform, reasoning-enabled, and deployment-efficient systems, while facing unresolved stability, alignment, safety, and resource constraints.

  • Generalist Sequence Modeling: Autoregressive VLA models generate actions step by step from prior context, perceptual inputs, and task prompts.
  • Generalist Sequence Modeling: Tokenizing heterogeneous modalities enables unified perception, task instruction, and action generation across diverse robotic tasks.
  • Reasoning and Generalization: Large-scale training and universal action abstractions have advanced autoregressive VLAs toward cross-platform compatibility, semantic reasoning, and efficient deployment.
  • Reasoning and Generalization: LLM integration extends VLA systems from semantic mediation toward interactive feedback, episodic memory, hierarchical planning, and platform-level orchestration.
  • Trajectory Generation: Autoregressive trajectory modeling supports grounded instruction following and action execution, with multimodal pretraining enabling cross-task generalization but often remaining simulation-bound.
  • Limitations and Efficiency: Current limitations include long-horizon instability, noisy-input grounding failures, inference latency, error accumulation, brittle multimodal alignment, and prohibitive data and compute demands.
  • Limitations and Efficiency: Structural optimization methods such as token compression, parallel decoding, quantization, caching, and adaptive computation improve efficiency for real-world deployment.

3.2 Diffusion-based Models in Vision-Language-Action Research

Diffusion-based VLA models recast robotic control as conditional generative action modeling, supporting diverse trajectories, geometry-aware representations, multimodal fusion, and structured reasoning. These advances improve generalization and deployment prospects, but temporal coherence, computation, dataset diversity, modality balance, and safety remain limiting factors.

  • Generalist Diffusion Methodologies: Diffusion-based VLAs formulate action generation as conditional denoising, enabling multiple valid trajectories from identical observations.
  • Generalist Diffusion Methodologies: Geometry-aware diffusion incorporates SE(3) constraints for physically consistent 3D actions, while video-generation formulations support temporal planning and cross-modal grounding.
  • Generalist Diffusion Methodologies: Trajectory-level diffusion with temporal and environmental conditioning supports zero-shot bimanual manipulation, while unified velocity fields and historical caching address temporal coherence and deployment.
  • Generalist Diffusion Methodologies: Diffusion VLA research transitions from deterministic to probabilistic generation, Euclidean to geometry-aware representations, and supervised to self-supervised paradigms.
  • Multimodal and Structured Architectures: Transformer-diffusion systems improve continuous action modeling, while token-space alignment and force-aware experts integrate heterogeneous visual, linguistic, proprioceptive, and tactile signals.
  • Multimodal and Structured Architectures: Structured reasoning, semantic scene graphs, pretrained-model reuse, affordance decomposition, and flowgraph methods move diffusion VLAs beyond purely end-to-end learning toward more interpretable designs.
  • Deployment and Adaptability: Efficiency-focused designs use pretrained backbones and parameter-efficient tuning, while task-specialized systems balance broad multimodal capability with domain-specific inductive biases.
  • Deployment and Adaptability: Diffusion VLAs improve trajectory diversity, geometric grounding, and reasoning integration, but remain constrained by fragile temporal coherence, high computation and data demands, limited dataset diversity, and underdeveloped safety-critical reliability.

3.3 Reinforcement-based Fine-Tune Models in Vision-Language-Action Research

Reinforcement-based VLA methods combine visual-language foundation models with reinforcement learning to produce context-aware actions, transferable reward representations, and policies across diverse embodiments. Their advances include hybrid offline-online optimization and safety-aware control, while noisy rewards, training instability, resource demands, and ambiguous-instruction generalization remain open challenges.

  • Reinforcement Fine-Tuning Strategies: Reinforcement-based VLAs integrate visual-language inputs with reinforcement learning to support perception, reasoning, and decision-making in interactive environments.
  • Reinforcement Fine-Tuning Strategies: Reward-aware methods learn dense or transferable reward proxies from language, images, demonstrations, and self-supervised representations for sparse-reward and complex-instruction settings.
  • Embodiment and Application Diversity: Reinforcement-based VLAs extend beyond robot arms to quadrupeds, humanoids, and autonomous driving, supporting navigation, obstacle avoidance, terrain adaptation, and whole-body control.
  • Optimization and Efficiency: Chain-of-thought reasoning and group relative policy optimization can produce discrete feasible actions that are reconstructed into continuous trajectories.
  • Optimization and Efficiency: Quantization, pruning, distillation, offline behavior cloning, and online reinforcement learning target faster inference, more stable optimization, and improved generalization.
  • Limitations: Key limitations include indirect or noisy rewards, instability between supervised fine-tuning and exploration, high computational costs, and unreliable generalization under ambiguous or adversarial instructions.

3.4 Other Advanced Researches

Advanced VLA research extends beyond individual generation paradigms through hybrid architectures, spatially grounded multimodal fusion, specialized domain adaptations, foundation-scale training, and deployment-oriented designs. These directions improve contextual reasoning, geometric grounding, adaptability, efficiency, and practical reliability, while remaining constrained by computational, data, robustness, and scalability challenges.

  • Hybrid Architectures: Hybrid architectures combine complementary generation paradigms to balance continuous action smoothness with precise contextual reasoning.HybridVLA combines diffusion-based trajectory generation with autoregressive token-level reasoning, while broader designs integrate attention mechanisms into diffusion planning.
  • Hybrid Architectures: Large-scale empirical evaluation and open-source design guidelines are helping standardize hybrid VLA development.OpenHelix benchmarks alternative reasoning–execution integration strategies and provides implementations alongside design guidance.
  • Advanced Multi-Modal Fusion and Spatial Understanding: Multimodal fusion is progressing from early 2D pathway separation toward 3D-aware representations that model geometry, affordances, and spatial constraints.CLIPort separates object identification from action localization, while later methods use multi-view fusion, relational keypoint graphs, affordance maps, and symbolic geometric constraints.
  • Specialized Methods: Specialized VLA adaptations extend the framework across domains by tailoring perception–reasoning–control pipelines to operational contexts.The survey identifies domain adaptations as a major direction and emphasizes their role across physical, digital, and hybrid environments.
  • Foundation Models and Large-Scale Training: Foundation-scale training combines massive multimodal datasets, efficient adaptation, and modular reasoning to support more generalist embodied agents.Fine-tuning studies examine action spaces, policy heads, and supervision signals, while lightweight reasoning achieved a 3× inference speedup compared with standard approaches.
  • Practical Deployment over Efficiency, Safety, and Human–Robot Collaboration: Deployment-oriented VLA research integrates real-time optimization, failure and adversarial robustness, safety, and human-in-the-loop refinement.The survey presents these elements as jointly necessary for persistent, reliable, and interactive robotic systems in real-world environments.
  • Discussion: The surveyed approaches remain limited by computational cost, scaling complexity, noisy inputs, narrow domain adaptation, resource-intensive foundation training, and deployment reliability challenges.The survey calls for more efficient training, broader evaluation standards, and tighter integration between research design and practical deployment.

4 DATASETS AND BENCHMARKS

VLA datasets and benchmarks combine multimodal real-world and simulated data with task-specific evaluation for embodied robotics and autonomous driving. The survey emphasizes expanding dataset scale and diversity, improving closed-loop simulation, and addressing gaps in realism, long-tail coverage, and metric adequacy.

  • Dataset Foundations: VLA datasets contain multimodal observations, labels, and language instructions collected from real-world or simulated robotic scenarios.Typical modalities include images, LiDAR point clouds, and IMU readings, supporting systematic dataset and benchmark organization.
  • Real-World Datasets and Benchmarks for Embodied Robotics: Real-world embodied robotics datasets provide temporally aligned sensory observations and actions for training and evaluating adaptive embodied policies.They capture visual, auditory, proprioceptive, and tactile inputs alongside motor actions, intentions, and environmental contexts.
  • Real-World Datasets and Benchmarks for Embodied Robotics: Real-world collection is costly, but BridgeData spans 71 tasks across 10 environments and joint training with unseen-domain tasks can double success rates over target-domain data alone.MIME, RoboNet, and MT-Opt also provide multiple demonstrations per task, supporting manipulation and VLA evaluation.
  • Real-World Datasets and Benchmarks for Embodied Robotics: Open X-Embodiment consolidates datasets across institutions to broaden skills, tasks, and environmental coverage for more generalizable manipulation policies.The survey presents collaborative multi-robot collection as a route toward geographically and contextually diverse embodied datasets.
  • Real-World Datasets and Benchmarks for Embodied Robotics: Embodied benchmarks commonly use Success Rate, Language Following Rate, and transfer to unseen environments to assess completion, instruction execution, robustness, and generalization.These measures target both task outcomes and policy transfer beyond training environments.
  • Real-World Datasets and Benchmarks for Autonomous Driving: Autonomous-driving datasets remain limited by open-loop collection dominated by normal behavior, restricting coverage of long-tail corner cases.Synthetic data, closed-loop interaction, and safety-critical-event curation are proposed responses to this coverage gap.
  • Simulation Datasets and Benchmarks: Simulation provides scalable training and evaluation through synthetic scenes, physics-based interactions, and annotations for navigation, manipulation, and task execution.ROBOTURK supplies high-quality 6-DoF manipulation states and actions, while iGibson0.5 evaluates interactive navigation through Path Efficiency and Effort Efficiency.
  • Simulation Datasets and Benchmarks for Autonomous Driving: Closed-loop simulation supports safety-critical autonomous-driving evaluation by modifying actors, trajectories, viewpoints, and multimodal sensor observations.UniSim reconstructs static and dynamic scene components to simulate LiDAR and camera data from novel viewpoints.

5 SIMULATORS

Robot simulators provide scalable, controllable environments that integrate physics, sensors, rendering, and task logic for developing and evaluating intelligent robotic systems. The survey covers platforms spanning photorealistic indoor scenes, realistic household replicas, physics engines, GPU simulation, and autonomous-driving environments.

  • Simulator capabilities: Simulators combine physics engines, sensor models, and task logic for navigation, manipulation, and multimodal instruction-following research.They support reinforcement learning, imitation learning, and large pre-trained models in safe, controllable, reproducible settings.
  • Indoor and embodied simulation: THOR offers near-photorealistic indoor scenes for navigation and object interaction, while Habitat and Habitat 2.0 provide scalable, physics-enabled 3D environments.ALFRED adds long-horizon, compositional tasks with irreversible state changes and both high- and low-level language instructions.
  • Household simulation: iGibson 1.0 and 2.0 support diverse household tasks in large-scale environments replicating real-world homes and their object layouts.These platforms contrast with earlier physics-based environments that used narrower scenarios and simplified scenes.
  • Advanced simulation: Advanced simulators support multi-agent interaction and diverse sensor and physics outputs, combining universal physics, flexible robotics simulation, and high-fidelity rendering.These capabilities support robotic simulation and generative-model evaluation.
  • Physics and GPU engines: MuJoCo provides an open-source physics engine, while NVIDIA Isaac Gym enables large-scale simulation and testing of AI-driven robots in physically realistic virtual environments.GPU-based simulation engines have become increasingly popular in robotics research and industry.
  • Autonomous driving: CARLA and LGSVL use game engines to simulate autonomous-driving environments containing static road elements and dynamic agents.Such simulators address the cost and difficulty of collecting real-world data for rare driving scenarios.

6 ROBOT HARDWARE

Robot hardware provides the physical basis for perception, locomotion, manipulation, and environmental interaction through integrated sensing, actuation, power, and control components.

  • Hardware components: Robot hardware typically includes sensors, actuators, power systems, and a control unit that support perception, movement, manipulation, and interaction.Cameras, LiDAR, inertial measurement units, and tactile arrays provide information about the environment and the robot’s internal state.

7 CHALLENGES AND FUTURE DIRECTIONS

The survey identifies bottlenecks in VLA data, architectures, interaction capabilities, evaluation, and responsible deployment. These challenges limit generalization, efficiency, causal grounding, and confidence in real-world use.

  • Data scarcity: Robotic datasets remain insufficient in scale and diversity, while simulation faces rendering, physics, task-modeling, and sim-to-real limitations.Open-source data emphasizes tabletop manipulation and grasping, and models may perform well in simulation yet fail on physical robots.
  • Architectural diversity: Architectural heterogeneity across vision encoders, language backbones, and action heads hinders comparison and reuse across VLA models.Examples span ViT, DINOv2, and SigLIP vision encoders; PaLM, LLaMA, and Qwen language backbones; and discrete, continuous, or diffusion-based actions.
  • Efficiency: Autoregressive Transformer decoding limits inference speed and execution efficiency because sequential action tokens accumulate latency.High-dimensional visual inputs and large parameter counts also impose substantial computation and memory costs, while dynamic tasks require millisecond-level responses.
  • Interaction and causality: Many VLA systems rely on static training patterns rather than genuine interaction grounded in environmental dynamics, causal reasoning, and sensor feedback.In unfamiliar settings, they may follow instructions superficially while failing to establish causal relationships.
  • Evaluation: Current benchmarks emphasize structured laboratory or simulated tabletop tasks and therefore inadequately measure open-world generalization and robustness.Performance often degrades substantially in outdoor, industrial, or complex household deployments.
  • Responsible deployment: VLA progress ultimately requires addressing controllability, trustworthiness, and safety alongside performance and generalization.The survey frames responsible deployment as a deeper long-term challenge for intelligent robotic agents.

7.2 Opportunities of Vision-Language-Action Models

The survey presents opportunities to unify language, perception, and action, strengthen causal interaction, scale diverse data ecosystems, and establish trustworthy societal integration. These directions aim to move VLA systems beyond instruction generation toward more general embodied intelligence.

  • Overview: VLA could become a central pathway for embodied intelligence by bridging semantic understanding and physical execution.The survey connects this opportunity to reshaping robotics research and supporting real-world deployment.
  • 7.2.1 Cross-modal unification: Cross-modal unification could let VLA jointly model environments, reasoning, and interaction within a single token stream.The survey describes this unified structure as a potential proto-world model that closes the loop from semantics to execution.
  • 7.2.2 Causal interaction: Causal modeling and interactive reasoning could enable robots to probe environments, validate hypotheses, and adapt strategies using feedback.This direction targets the gap between statistical pattern matching and genuine interaction with dynamic environments.
  • 7.2.3 Virtual–Real Integration and Large-Scale Data Generation: Integrating high-fidelity simulation, synthetic generation, and multi-robot sharing could produce datasets containing trillions of trajectories across diverse tasks.The survey presents such virtual–real data ecosystems as a route toward more robust open-world operation.
  • Societal integration: Safety, trustworthiness, ethical alignment, risk assessment, explainability, and accountability will shape VLA adoption in public and domestic spaces.Standardized frameworks could support a transition from laboratory artifacts to trusted partners and human–AI interfaces.

8 CONCLUSION

This survey systematically charts VLA approaches through a unified taxonomy, analyzes over 300 articles, and reviews the datasets, benchmarks, and simulation platforms supporting training and evaluation. It also identifies current strengths and limitations and outlines future directions toward trustworthy, continually evolving VLA systems.

  • The survey analyzes over 300 articles to systematically chart VLA motivations, methodologies, and applications.
  • It proposes a unified taxonomy covering autoregression-based, diffusion-based, reinforcement-based, hybrid, and efficiency-oriented VLA approaches.
  • The review examines datasets, benchmarks, and simulation platforms that support VLA training and evaluation.
  • The survey analyzes the strengths and limitations of current approaches and highlights potential directions for future research.
  • Its consolidated insights provide a reference and roadmap for developing trustworthy, continually evolving VLAs in robotic systems.
Loading 2509.19012v3…