Source-linked AI summary
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, Remi Cadene
TL;DR
Robotics VLAs remain costly, large, and dependent on non-community datasets, limiting accessible real-world deployment and participation. SmolVLA combines a compact VLA, community-collected training data, and asynchronous inference, and matches or surpasses much larger VLAs across simulated and real-world tasks. Its current pretraining uses data from only one robot type, leaving cross-embodiment generalization as an open limitation.
Problem
Existing VLAs are often massive and rely on academic or industrial datasets, creating cost, deployment, and data-access limitations for robotics.
Method
SmolVLA combines a compact pretrained VLM with a flow-matching action expert, community-collected imitation-learning data, and asynchronous execution that decouples action execution from perception and prediction.
Results
SmolVLA matches or surpasses much larger VLA models across simulated and real-world robotic tasks.
Takeaways & Limitations
The open-source model, code, datasets, hardware, and instructions support reproducible VLA research on consumer-grade hardware.
Takeaways & Limitations
SmolVLA’s pretraining data currently comes from a single robot type, so training across multiple embodiments may be important for generalizing to new platforms.
Abstract
from arXiv · showhide
Vision-language models (VLMs) pretrained on large-scale multimodal datasets encode rich visual and linguistic knowledge, making them a strong foundation for robotics. Rather than training robotic policies from scratch, recent approaches adapt VLMs into vision-language-action (VLA) models that enable natural language-driven perception and control. However, existing VLAs are typically massive--often with billions of parameters--leading to high training costs and limited real-world deployability. Moreover, they rely on academic and industrial datasets, overlooking the growing availability of community-collected data from affordable robotic platforms. In this work, we present SmolVLA, a small, efficient, and community-driven VLA that drastically reduces both training and inference costs, while retaining competitive performance. SmolVLA is designed to be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs. To further improve responsiveness, we introduce an asynchronous inference stack decoupling perception and action prediction from action execution, allowing higher control rates with chunked action generation. Despite its compact size, SmolVLA achieves performance comparable to VLAs that are 10x larger. We evaluate SmolVLA on a range of both simulated as well as real-world robotic benchmarks and release all code, pretrained models, and training data.
1 Introduction
SmolVLA addresses limited robotic generalization, high VLA resource demands, and restricted openness by combining a compact open-source model with community data and asynchronous inference. It matches or surpasses much larger VLAs across simulated and real-world tasks.
- Robotic policies struggle to generalize across objects, positions, environments, and tasks, partly because high-quality, diverse data remains limited.
- Open-source, reproducible VLA models and training recipes are presented as important for broader participation and faster robotics progress.
- SmolVLA is an open-source compact VLA released with reproducible, efficient training and inference recipes.
- Its lightweight architecture targets consumer-grade GPU training and CPU deployment, using layer skipping, fewer visual tokens, small pretrained VLMs, and lighter cross-attention.
- SmolVLA uses fewer than 30k episodes from publicly available community-contributed datasets, an order of magnitude less data than prior art.
- Asynchronous inference decouples action execution from observation processing and prediction, reducing latency and enabling fast, resource-efficient inference.
- SmolVLA matches or surpasses much larger VLA models despite being significantly smaller, across simulated and real-world evaluations.
2 Related work
Related VLA research progresses from models trained from scratch toward pretrained VLM-based policies, open-source systems, and efficiency-focused designs. SmolVLA shares the goal of combining strong performance with efficient training and inference.
- VLAs process natural-language instructions, visual observations, and proprioceptive inputs to output control actions across tasks, environments, and robot embodiments.
- Octo and RT-1 trained transformer policies from scratch on large-scale robotic demonstrations, while RT-2 adapted pretrained VLMs with robotics-specific data.
- OpenVLA promotes openness with a 7B-parameter VLA trained on publicly available data, generating discrete action tokens.
- Diffusion-based decoders were proposed to address limitations of action tokenization for continuous control, while TinyVLA pursued a lightweight sub-1B model.
- TinyVLA’s lack of large-scale robotics pretraining may hinder wider generalization, whereas SmolVLA targets open-source performance with efficient training and inference.
3 SmolVLA: small, efficient and capable
SmolVLA combines a compact pretrained VLM with a flow-matching action expert, community-collected training data, and an asynchronous inference stack. Its design reduces visual and computational overhead while balancing responsiveness against resource use.
- Model architecture: SmolVLA combines a pretrained VLM for perception with a flow-matching action expert that predicts action chunks from multimodal observations.The VLM processes multiple RGB images and language instructions, while the action expert produces controls conditioned on VLM features.
- Lightweight architecture: 64 visual tokens per frame reduce inference cost by using only the global image rather than tiled high-resolution crops.SmolVLM-2 applies pixel shuffling and omits image tiling to limit the visual-token budget.
- Model architecture: Interleaved cross-attention and self-attention layers connect the action expert to VLM features, with self-attention contributing to smoother real-robot action chunks.Cross-attention accesses VLM keys and values, while self-attention operates over action tokens.
- Community-driven pretraining: Community datasets provide diverse real-world settings but require annotation cleanup and camera-view normalization before pretraining.The authors address vague or missing task descriptions with a VLM and standardize camera ordering across datasets.
- Asynchronous inference: Synchronous inference lowers average computational burden but can create blind idle lags, whereas asynchronous inference overlaps prediction with execution to improve responsiveness.The asynchronous stack aggregates newly predicted chunks with the remaining action queue and supports remote policy-server execution.
4 Experiments
SmolVLA is evaluated across simulated and real-world manipulation benchmarks, with experiments covering overall performance, efficiency, inference speed, and architectural design choices. The results show strong performance from a compact model, benefits from community-data pretraining and multitask learning, and faster asynchronous execution.
- Simulation Evaluation: SmolVLA outperforms Octo, OpenVLA, and diffusion policy baselines across LIBERO and Meta-World, while remaining competitive with robotics-pretrained π0.It also consistently outperforms the VLM-initialized π0 variant despite not being pretrained on robotics data.
- Effect of pretraining and multitask learning: Pretraining on community datasets improves performance from 51.7 to 78.3, while multitask finetuning provides further gains through cross-task knowledge transfer.
- Asynchronous inference: 9.7 seconds versus 13.75 seconds shows asynchronous inference completes the task approximately 30% faster than synchronous inference.With a fixed time limit, asynchronous inference completes 19 successful pick-and-place cycles versus 9 synchronously, while maintaining comparable success rates.
- Ablation Study: More frequent observation updates improve success rate, whereas executing longer predicted action chunks improves inference speed but reduces responsiveness to environmental changes.
5 Discussion
SmolVLA is presented as a compact, efficient, open-source VLA for consumer-grade hardware and low-cost robots, with asynchronous inference for faster responsiveness. The authors identify dataset diversity, dataset scale, model scaling, and VLM-backbone choice as remaining limitations.
- SmolVLA runs on consumer-grade hardware, controls low-cost robots, and rivals significantly larger VLAs.Its architecture targets efficient training and inference without compromising success rate.
- Asynchronous inference decouples action execution from observation processing and action prediction to enable faster real-world manipulation.The strategy is model-agnostic and applies to policies that output action chunks.
- Limitations: Pretraining currently uses datasets collected from a single robot type, SO100, limiting dataset diversity across embodiments.The authors report fine-tuning to different robots but argue that multiple embodiments are likely critical for generalization.
- Limitations: The training dataset contains approximately 23k trajectories, substantially fewer than the approximately 1 million used by OpenVLA.The authors identify dataset expansion as a direction for improving performance and generalization.
- Limitations: SmolVLA has less than 0.5 billion parameters, while future work should explore scaling architectures without sacrificing speed or accessibility.The authors also question whether the off-the-shelf VLM backbone, pretrained mainly on document reading and OCR, is optimal for robotic interaction.
6 Aknowledgements
The authors acknowledge contributors who supported the community and simulated datasets, asynchronous inference, and the project’s broader development.
- The authors thank contributors for assistance with community and simulated datasets, asynchronous inference, and project support.
A.1 Community datasets
The community-dataset appendix lists the datasets used for SmolVLA’s task-annotation and pretraining pipeline. These entries comprise numerous publicly identified robot-task datasets, many associated with SO100 data collection.
- Task annotation begins by prompting a VLM with a current task description.The prompt asks for a concise one-sentence description of the robot arm’s action.
- The appendix lists community datasets covering tasks such as picking, placing, folding, pouring, pressing, sorting, and manipulation.The listed sources include many SO100-associated datasets and additional community collections.