Source-linked AI summary
ROS2SmolVLA: Enabling Small Vision-Language-Action Models for Integration into Industrial-Grade Lightweight Robots
Nils Mandischer, Noah Böckmann, Ludwig Holl, Lars Mikelsons
TL;DR
Industrial settings need adaptive robot automation, but smaller VLAs have not been comprehensively evaluated on industrial-grade hardware under real production constraints. This paper integrates SmolVLA with Universal Robots through ROS 2, validates it on a UR10e pick-and-place task, and finds general feasibility despite insufficient robustness and currently low success rates for real-world deployment.
Problem
Smaller VLAs suitable for on-premise and edge computation have not been comprehensively benchmarked with industrial-grade robots under real production constraints.
Method
The authors develop ROS2SmolVLA, a ROS 2 interface integrating SmolVLA with Universal Robots, decentralized edge-to-workstation inference, heterogeneous camera inputs, and robot actuation.
Results
Overall success rates were 77.72% for picks, 63.59% for places, and 81.69% for places after successful picks across all test cases.
Takeaways & Limitations
The validation indicates general feasibility for SmolVLA on industrial-grade hardware and suggests viability for small-scale industrial applications with suitable improvements.
Takeaways & Limitations
SmolVLA lacks robustness across holistic deployment scenarios, and current success rates are too low for real-world deployment, mainly because of insufficient training data.
Abstract
from arXiv · showhide
Industrial demand changes the paradigms of production. Due to smaller batch sizes and more variations in products, companies face a growing challenge to adopt more adaptive production systems. In particular, robot-based automation is usually static and fails to respond to constantly changing processes. Vision-Language-Action (VLA) Models are a promising opportunity to mitigate this challenge by generating robot actions based on the observed system state. However, current research either focuses on large models that cannot be computed on premise, creating compliance and security challenges, or use lab-grade robot hardware that obscures exploitation in real industrial settings. In this work, we adapt Hugging Face's SmolVLA for Universal Robots lightweight robots. Further, we release the open-source repository ROS2SmolVLA that implements an interface for ROS 2 to SmolVLA, and makes it applicable for industrial-grade hardware. By this, we allow a lenient adoption into lab and industrial environments. We validate the functionality of SmolVLA for a Universal Robots UR10e using a pick-and-place task and give implementation guidelines. Our findings support that SmolVLA is a well-suited option for small-sized tasks that need to be computed on premise.
1. Introduction
Industrial production is shifting toward flexible, adaptive systems, while conventional automation struggles with changing products and processes. ROS2SmolVLA addresses this challenge by bringing SmolVLA to Universal Robots and validating it on industrial-grade hardware.
- Motivation: Smaller batches and greater product variety require production systems that can adapt to changing processes.Established automation performs well in structured environments but is less suited to on-demand, variable production.
- Motivation: Flexible and reconfigurable manufacturing systems extend automation toward broader task sequences, functionality, and adaptability.AI contributes capabilities associated with smart reconfigurable manufacturing, including customization and scalability.
- Contribution: ROS2SmolVLA provides a ROS 2 interface that bridges SmolVLA from laboratory experiments to industrial-grade hardware.The system targets direct execution on Universal Robots using edge devices and consumer-grade hardware.
- Validation: The authors validate ROS2SmolVLA on a Universal Robots UR10e using a pick-and-place task and report deployment lessons for larger workspaces.The paper also releases the project and associated software and data resources.
2. Related Work
VLA research has advanced robot action generation and cross-embodiment control, but industrial deployment remains constrained by latency, hardware demands, and limited hardware validation. SmolVLA offers a lightweight route toward on-premise inference, motivating its adaptation to industrial robots.
- Vision-Language-Action Models: VLA architectures generate robot actions from multimodal observations and language, extending semantic reasoning to robotic control.Later systems co-train vision-language backbones with physical robot trajectories and formulate action generation as token prediction.
- Cross-Embodiment Control: Cross-embodiment policies aim to control robots with diverse kinematics, while smaller models support adaptation and inference on consumer-grade hardware.Octo is described as a lightweight example with 27M or 93M parameters and flexible observation handling.
- Industrial Constraints: Industrial deployment is constrained by the fast control frequencies required in production, making inference latency a significant concern for large VLAs.Quantization, tensor parallelism, and action chunking are identified as strategies used to address latency.
- Industrial Constraints: Cloud-based inference can raise compliance and privacy concerns, while smaller VLAs reduce compute demands but may provide less semantic generalization.The related work frames on-premise and edge computation as an alternative to proprietary cloud services.
- Industrial Constraints: Smaller VLAs have mainly been benchmarked on simple laboratory hardware, leaving industrial control loops, safety requirements, payloads, and workspaces insufficiently represented.The paper identifies this hardware gap as obscuring real industrial applicability.
- SmolVLA: SmolVLA is a 450M-parameter lightweight VLA designed for on-premise execution and fine-tuning on consumer-grade hardware.Its efficiency mechanisms include bounded visual tokenization, layer skipping, interleaved attention, and a Flow Matching Transformer action head.
3. ROS2SmolVLA: Adapting SmolVLA for Industrial-Grade Lightweight Robots
ROS2SmolVLA is a ROS 2-based interface that integrates SmolVLA with Universal Robots through reusable software, data, and hardware components. Its architecture separates robot execution from multimodal inference and supports simulation, heterogeneous cameras, and Cartesian control.
- System Components: ROS2SmolVLA adapts SmolVLA for industrial-grade lightweight robots and validates the interface with Universal Robots UR10e data.The project builds on LeRobot, a Cartesian Motion Controller, and a Robotiq Hand-E driver.
- System Components: The project includes Docker integration, UR10e simulation and real execution, ROS 2 camera integration, a LeRobot/SmolVLA interface, and released data and models.These resources are distributed through Docker, GitHub, and Hugging Face repositories.
- System Architecture: The architecture uses a decentralized edge-to-workstation design to separate real-time hardware execution from resource-intensive multimodal inference.The lerobot-ros communication layer brokers interactions between LeRobot and ROS 2 without requiring policies to manage low-level industrial bus cycles.
- System Architecture: Cartesian control and published pose feedback make the interface more portable across robot embodiments than direct joint control.Joint states and the end-effector’s Cartesian pose are broadcast across the local ROS 2 ecosystem.
- System Architecture: ROS2SmolVLA’s architecture connects self-developed components and external dependencies through ROS 2 topics and the LeRobot interface.Figure 1 distinguishes self-developed software from external dependencies and uses different arrow types for the two communication paths.
- Data and Execution: The data-collection layer aggregates asynchronous inputs from two static cameras and an end-effector-mounted camera across physical and Gazebo platforms.The self-developed camera package standardizes these streams through the LeRobot camera interface.
- Data and Execution: Containerization isolates CUDA, PyTorch extensions, and ROS 2 packages to support reproducible execution across development machines, edge devices, and production workstations.The containerized wrapper also manages model execution on the inference workstation.
4. Validation
The validation evaluates ROS2SmolVLA with an industrial-grade UR10e in a structured pick-and-place task spanning positions, configurations, colors, geometries, and object counts. Results indicate general feasibility on industrial hardware, alongside weaknesses in spatial reasoning, color adherence, receptacle interpretation, visual coverage, and recovery robustness.
- Validation setup: The validation uses a UR10e pick-and-place task with nine scenarios testing colors, positions, configurations, geometries, and object count.The action is evaluated through pick and place sub-tasks, with automatic recovery counted as success.
- Validation setup: The distributed setup combines two Azure Kinect cameras, a wrist-mounted webcam, a Robotiq gripper, Jetson AGX Orin processing, and RTX 4080 inference hardware.Camera streams are concatenated on the edge PC, while the VLA runs on a workstation connected through a 10Gbit Ethernet switch.
- Test plan: The test plan varies cube and box positions, robot configuration, cube color, box color, and box shape across ID and OOD conditions.T1–T5 cover position, configuration, and color variations; T6–T9 add object-count, color-prompt, receptacle-color, and receptacle-shape tests.
- Results: 77.72% pick, 63.59% place, and 81.69% place-after-successful-pick success were achieved across all test cases.ID success rates were [78.33%, 72.50%, 92.47%], while OOD rates were [76.56%, 46.88%, 61.22%] for the same metrics.
- Results: The system showed spatial failures at some box positions, strong color bias, poor prompt adherence for gray cubes, and limited interpretation of novel receptacles.The policy often defaulted to green cubes and appeared to use blue wrist-camera color as a dropping trigger; black-box detection mostly failed.
- Lessons learned: Low image resolution, occlusion, small objects, training-set color and texture reliance, and sparse recovery data constrain robustness across holistic deployment scenarios.Failure-recovery episodes improved robustness in larger datasets but could induce unwanted behavior during sparse-data training; synthetic-data validation remained incomplete.
5. Conclusion
ROS2SmolVLA integrates SmolVLA with industrial-grade lightweight robots through ROS 2 and supports synchronized vision, actuation, simulation, and containerized deployment. Validation on a UR10e found general viability, but success rates remain too low for real-world deployment, mainly because of insufficient training data.
- ROS2SmolVLA integrates SmolVLA with industrial-grade lightweight robots through ROS 2.The framework includes components for containerization, SmolVLA wrapping, heterogeneous camera inputs, robot actuation, and Gazebo simulation.
- The system supports synchronized vision from multiple heterogeneous cameras and actuation on both physical robot hardware and its digital twin.
- The model was trained for pick-and-place using 349 episodes and evaluated on compute-restricted hardware.
- Validation showed general viability, but success rates are currently too low for real-world deployment.The paper attributes this mainly to a lack of training data.
- The authors provide implementation lessons and release code, data, and model parameters for adopters.Future work could leverage camera depth information, which was considered during system design but not used.