PaperlayerView exact revision on arXiv

Source-linked AI summary

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

Hengyi Xie, Chenfei Yao, Xianjin Wu, Yingying Zhu, Dingkang Liang, Xiang Bai, Han Ding

arXiv:2607.27205v2cs.CVcs.RO

TL;DR

LLM-centric VLA policies impose substantial computation and memory costs for real-time robotic execution. TurboVLA directly couples lightweight vision and language features for action prediction, achieving 97.7% average success on LIBERO with 31.2 ms latency and 0.9 GB inference VRAM.

  • Problem

    LLM-centric VLA models create a substantial bottleneck for real-time robotic execution because of their computation and memory demands.

    Source: §1 · p. 2

  • Method

    TurboVLA separately encodes visual observations and instructions, directly fuses them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks.

    Source: §1 · p. 3, §4.1 · p. 6

  • Results

    97.7% average success on LIBERO is achieved with 31.2 ms latency, 0.2B parameters, and less than 1 GB inference VRAM on an RTX 4090.

    Source: §1 · p. 3

  • Takeaways & Limitations

    TurboVLA suggests that execution-level robotic control need not use a general-purpose LLM as the central interface between perception and action.

    Source: §5.3 · p. 8, §6 · p. 11

Abstract

from arXiv · show

Vision-language-action (VLA) models commonly adopt an LLM-centric $V \to L \to A$ pathway, where visual observations are projected into the representation space of a large language model before being decoded into robot actions. Although effective, this design incurs substantial computation and memory overhead at every policy invocation. In this work, we introduce TurboVLA, a new VLA paradigm that reformulates the conventional $V \to L \to A$ pathway as a direct $V + L \to A$ mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks with a compact decoder. This simple design constructs task-conditioned representations directly from visual and linguistic features, significantly reducing the computational and memory costs of VLA inference. On LIBERO, TurboVLA achieves 97.7% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB inference VRAM on a consumer-grade RTX 4090, matching or outperforming substantially larger VLA policies. These results establish TurboVLA as a simple and effective alternative to the prevailing LLM-centric VLA paradigm, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.

1 INTRODUCTION

TurboVLA removes the large language model from execution-level control by directly combining visual and linguistic features for continuous action prediction. On LIBERO, it achieves 97.7% average success at 32 Hz with 0.2B parameters and 0.9 GB inference VRAM on an RTX 4090.

Motivation

  • TurboVLA replaces the LLM-centered V → L → A pathway with direct V + L → A execution-level control.The approach retains language conditioning while removing the large language model from the execution pathway.Source: §1 · pp. 2–3
  • LLM-centric VLA models impose substantial computation and memory overhead because multimodal inputs pass through billion-parameter language models before action prediction.Autoregressive models additionally inherit sequential decoding costs, while action experts still retain large language-model cores.Source: §1 · p. 2
  • The design targets execution-level manipulation because instructions specify the intended skill, making open-ended language generation and autonomous task decomposition unnecessary.TurboVLA combines lightweight instruction encoding, direct vision-language interaction, robot-state conditioning, and nonautoregressive action prediction.Source: §1 · pp. 2–3

Method

  • TurboVLA separately encodes vision and instructions, fuses them with compact cross-attention, and predicts continuous action chunks in one forward pass.A lightweight text encoder supplies execution-relevant instruction semantics, while direct vision-language interaction forms a control-oriented representation.Source: §1 · p. 3

Results

  • TurboVLA achieves 97.7% average success on LIBERO at 31.2 ms latency (32 Hz), with 0.2B parameters and less than 1 GB inference VRAM on an RTX 4090.This matches substantially larger VLA systems while lowering the hardware barrier for latency-sensitive and resource-constrained deployment.Source: §1 · p. 3

2 RELATED WORK

Prior VLA research advances multimodal generalization, richer control representations, and efficient execution, but typically retains large multimodal backbones in the control pathway. TurboVLA instead directly combines compact visual, textual, and proprioceptive features for execution-level continuous control.

Vision-language-action models

  • VLA research spans pretrained multimodal policies, continuous action experts, cross-embodiment datasets, diffusion policies, and increasingly predictive or reasoning-based representations.Representative systems include RT-1 (Brohan et al., 2023), RT-2, OpenVLA (Kim et al., 2024), π0, and π0.5 (Physical Intelligence et al., 2025).Source: §2 · p. 3
  • Geometry-aware and dynamic-manipulation methods add pose, point-action, temporal-motion, or future-prediction representations, whereas TurboVLA directly integrates task text, vision, and robot state for action prediction.This architectural choice avoids routing every control step through a large generative multimodal backbone.Source: §2 · p. 4

Efficient execution in VLA policies

  • Efficient VLA methods redesign actions, compact models, or backbone computation, but largely retain large multimodal backbones as execution representations; TurboVLA removes that generative language backbone.Prior efficiency techniques include continuous action experts, structured tokenizers, action chunking, quantization, token pruning, dynamic depth, distillation, asynchronous execution, streaming inference, and speculative inference.Source: §2 · p. 4

Language interfaces for robot control

  • Language-conditioned robot policies establish text as an effective task input, while TurboVLA tests whether compact text encoding and direct vision-text interaction suffice for real-time continuous control.Earlier systems range from direct imitation policies and CLIPort (Shridhar et al., 2022) to long-horizon control, voxel policies, interleaved prompts, and planning-oriented embodied multimodal models.Source: §2 · p. 4

3 PRELIMINARIES

The preliminaries characterize prevailing VLA models as an LLM-centered V → L → A pathway and motivate direct bidirectional vision-language interaction for control-oriented action prediction.

LLM-centric vision-language-action models

  • LLM-centric VLA models project visual features into language-model space, combine them with task instructions, and use the LLM as the central bridge to action prediction.Both autoregressive and action-expert designs retain this representational dependency, despite differing in whether actions are generated sequentially or in parallel.Source: §3 · pp. 4–5

Direct vision-language interaction

  • Direct bidirectional cross-attention exchanges visual and language features, producing vision-aware instruction features and language-conditioned visual features for continuous action prediction.This interaction lets task language shape visual processing while visual context refines instruction representations, adapting cross-modal grounding for control.Source: §3 · p. 5, Figure 3 · p. 5

4 TURBOVLA

TurboVLA replaces the LLM-centric V → L → A pathway with a direct V + L → A architecture for execution-level language-conditioned manipulation. It uses compact modality-specific encoding, bidirectional vision-language interaction, and parallel continuous action-chunk decoding.

4 TURBOVLA

  • TurboVLA directly combines encoded visual observations, task instructions, and robot state to predict continuous action chunks without routing inputs through a large language model.The architecture uses a compact vision-language interaction module followed by an action chunk decoder in a single forward pass.Source: §4 · p. 6

4.1 MULTIMODAL FEATURE ENCODING

  • Complete instruction-token sequences preserve objects, attributes, and spatial relations for fine-grained visual conditioning, while modality-specific encoders reduce intermediate activation memory and downstream attention cost.Visual features retain spatial and camera-view information across multiple viewpoints, and robot-state features enter only the action decoder.Source: §4.1 · p. 6

4.2 VISION-LANGUAGE INTERACTION MODULE

  • Bidirectional cross-modal layers inject scene context into instructions and task semantics into visual features, producing task-specific multimodal representations without a large language model.The interaction module uses layer normalization, bidirectional cross-attention, modality-specific feedforward networks, and residual connections.Source: §4.2 · pp. 6–7

4.3 CONTINUOUS ACTION CHUNK PREDICTION

  • Parallel learnable action queries decode the complete H-step continuous action chunk in one forward pass, trained with behavior-cloning L1 loss without action tokenization or auxiliary language modeling.Robot-state features are introduced at decoding, preserving the preceding interaction module’s focus on task-conditioned scene understanding.Source: §4.3 · p. 7

5 EXPERIMENTS

TurboVLA is evaluated across simulation, bimanual, and real-world manipulation, achieving strong success rates while substantially reducing model size, latency, and inference memory. Ablations show that semantic language conditioning, lightweight instruction encoding, bidirectional interaction, and tuned interaction depth and action horizon are central to performance.

5.1 IMPLEMENTATION DETAILS

  • TurboVLA uses DINOv3 and BERT encoders, six bidirectional interaction layers in a 256-dimensional shared space, and an ACT-style decoder trained with ℓ1 behavior cloning.Training uses a learning rate of 5 × 10−5 on four RTX 4090 GPUs.Source: §5.1 · p. 7

5.2 BENCHMARKS AND METRICS

  • The experiments measure task success across LIBERO, RoboTwin 2.0, and real deployment, alongside parameter count, latency, and peak inference VRAM under batch size one on an RTX 4090.LIBERO reports average success over 2,000 trials, RoboTwin reports average success over 50 clean-setting bimanual tasks, and real-world tasks use 40 trials each.Source: §5.2 · pp. 7–8

5.3 MAIN RESULTS

  • 97.7% average success on LIBERO and 60.2% on RoboTwin 2.0 show TurboVLA matching or exceeding larger VLAs while using lower latency and a compact direct V + L → A pathway.On LIBERO, TurboVLA reaches 97.7% versus 96.9% for π0.5 at 31.2 ms versus 93.6 ms; on RoboTwin 2.0, it reaches 60.2% versus 57.0% for π0.5 and 50.3% for StarVLA-α, at 43.4 ms.Source: §5.3 · p. 8, Table 1 · p. 9, Table 2 · p. 9
  • 92.5%, 80%, 90%, and 87.5% success across four AgileX Piper tasks consistently outperform π0.5, while the complete policy uses 31.2 ms inference and 0.9 GB VRAM.The real-world evaluation uses a LIBERO-pretrained checkpoint fine-tuned on 4 × 65 demonstrations, with each task evaluated over 40 trials.Source: §5.2 · p. 8, §5.3 · pp. 9–10, Figure 4 · p. 10

5.4 ABLATION STUDY

  • Removing language drops average success from 97.7% to 70.8%, while T5-small reaches 97.1% and SigLIP reaches 95.5%, showing lightweight semantic encoders remain effective.The largest language-removal decline occurs on LIBERO-Goal, from 97.4% to 11.6%; task-ID embeddings remain 2.3% below the full model.Source: §5.4 · p. 10, Table 3 · p. 11, Table 4 · p. 11
  • Bidirectional interaction achieves 97.7% versus 95.2% for concatenation and 96.1% or 96.5% for one-way variants, while N = 6 and H = 12 produce the best reported settings.Average success rises from 93.5% at N = 2 to 97.7% at N = 6 before falling to 96.6% at N = 8; varying H gives 96.4% at H = 8, 97.7% at H = 12, and 95.6% at H = 15.Source: §5.4 · pp. 10–11, Table 5 · p. 11, Figure 5 · p. 11

6 CONCLUSION

TurboVLA presents an efficient V + L → A architecture that retains task-conditioned manipulation while reducing model size, inference latency, and memory use. Its execution-focused design leaves complex high-level planning as future work through integration with LLM-based planners.

Conclusion

  • TurboVLA combines lightweight instruction encoding, compact visual representations, bidirectional vision-language interaction, and action-chunk decoding to reduce model size, inference latency, and memory consumption while preserving task-conditioned manipulation.The architecture moves beyond the conventional LLM-centered execution pathway and questions whether a general-purpose LLM must mediate perception and action.Source: §6 · p. 11
  • TurboVLA is designed for concrete execution-level instructions and may not provide the complex semantic understanding and reasoning required for high-level task planning.Future work will combine LLM-based high-level planning with TurboVLA’s efficient execution pathway in hierarchical systems.Source: §6 · p. 11

Related papers

Loading 2607.27205v2…