Source-linked AI summary
EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models
Yantai Yang, Yuhao Wang, Zichen Wen, Luo Zhongwei, Chang Zou, Zhipeng Zhang, Chuan Wen, Linfeng Zhang
TL;DR
Diffusion-based VLA models face high computational and memory demands, while prior acceleration methods often address isolated inefficiencies. EfficientVLA is a training-free framework that jointly prunes redundant language layers, selects compact diverse visual tokens, and caches diffusion features. On CogACT in SIMPLER, it achieves a 1.93× inference speedup with FLOPs reduced to 28.9% and a 0.6% success-rate drop.
Problem
Diffusion-based VLA architectures have high computational and memory overhead, and existing acceleration efforts often target isolated inefficiencies across the pipeline.
Method
EfficientVLA combines similarity-guided pruning of redundant language layers, task-aware diverse visual-token selection, and caching of intermediate attention and MLP features during diffusion denoising.
Results
1.93× inference speedup and FLOPs reduced to 28.9% are achieved on CogACT in SIMPLER with only a 0.6% success-rate drop.
Takeaways & Limitations
A coordinated, training-free treatment of language, visual, and action-module redundancies improves VLA inference efficiency within the reported CogACT and SIMPLER evaluation.
Takeaways & Limitations
The demonstrations are primarily on CogACT because open-source diffusion-based VLA models are limited, and fixed cache intervals trade acceleration against action fidelity.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models, particularly diffusion-based architectures, demonstrate transformative potential for embodied intelligence but are severely hampered by high computational and memory demands stemming from extensive inherent and inference-time redundancies. While existing acceleration efforts often target isolated inefficiencies, such piecemeal solutions typically fail to holistically address the varied computational and memory bottlenecks across the entire VLA pipeline, thereby limiting practical deployability. We introduce EfficientVLA, a structured and training-free inference acceleration framework that systematically eliminates these barriers by cohesively exploiting multifaceted redundancies. EfficientVLA synergistically integrates three targeted strategies: (1) pruning of functionally inconsequential layers from the language module, guided by an analysis of inter-layer redundancies; (2) optimizing the visual processing pathway through a task-aware strategy that selects a compact, diverse set of visual tokens, balancing task-criticality with informational coverage; and (3) alleviating temporal computational redundancy within the iterative diffusion-based action head by strategically caching and reusing key intermediate features. We apply our method to a standard VLA model CogACT, yielding a 1.93X inference speedup and reduces FLOPs to 28.9%, with only a 0.6% success rate drop in the SIMPLER benchmark.
1 Introduction
Diffusion-based VLA models combine powerful multimodal reasoning and iterative action generation but face substantial computational and memory overhead. EfficientVLA addresses bottlenecks and redundancies across the language, visual, and action modules with a structured, training-free framework.
- Motivation: Diffusion-based VLAs integrate a vision encoder, LLM, and iterative diffusion action decoder, creating substantial computational and memory overhead.The language module and diffusion head are identified as primary contributors to latency and computational load.
- Motivation: Visual token pruning reduces inference time initially but becomes less effective when the system is memory-bound by the LLM.This reveals that optimizing visual inputs alone does not address the full inference bottleneck.
- Motivation: Existing acceleration efforts often optimize isolated modules, leaving other VLA inefficiencies and bottlenecks unaddressed.The paper motivates a holistic approach because gains can be limited when memory-intensive language processing or compute-intensive action generation remains unchanged.
- EfficientVLA: EfficientVLA prunes functionally inconsequential language layers using similarity-derived importance to reduce model depth and memory demands without retraining.The framework targets depth-wise redundancy in language representations.
- EfficientVLA: EfficientVLA selects task-relevant visual tokens, augments them for informational diversity, and caches intermediate attention and MLP features across diffusion steps.These strategies address visual redundancy and temporal redundancy in the action generator.
- Results: 1.93× inference speedup and FLOPs reduced to 28.9% are achieved with only 0.6% accuracy degradation in SIMPLER experiments on CogACT.The reported results support the framework’s effectiveness as a combined acceleration strategy.
2 Related Work
Diffusion-based VLAs use pretrained vision-language models to process multimodal inputs and separate diffusion modules to generate continuous action trajectories. EfficientVLA is presented as a training-free, structured approach that combines language-layer pruning, task-aware visual-token selection, and diffusion-feature caching.
- Vision-Language-Action Models: Diffusion-based VLAs commonly use a componentized design in which a VLM produces representations conditioning a separate diffusion-based action module.The action module iteratively generates precise continuous action trajectories.
- Efficient Vision-Language-Action Models: VLA acceleration methods are categorized into training-aware and training-free approaches targeting the computational complexity of vision-language processing and action generation.The related-work discussion places EfficientVLA among training-free acceleration methods.
- Efficient Vision-Language-Action Models: EfficientVLA combines redundant language-layer pruning, task-aware and diverse visual-token selection, and temporal caching in the diffusion action head.The framework overview describes these as coordinated components of a training-free acceleration strategy.
3 Method
EfficientVLA reduces VLA inference overhead by identifying redundant language layers and visual tokens, then retaining task-relevant and diverse visual information. Its method combines importance-driven layer pruning with relevance- and diversity-aware visual token selection.
- Language Layer Pruning: The language module exhibits depth-wise redundancy, with high similarity between layer inputs and outputs indicating functionally less critical layers.The method uses this observation to identify layers whose transformations minimally change hidden representations.
- Language Layer Pruning: Layer importance is computed as one minus average input-output cosine similarity across representative VLA samples and token positions.Low importance scores indicate layers with minimal transformative effects.
- Language Layer Pruning: Layers are ranked by importance and the lowest-scoring n layers are removed using a non-contiguous pruning strategy.Importance scores are computed for every layer before selecting layers for removal.
- Visual Token Selection: Visual pruning addresses both task-irrelevant tokens and tokens that duplicate information because of inherent visual similarity.Task relevance is estimated from cross-attention to contextual embeddings and normalized into scores si ∈[0, 1].
- Visual Token Selection: The visual selection first retains Kkey highly task-relevant tokens, then augments them to reach Kfinal using task relevance and feature diversity.The augmentation quota is Kaug = Kfinal −Kkey, with Ktask = ⌊α·Kaug⌋ relevance-driven tokens and Kdiv = Kaug −Ktask diversity-driven tokens.
- Visual Token Selection: The final visual set Vpruned is the union of key, task-driven, and diversity-driven tokens and is used for downstream VLA processing.This reduces visual sequence length while retaining task-specific and diverse visual information.
3.4 Caching Intermediate Features in Action Prediction
EfficientVLA addresses temporal redundancy in diffusion-based action generation by caching intermediate attention and MLP features across denoising timesteps. Periodic recomputation and reuse bypasses expensive operations while exposing a tunable acceleration–fidelity trade-off.
- Feature Generation and Temporal Coherence in DiT Blocks: Strong temporal coherence in intermediate action-generation features motivates caching attention and MLP outputs across denoising timesteps.The diffusion action head repeatedly computes self-attention and MLP operations over T timesteps.
- Feature Generation and Temporal Coherence in DiT Blocks: At each denoising timestep, DiT blocks sequentially process input features through self-attention and an MLP to produce intermediate hidden states.The input may combine upstream VLM cognitive features with the current noise estimate.
- Static N-Step Caching Implementation: A cache interval N determines when intermediate features are recomputed and cache entries are updated during the denoising trajectory.The cache is initialized at t = Tstart and refreshed at timesteps satisfying t (mod N) = 0.
- Static N-Step Caching Implementation: At non-recomputation timesteps, EfficientVLA retrieves cached features and bypasses the computationally intensive self-attention and MLP operations.This avoids recomputing both core modules between cache updates.
- Static N-Step Caching Implementation: The static schedule prunes execution for N −1 out of every N post-initialization timesteps, reducing action-generation FLOPs and latency.Larger cache intervals may accelerate inference more but can slightly reduce action fidelity if representations change rapidly.
4 Experiment
Experiments evaluate EfficientVLA on CogACT using SIMPLER, showing that coordinated layer, token, and diffusion-head optimizations outperform isolated acceleration strategies while preserving task performance.
- Experimental Setup: SIMPLER evaluates CogACT, which combines DINOv2 and SigLIP vision encoders, a Llama2-7B language module, and a Diffusion Transformer action module.
- Main Results: 71.1% FLOPs reduction and 1.93× inference speedup were achieved with a 22-layer, 56-token configuration and only a 0.6% drop in average success rate.
- Efficiency Analysis: 1.38× speedup and 19.9% FLOPs reduction for VLA-Cache contrast with EfficientVLA’s broader gains from addressing both computational and memory bottlenecks.
- Scalability Evaluation: 2.0× inference speedup on CogACT-Large accompanied a decrease in performance from 76.7% to 76.1%, indicating effectiveness across model scales.
- Component Sensitivity: Visual-token reduction to 22% substantially lowers computation, but further reduction yields marginal speed gains, while longer cache intervals accelerate action generation.
- Ablation Study: 1.93× speedup with all components exceeded the 1.43× speedup from model compression alone and the 1.23× speedup from visual-token optimization alone.
5 Conclusion
EfficientVLA is a training-free framework that reduces VLA inference costs through coordinated compression and redundancy removal, retaining near-original SIMPLER performance.
- EfficientVLA combines language-layer pruning, diverse task-relevant visual-token selection, and caching of intermediate action-module computations.
- 1.93× inference speedup and FLOPs reduced to 28.9% were achieved on CogACT in SIMPLER with only a 0.6% accuracy degradation.
A.1 SIMPLER Environment
SIMPLER is a simulation benchmark designed to approximate real-world robotic setups while testing robustness under both matched and varied visual conditions.
- SIMPLER evaluates VLA models in a simulation environment designed to mirror real-world robotic setups and reduce the real-to-sim control and visual gap.
- Evaluation Settings: Visual Matching minimizes simulation-to-reality discrepancies by prioritizing fidelity to real-world appearances.
- Evaluation Settings: Variant Aggregation introduces changes in backgrounds, lighting, distractors, and table textures to test generalization across diverse conditions.
- Tasks: The Google robot evaluation uses four tasks: Pick coke can, Move near, Open/close drawer, and Open top drawer and place apple.
A.2 Baselines
The experiments compare EfficientVLA with CogACT and training-free methods that accelerate VLA inference through visual-token or temporal-feature reuse.
- Backbone: CogACT is the primary validation platform, combining DINOv2 and SigLIP encoders, a Llama2-7B language module, and a DiT action module.
- Training-Free Baselines: VLA-Cache selects visually static and task-relevant tokens across sequential steps, reusing results through KV-cache with layer-adaptive reuse ratios.
- Training-Free Baselines: FastV dynamically prunes less impactful visual tokens after a selected layer using attention scores to reduce FLOPs without sacrificing performance.
B Impact Statement
EfficientVLA addresses VLA computational and memory demands through a training-free framework that combines language-layer pruning, visual-token optimization, and diffusion-feature caching. The framework is intended to support practical deployment on resource-constrained robotic platforms.
- EfficientVLA is a training-free framework for reducing computational and memory demands in VLA models.
- The framework prunes redundant language layers, selects visual tokens for task relevance and diversity, and caches intermediate diffusion-head features.
- EfficientVLA is intended to enable practical deployment of powerful VLA models on resource-constrained robotic platforms.
C Limitations
The paper identifies limitations involving the achievable compression and speedup, cache-interval trade-offs, and the narrow validation scope caused by limited open-source diffusion-based VLA models.
- Training-free acceleration may not achieve the maximal compression or speedup attainable by training-aware methods.
- The fixed cache interval N introduces a trade-off between acceleration and action fidelity.
- Current demonstrations are primarily on CogACT because open-source diffusion-based VLA models are limited.
- Future work is needed to validate scalability and effectiveness across a wider range of models and tasks.