PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaVLA: Adaptive Step Flow Matching for Training-free Acceleration of Vision-Language-Action Models

Sunghwan Han, Youngtae Han, Youngmin Yi

arXiv:2608.29208v1cs.ROcs.LG

TL;DR

VLAs face edge-deployment barriers from high computation, training-data requirements, and iterative flow-matching inference that existing accelerators do not fully address. AdaVLA provides training-free online adaptation using trajectory curvature to adjust inference steps and MLP pruning, achieving 1.87× to 2.24× speedups with negligible success-rate degradation on LIBERO and 1.65× speedup in real-world experiments.

  • Problem

    VLA acceleration is limited by high computational cost, restricted training-data access, and methods that largely overlook iterative flow-matching ODE inference.

  • Method

    AdaVLA uses flow-matching trajectory curvature to adapt inference steps and MLP pruning ratios online without training data.

  • Results

    1.87× to 2.24× speedups were achieved for π0.5 and X-VLA on LIBERO with negligible success-rate degradation, alongside 1.65× speedup in real-world experiments.

  • Takeaways & Limitations

    AdaVLA supports practical deployment of flow-matching VLAs on resource-constrained robotic systems without fine-tuning or calibration data.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models, built upon Vision-Language Models (VLMs), have significantly enhanced robotic capabilities by leveraging internet-scale knowledge and multimodal reasoning. However, the intensive computational overhead of VLAs constrains on-device deployment, hindering real-time responses to environmental changes. While various acceleration techniques have been proposed, they often rely on fine-tuning or access to training datasets, which are frequently unavailable due to privacy and proprietary concerns. Moreover, although flow-matching-based VLAs have emerged as efficient alternatives to standard diffusion models, current acceleration efforts largely target VLM inference costs, failing to address the iterative ODE solving process inherent in flow matching inference. To address these limitations, we propose AdaVLA, an online, training-free adaptive framework for fast yet accurate flow-matching-based Vision-Language-Action models. We introduce a novel metric derived from the flow matching trajectory curvature to quantify action generation confidence during inference. This metric enables the dynamic reduction of inference steps and the adaptive adjustment of MLP pruning ratios through an efficiently computed importance evaluation, requiring no access to training data. Experimental results on the LIBERO benchmark using a Jetson AGX Orin device demonstrate that our method achieves $1.87\times$ and $2.24\times$ speedups for $π_{0.5}$ and X-VLA, respectively, with negligible degradation in success rates. Furthermore, we validate the robustness of our approach on real-world robotic tasks using SmolVLA.

I. INTRODUCTION

VLAs improve multimodal robotic action planning but remain difficult to deploy on edge devices because of computational cost, limited training-data access, and iterative flow-matching inference. AdaVLA addresses these constraints with online, training-free adaptation across the VLM and Action Expert.

  • VLAs extend VLMs from textual responses to direct robotic control actions, improving robustness and generalization across diverse environments.
  • Existing acceleration methods commonly require training data for fine-tuning or calibration, which may be restricted by proprietary concerns or impractical to process on edge devices.
  • Flow-matching VLAs shift a larger share of latency to iterative Action Expert inference, making VLM-only optimization insufficient.
  • AdaVLA provides training-free online adaptation that reduces runtime computation in both the VLM and Action Expert.
  • 1.87× to 2.24× latency reductions were achieved for π0.5 and X-VLA on LIBERO using Jetson AGX Orin, with minimal success-rate impact.

II. RELATED WORK

VLA research has progressed from discrete and autoregressive policies toward continuous generative action modeling, while acceleration methods target computational cost through architectural changes, early exiting, pruning, and layer skipping. However, many such methods depend on retraining or calibration data.

  • Continuous generative approaches, including diffusion and flow-matching architectures, extend earlier discrete and autoregressive VLA policies for complex action distributions.
  • Acceleration methods include state-space models, early exiting, parallel decoding, token or layer pruning, and dynamic layer skipping.
  • Many existing efficiency methods rely on retraining or calibration for layer-importance assessment, limiting use in data-limited and privacy-sensitive settings.
  • VLA models map multimodal observations, language instructions, and robot state to action chunks for general-purpose robot learning.
  • The VLM encodes multimodal observations once, then shares its KV cache with the Action Expert through cross-attention during action generation.

B. Action Expert & Flow Matching

The Action Expert uses conditional flow matching to transform noise into actions along a learned vector-field trajectory. At inference, an Euler ODE solver iteratively follows this trajectory toward the data state.

  • Conditional flow matching learns a vector field along an interpolation path from Gaussian noise to the clean action.
  • The model learns a vector field approximating the target field, whose τ-independent formulation induces a linear target trajectory.
  • At inference, the model solves an ODE from the noise state at τ = 0 to the data state at τ = 1, conditioned on the observation.

C. Latency Analysis

Latency analysis identifies iterative flow-matching ODE solving as the main edge-device bottleneck, while VLM MLP blocks remain an important pruning target. AdaVLA combines curvature-based adaptive inference with training-free, importance-aware pruning to trade latency against success rate.

  • C. Latency Analysis: The VLM backbone contributes approximately 40% (386.53 ms) of latency, with MLP blocks substantially slower per layer than self-attention blocks.
  • C. Latency Analysis: 58% (557.78 ms) of π0.5 inference latency comes from iterative flow-matching ODE solving across 10 iterations.
  • C. Latency Analysis: AdaVLA reduces redundant Action Expert solver steps and modulates VLM pruning ratios using training-free Participation Ratio-based block importance.
  • A. Adaptive Inference via Trajectory Curvature: Trajectory curvature serves as a confidence proxy, enabling direct jumps for nearly linear trajectories and bounded progression when curvature is high.
  • A. Adaptive Inference via Trajectory Curvature: The framework uses midpoint-vector updates and adaptive pruning, recalculating importance and reordering channels after significant context shifts.
  • A. Adaptive Inference via Trajectory Curvature: The curvature threshold Cth consolidates the latency–success rate trade-off into one tunable design parameter.

B. Training-free & Efficient MLP Importance Assessment

AdaVLA estimates MLP layer importance without training data or costly SVD, using activation-representation diversity to guide layer-specific pruning. Highly expressive blocks are preserved while redundant blocks are pruned more aggressively.

  • Training-free importance assessment: AdaVLA evaluates layer importance using forward passes only, without access to training data, using an Effective Rank-inspired metric for MLP activations.The metric treats higher-rank representations as more diverse and expressive, while lower-rank representations indicate redundancy.
  • SVD-free assessment: Participation Ratio provides a computationally tractable alternative to exact singular-value computation on edge devices.It uses the Gram matrix G = A^T A, whose eigenvalues correspond to squared singular values of the activation matrix.
  • Layer-adaptive pruning: δ_l assigns each layer a normalized importance score, enabling layer-specific rather than uniform pruning decisions.Uniform pruning is suboptimal because layers differ in parameter sensitivity and information density.
  • Pruning policy: Highly important MLP blocks are pruned conservatively, whereas less important blocks are pruned aggressively to preserve representational diversity and optimize the latency–success rate trade-off.This policy links representation rank to the amount of capacity retained in each layer.

C. MLP Channel Reordering

AdaVLA reorders MLP intermediate channels online using an importance score that combines activation magnitude with the following projection layer’s weight norm. Reordering is delayed until initialization or a detected context shift to limit overhead.

  • C. MLP Channel Reordering: AdaVLA uses structured pruning because MLP intermediate channels create computational bottlenecks and deployment-time training data may be unavailable.The method dynamically reorders channels rather than relying on offline calibration.
  • C. MLP Channel Reordering: AdaVLA defines channel importance as the product of activation magnitude and the corresponding projection weight norm.This accounts for the subsequent projection layer’s scaling effect, unlike activation-magnitude-only importance.
  • C. MLP Channel Reordering: Channels are sorted by importance and only the top fraction is retained to preserve critical features during pruning.The resulting ordering is intended to reduce performance degradation while pruning intermediate channels.
  • C. MLP Channel Reordering: Channel reordering occurs during the initial forward pass or after the maximum allowed iterations, which signals a significant context shift.This scheduling limits reordering overhead while adapting to changing contexts.

A. Experimental Setup

The experiments evaluate AdaVLA on two flow-matching VLAs across LIBERO’s task suites, using curvature thresholds and latency and success-rate measurements designed for edge deployment. Success rates are primarily measured on an NVIDIA A10, with edge-device comparability checked on Jetson AGX Orin.

  • Models: AdaVLA is evaluated on π0.5 and X-VLA, using a default step size of s = 0.1 corresponding to 10 steps.These are representative state-of-the-art flow-matching-based VLA models.
  • Simulation Benchmarks: LIBERO evaluation covers the Spatial, Object, Goal, and Long task suites to test diverse semantic and spatial reasoning capabilities.The curvature thresholds are 0.15 for π0.5 and 0.002 for X-VLA, selected by grid search.
  • Hardware & Metrics: Success rate is measured over 50 rollouts per task, while latency is measured for generating a single action on the deployment hardware.Average success rates are measured on an NVIDIA A10 to avoid lengthy full-benchmark evaluation on the Jetson AGX Orin.
  • Hardware & Metrics: Success rates on the Jetson AGX Orin are additionally checked on a subset of tasks against the A10 results.The study reports that the edge-device success rates were comparable to those obtained on the server-grade GPU.

B. Main Results on LIBERO Benchmark

On LIBERO with Jetson AGX Orin, AdaVLA reduces latency for π0.5 and X-VLA while preserving high success rates, with adaptive inference and pruning contributing distinct benefits.

  • Main benchmark results: 2.24× latency reduction accelerates X-VLA from 387.52 ms to 173.09 ms, with 3.06× lower energy consumption and near-perfect success rates.The corresponding energy reduction is from 10.73 J to 3.51 J.
  • Main benchmark results: AdaVLA maintains stable per-step timing on Jetson AGX Orin, with standard deviations of 19.12 ms for π0.5 and 21.06 ms for X-VLA.These measurements support the timing stability reported for deployment.
  • Ablation study: Adaptive inference is the primary latency-control component, while PR-based pruning preserves robustness in the π0.5 ablation.Removing adaptive inference raises latency to 925.71 ms, whereas removing pruning lowers success to the 98.65% baseline.
  • Ablation study: Increasing Cth from 0.15 to 0.25 reduces latency to 413.26 ms while success rate declines marginally to 98.40%.Cth provides a deployment control knob for selecting a latency–success rate operating point.
  • Ablation study: Static one-step reduction reaches 447.45 ms latency but lowers success rate to 97.05%, whereas five-step and random-step variants increase latency to 677.56 ms and 694.13 ms.AdaVLA instead concentrates computation on critical trajectory segments.

D. Real-World Robot Experiments

AdaVLA was evaluated on a SO-ARM101 robot with SmolVLA running on a Jetson AGX Orin, reducing real-world action-generation latency while improving overall average success rate. The experiments also identify trajectory coarseness as a localized consequence of larger integration steps.

  • Experimental Setup: Each of four tasks was evaluated over 30 independent trials using a uniform curvature threshold of 0.075.The tasks involved placing a banana, socks, sausage, and cup in specified locations.
  • Quantitative Results: 1.65× speedup reduced average latency from 806.73 ms to 488.49 ms in real-world SO-ARM101 experiments.The evaluation used SmolVLA on a Jetson AGX Orin, with success rate reported as successes over trials.
  • Quantitative Results: Overall average success rate increased from 87.50% to 90.00%, despite marginal degradation in Tasks 3 and 4.The reported degradation was associated with coarser action trajectories caused by larger integration steps.
  • Analysis: Action smoothing was identified as a lightweight post-processing option for mitigating coarser trajectories at negligible computational cost.The reported trajectory issue was linked to larger integration steps rather than a broad loss of operational reliability.

E. Analysis of Adaptive Inference

AdaVLA uses trajectory analysis to reduce redundant flow-matching computation while retaining extra inference steps for difficult or changing task moments. The visualized behavior shows additional computation concentrated around grasping, placing, and unexpected environmental changes.

  • Adaptive Inference Behavior: AdaVLA generated clean actions with significantly fewer forward passes than the baseline while adding steps only when necessary.The analysis covered 250 randomly sampled LIBERO-Long trajectories for π0.5 and X-VLA.
  • Critical Task Segments: Additional inference steps were concentrated during critical moments such as grasping and placing objects.These segments were among those where the model reached the maximum step limit.
  • Critical Task Segments: AdaVLA also invoked additional steps when the environment changed unexpectedly, such as when an object fell during pickup.This illustrates adaptive allocation of computational effort to changing task conditions.
  • Framework Summary: The framework is training-free and adjusts inference steps and MLP pruning ratios using flow-matching trajectory curvature.The conclusion reports 1.87× to 2.24× speedups on LIBERO with negligible success-rate degradation and 1.65× speedup in real-world experiments.
Loading 2608.29208v1…