PaperlayerView exact revision on arXiv

Source-linked AI summary

FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference

Zekai Li, Jiaming Tang, Zhijian Liu

arXiv:2608.27384v1cs.RO

TL;DR

Flow-matching VLAs face high action-decoding latency and unstable asynchronous execution because chunks are decoded independently. FlashVLA jointly decodes buffered chunks with staggered noise levels and chunk-wise causal attention, reducing latency while preserving continuity. Across simulated and real-world tasks, it maintains or improves task success, reaches up to 2.43× end-to-end speedup, and sustains ≥30 Hz smooth control on one GPU.

  • Problem

    Flow-matching VLA deployment is limited by iterative action-decoding latency and asynchronous execution from stale observations and temporal mismatch.

  • Method

    FlashVLA uses a streaming buffer of staggered-noise action chunks with chunk-wise causal attention to decode chunks jointly and emit one executable chunk per inference step.

  • Results

    Across LIBERO, RoboTwin 2.0, and real-world tasks, FlashVLA matches or improves π0.5 success, achieves up to 2.43× end-to-end per-action speedup, and sustains ≥30 Hz smooth control on one GPU.

  • Takeaways & Limitations

    Joint streaming decoding makes asynchronous continuity structural while reducing action-decoding latency without an explicit future-state predictor.

  • Takeaways & Limitations

    FlashVLA inherits the pretrained model’s independent-chunk training objective and incurs an N−1-step cold start that is more noticeable on very short tasks.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models are increasingly promising for robotic manipulation, yet their real-world deployment remains bottlenecked by high inference latency and unstable asynchronous execution. This challenge is particularly pronounced in flow-matching-based VLA models, where action decoding requires multiple iterative steps conditioned on the VLM context. While efficient inference methods improve control frequency and asynchronous methods reduce execution idle time, existing approaches often fail to jointly achieve low-latency inference and accurate, temporally consistent asynchronous execution. We introduce \textbf{FlashVLA}, a streaming action decoding framework that addresses both challenges in a unified formulation. FlashVLA maintains a streaming action buffer with multiple chunks at different noise levels and decodes them using chunk-wise causal attention. This design allows FlashVLA to produce one executable action chunk per inference step. Moreover, its chunk-wise autoregressive formulation implicitly preserves action continuity, enabling smooth asynchronous execution without extra future-state conditioning. Across extensive simulated and real-world experiments, FlashVLA substantially improves inference speed while maintaining strong task performance. It can achieve $\geq$30\,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.

1 Introduction

VLA deployment is limited by action-decoding latency and asynchronous temporal mismatch. FlashVLA jointly decodes buffered action chunks to reduce latency and preserve continuity, improving speed while maintaining task performance.

  • Challenge: Action decoding consumes 75% of π0.5 per-step inference time because each chunk requires ten sequential denoising steps.The profile was measured on an RTX 4090 with two views and no system-level optimization.
  • Challenge: Synchronous inference stalls the robot at chunk boundaries, whereas asynchronous inference overlaps execution with prediction from stale observations.Longer lookahead increases the temporal mismatch between predicted actions and the robot state that executes them.
  • FlashVLA: FlashVLA maintains action chunks at staggered noise levels and advances them jointly with chunk-wise causal attention.This replaces isolated chunk decoding with a streaming buffer formulation.
  • FlashVLA: After warm-up, each forward pass emits one executable chunk while amortizing every chunk’s N denoising steps across time.Per-step action-decoding latency drops by up to 20×.
  • Results: Across LIBERO, RoboTwin 2.0, and real-world Franka tasks, FlashVLA matches or improves π0.5 success and sustains ≥30 Hz smooth control on one GPU.It also achieves up to a 2.43× end-to-end per-action speedup under one-step asynchronous delay.

2 Related Work

Prior VLA acceleration either makes individual passes cheaper or overlaps execution while leaving temporal mismatch unresolved. FlashVLA transfers streaming chunk-wise diffusion to action decoding, making asynchronous continuity structural while reducing decoding cost.

  • Efficient VLA inference: Efficient-inference methods reduce the cost of each forward pass, while few-step distillation shortens iterative decoding at the cost of retraining and a sample-quality tradeoff.The surveyed methods include lightweight architectures, compression, token pruning, faster action heads, and diffusion or flow-matching distillation.
  • Efficient VLA inference: The ten-to-one pass reduction yields a 9.3× reduction in GPU work and a measured 19.9× wall-clock speedup after eliminating kernel-launch serialization.The wall-clock result includes system-level effects beyond the reduction in GPU work.
  • Asynchronous VLA execution: Asynchronous VLA methods overlap prediction with execution but patch stale-observation mismatch using future-state prediction, planned actions, or training augmentation.These corrections incur auxiliary supervision or architectural changes, and their cost grows with lookahead.
  • Streaming chunk-wise diffusion: Streaming chunk-wise diffusion jointly denoises buffered chunks at staggered noise levels with causal masking, emitting one clean chunk per pass.FlashVLA applies this paradigm to robot actions using chunk-level causal masking and a control-frequency-matched buffer and noise schedule.
  • Streaming chunk-wise diffusion: FlashVLA’s future action chunks attend to lower-noise near-execution chunks, implicitly conditioning them on the trajectory the robot is following.This recovers asynchronous continuity without explicit future-state prediction.

3 Method

FlashVLA jointly decodes action chunks in a staggered-noise buffer, using chunk-wise causal attention to amortize denoising and condition future chunks on the evolving trajectory. Its streaming queue emits one executable chunk per step after warm-up, while packed fine-tuning covers cold-start and steady-state buffer configurations.

  • Chunk-wise autoregressive formulation: FlashVLA is a drop-in framework for flow-matching VLAs that decodes action chunks jointly rather than in isolation.It requires light architectural changes and a fine-tuning pass.
  • Chunk-wise autoregressive formulation: A flow-matching VLA iteratively denoises each action chunk from a Gaussian sample while conditioning on the current observation.The original inference process uses multiple sequential action-expert passes and no inter-chunk attention.
  • Chunk-wise autoregressive formulation: FlashVLA maintains N chunks at staggered noise levels, advancing every chunk by one denoising step in each forward pass.The cleanest chunk is positioned first for execution, while the noisiest chunk is positioned last for later execution.
  • Streaming inference: The monotone noise staircase aligns denoising progress with execution order, allowing one executable chunk to emerge per step instead of in batches.Without staggering, chunks would emerge in bulk and reintroduce chunk-boundary stalls.
  • Chunk-wise autoregressive formulation: Chunk-wise causal attention lets noisier future chunks attend to cleaner earlier chunks, preserving temporal information flow without bidirectional interference.This provides implicit conditioning on near-execution chunks without an explicit future-state predictor or auxiliary action-conditioning module.
  • Streaming inference: 20×: per-step action-decoding latency drops by up to 20× in the reported profile through amortized denoising.Each chunk still traverses all N noise levels, but those passes are distributed across N forward steps.
  • Streaming inference: FlashVLA uses N−1 warm-up steps before execution, then advances, executes, shifts, and replenishes the buffer at every steady-state step.During cold start, the robot executes a safe default while the buffer is populated.
  • Multi-buffer joint fine-tuning: Multi-buffer joint fine-tuning packs all N cold-start configurations for one observation into one sample while isolating buffers with attention masks.The shared observation encoding is computed once, and the standard flow-matching loss covers valid buffer prefixes and both operating phases.

4 Simulated Experiments

Simulated experiments evaluate FlashVLA’s speed, asynchronous robustness, synchronous quality, long-horizon behavior, and transfer across VLA architectures. FlashVLA jointly improves latency and task performance while preserving quality under asynchronous delay.

  • 4.2 Asynchronous Execution: FlashVLA reaches 90.6% success on RoboTwin 2.0 at one-step delay while reducing time per step from 47.4 to 43.8 ms.On LIBERO, it raises average success from 96.9% to 97.8% and reduces time per step from 53.8 to 22.1 ms, a 2.43× speedup.
  • 4.2 Asynchronous Execution: Across delays d=1–4, FlashVLA maintains 97.5–98.3% LIBERO success and a 2.43–2.62× speedup while remaining above synchronous performance.On RoboTwin 2.0 at d=4, success is 89.8% versus 86.0% for synchronous π0.5, with a 1.08–1.10× speedup.
  • 4.3 Decoding Latency and Reaction Speed: 20.3 ms per policy invocation on RTX 5090 with two views corresponds to a sustained ∼50 Hz policy update rate.FlashVLA is fastest across RTX 4090 and RTX 5090 configurations with two- and three-camera inputs, outperforming optimized π0.5 and Realtime-VLA.
  • 4.3 Decoding Latency and Reaction Speed: FlashVLA reduces TTFA to 37.1 ms and expected TTR to 70.4 ms, achieving 1.7× lower TTFA and 1.6× lower TTR than FASTER.The comparison uses steady-state latency after buffer filling; cold-start overhead is amortized in episode completion time.
  • 4.4 Synchronous Quality and Long-Horizon Gain: FlashVLA improves synchronous success from 96.9% to 97.9% on LIBERO and from 86.1% to 90.8% clean and 85.8% to 90.2% randomized on RoboTwin 2.0.Matched training budgets and execution horizons isolate the effect of joint chunk decoding without prediction-execution overlap.
  • 4.4 Synchronous Quality and Long-Horizon Gain: Long-horizon RoboTwin 2.0 gains reach 36.6 points, compared with 4.3 points on medium-horizon tasks, while short-horizon performance is preserved.The reported trend is associated with chunk-level memory from causal attention over earlier streaming-buffer chunks.
  • 4.5 Cross-Architecture Generalization: FlashVLA transfers across architectures, delivering 1.95× and 2.81× latency speedups on SmolVLA and LingBot-VLA while preserving or improving task quality.LingBot-VLA success improves by 3.4 points synchronously and 4.1 points asynchronously.

5 Real-World Deployment

Real-world evaluation on a 7-DoF Franka arm tests whether FlashVLA’s simulated latency and asynchronous-continuity benefits transfer to 30 Hz robotic control. Across short-, medium-, and long-horizon tasks, FlashVLA achieves the strongest reported task scores and faster completion.

  • Real-World Evaluation: FlashVLA outperforms three π0.5 configurations on every real-world task in both task score and completion time.The evaluated tasks span pick-and-place, whiteboard wiping, and table cleaning across short, medium, and long horizons.
  • Real-World Results: 84.4% average task score exceeds 80.0% for synchronous π0.5 and 75.6% for naive asynchronous execution.The comparison aggregates successful trials across the three Franka tasks.
  • Real-World Results: 1.3× average completion-time speedup over synchronous π0.5 and 1.2× over RTC is reported across successful trials.Completion time excludes early termination on failed trials.

6 Conclusion

FlashVLA unifies streaming action decoding and asynchronous execution through a staggered multi-buffer design with chunk-wise causal attention. It improves latency and preserves asynchronous continuity, while retaining a warm-up cost and the pretrained model’s independent-chunk objective.

  • FlashVLA uses staggered action chunks with chunk-wise causal attention to jointly address decoding latency and asynchronous mismatch.The design is presented as a drop-in streaming decoder for flow-matching VLAs.
  • Up to 20× lower per-step action-decoding latency and ≥30 Hz smooth control are reported across simulated and real-world tasks.
  • FlashVLA recovers asynchronous continuity without an explicit future-state predictor.
  • The streaming buffer requires N−1 warm-up steps per episode, with the cost more noticeable on very short tasks.The warm-up cost amortizes quickly over multi-second rollouts.
  • Multi-buffer joint fine-tuning inherits the pretrained model’s independent-chunk training objective.Pretraining from scratch with the chunk-wise causal formulation is identified as future work.

A.1 Simulated Experimental Settings

The simulated experiments fine-tune VLA models on LIBERO, RoboTwin 2.0, and cross-architecture settings using specified datasets, optimization schedules, and FlashVLA buffer configurations.

  • LIBERO fine-tuning uses 50K steps on 8 H200 GPUs, with FlashVLA configured at chunk size 10 and buffer length 4.The learning rate is 10−4 with cosine scheduling and 1,000 warm-up steps.
  • RoboTwin 2.0 models train on the union of clean and randomized data from all 50 tasks for multitask and horizon-grouped evaluations.Both π0.5 and FlashVLA use a learning rate of 5 × 10−5 with cosine decay.
  • Cross-architecture evaluation fine-tunes SmolVLA on all four LIBERO suites and LingBot-VLA on RoboTwin 2.0.The supplied settings use chunk size 10 and buffer length 5 for SmolVLA, and chunk size 20 and buffer length 4 for LingBot-VLA.

A.2 Real-World Experiment Settings

Real-world experiments fine-tune on human-collected task data and evaluate deployment at 30 Hz with standardized execution and asynchronous-delay settings.

  • Real-world task models are fine-tuned on human-collected data using task-specific training durations and 8 H200 GPUs.Pick-and-place uses 10K steps, while whiteboard wiping and table cleaning use 30K steps.
  • FlashVLA uses chunk size 20 and buffer length 4, while π0.5 uses chunk size 50.
  • Deployment runs on an RTX A4000 at 30 Hz, with execution horizon 16 and asynchronous delay of two steps for all asynchronous methods.Each task score averages 15 rollouts, and completion time averages successful rollouts only.
  • Figure 6 ablates chunk-wise causal attention by removing the causal mask while retaining the same streaming buffer.The figure reports the resulting asynchronous success-rate change.

A.3 Ablations and Additional Results

Ablations show that chunk-wise causality is necessary for asynchronous continuity, while chunk and buffer sizes shape performance and latency in distinct ways.

  • Chunk-wise causal attention: Removing the chunk-wise causal mask lowers asynchronous success rate by roughly 10 points on average, despite retaining the streaming buffer.The causal attention from cleaner to noisier chunks is identified as the source of asynchronous continuity.
  • Chunk size: On LIBERO, chunk size C=10 is the sweet spot across synchronous, one-step asynchronous, and two-step asynchronous execution.Larger chunks degrade success rate, while apparent per-action-time improvements can reflect more reset steps from lower success.
  • Chunk size: With N=5, one-step asynchronous execution outperforms synchronous execution even at chunk sizes C=15 and C=20.This indicates the streaming advantage persists outside the sweet spot.
  • Buffer length: For buffer lengths N∈{4,5,6} with C=10, all settings match or exceed the π0.5 baseline at one-step delay and achieve 1.8× average speedup.Per-step latency remains nearly unchanged across this range, making N=4 a stable default.

A.4 Implementation Details

FlashVLA adapts pretrained VLA models through multi-buffer joint fine-tuning and performs streaming inference in cold-start and steady-streaming stages. Its buffer stores action chunks at different flow-matching timesteps, enabling one chunk to be output per inference step.

  • Model architecture: FiLM generates scale, shift, and gate parameters from multi-level timestep embeddings to strengthen timestep conditioning in the action expert.The paper reports substantially better performance from strengthening timestep conditioning; the implementation is natural for models with FiLM-style conditioning.
  • Multi-buffer joint fine-tuning: Multi-buffer joint fine-tuning adapts pretrained VLA models to FlashVLA’s streaming action decoding paradigm.For each observation, training retrieves future action chunks and constructs buffer states across different flow-matching timesteps.
  • Multi-buffer joint fine-tuning: Each buffer state contains j real action chunks followed by N−j padding chunks, for j = 1, . . . , N.The buffer length N is the number of different flow-matching timesteps, while C denotes the chunk size.
  • Streaming inference: During cold-start, the buffer is initialized with padding and noise, repeatedly updated without executing predictions, and paired with a cold-start action.After N−1 updates, steady streaming begins by popping and executing one predicted chunk per inference step, then appending fresh noise.
  • Multi-buffer joint fine-tuning: All N buffer states share observation o_t during training, while attention masks prevent cross-buffer interaction.This multi-buffer fine-tuning strategy is reported as crucial for efficient adaptation to streaming action decoding.
  • Streaming inference: Streaming inference has two stages: cold-start and steady-streaming.Algorithm 1 specifies separate initialization and execution procedures for these stages.
Loading 2608.27384v1…