RAVEL: Asynchronous Rolling Inference for Flow-Based Vision-Language-Action Models

Motivation

• Sequential VLM encoding and multi-step action generation create high inference latency, while fixed action chunks restrict how frequently new observations influence control.Under synchronous inference, robots may pause between chunks while waiting for inference.Source: §1 · p. 2

Approach

• RAVEL carries partially denoised future actions in a rolling buffer, so each near-term action needs only one final denoising update before execution.Each control step advances the buffer once, releases its first action, and appends a new noisy action.Source: §1 · p. 2
• RAVEL decouples VLM encoding from action generation, uses the latest available context with temporal-offset conditioning, and adds FOP for current visual and proprioceptive features.These contributions are presented as enabling continuous action generation while accounting for delayed context and current observations.Source: §1 · p. 2

Evaluation

• RAVEL maintains comparable task performance on LIBERO and RoboTwin 2.0, improves success over evaluated baselines on Dynamic Object Manipulation, and reduces observation-to-command response times.Real-world table-tennis return and tennis-ball grasping experiments also demonstrate rapid responses and smooth motion.Source: §1 · p. 2

Flow-based VLA and action chunking

• Action chunking in flow-based VLAs still requires VLM encoding and multi-step denoising before the first action becomes available.The paper identifies this as a latency bottleneck in flow-based VLAs.Source: §2 · pp. 2–3

Real-time and asynchronous action generation

• Existing approaches address denoising latency or feedback freshness, while RAVEL combines rolling action generation with decoupled VLM encoding and FOP.The cited approaches include reduced denoising computation and methods that decouple vision-language processing from fast control or correct actions using current observations.Source: §2 · p. 3

3.1 OFFSET-AWARE ROLLING FLOW DENOISING

• One action-expert pass advances every buffer slot by 1/K flow time; the first action is released and a new noisy action appended, so each action is refined over K updates.Later-executed actions begin at higher noise levels, and temporal-offset conditioning accounts for the gap between observation and execution.Source: §3.1 · p. 3

3.2 ASYNCHRONOUS CONTEXT UPDATES AND FAST OBSERVATION PATHWAY

• The action expert reuses the latest available VLM context with its temporal offset, while FOP supplies current visual and proprioceptive information for action refinement.The Figure 2 overview shows asynchronous context production alongside rolling buffer updates, with FOP conditioning on the current observation.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 STREAMING-ALIGNED TRAINING

• Streaming-aligned training varies the observation-to-action offset and conditions the action expert on it, using a training-buffer noise profile matched to inference.The action expert is trained to predict slot-wise flow-matching velocities from the buffer, VLM context, and temporal offset; FOP features are also supplied when enabled.Source: §3.3 · pp. 4–5

4.1 SYNCHRONOUS SIMULATION

• Across LIBERO and RoboTwin 2.0, RAVEL maintains comparable or slightly improved baseline performance; on one RTX 4090, first-action latency is 2.59–2.92× faster.The synchronous trials aggregate 2,000 LIBERO and 5,000 RoboTwin 2.0 trials.Source: Table 1 · p. 5, Figure 3 · p. 5, §4.1 · pp. 5–6

4.2 ASYNCHRONOUS SIMULATION

• On DOM, RAVEL achieves 44.06% overall success, 8.56 percentage points above CI & LAAS; FOP raises it to 46.94%, while mean completion time is 6.04 s, 23.0% below DynamicVLA over common-success episodes.Source: §4.2 · p. 6, Table 2 · p. 6

4.3 REAL-WORLD EVALUATION

• Table-tennis return success is 90% versus 25% for FASTER and VLASH, with observation-age P50 121.76 ms and P95 132.20 ms; grasping success is comparable, with lowest EHF and TCP jerk RMS.Lower EHF and jerk RMS indicate fewer high-frequency oscillations and smoother acceleration changes.Source: §4.3 · pp. 7–8, Table 3 · p. 8

4.4 ANALYSIS AND ABLATION STUDIES

• With rolling inference fixed, shifted-horizon training reaches 78.00% LIBERO and 31.54% RoboTwin 2.0 success; adding temporal-offset conditioning raises these to 97.85% and 46.82%.Noise-profile matching alone is insufficient to recover performance from the pretrained checkpoint under rolling inference.Source: §4.4 · p. 8
• 95.00–97.25% success is maintained across 0–15 delayed observation steps on LIBERO, while the baseline falls from 98.25% to 92.50%.These synchronous X-VLA rollouts test robustness to temporal misalignment from inference latency.Source: §4.4 · p. 8
• 90% table-tennis success peaks at δ0 = 0.3; smaller bias undercompensates execution delay, while δ0 = 0.7 reaches 40% as predictions shift too far forward.The sweep uses the same policy checkpoint.Source: Figure 5 · p. 9, §4.4 · p. 9
• 44.89% success with the actual VLM-computation offset falls to 16.67% when the offset resets to zero, across 450 paired episodes.The comparison holds the checkpoint and non-blocking runtime fixed.Source: §4.4 · pp. 8–9

5 LIMITATIONS AND FUTURE WORK

• RAVEL currently requires streaming-aligned training and architecture-specific action-expert modifications for temporal-offset conditioning and FOP.The authors cite the lack of a unified action-expert design across existing VLA models.Source: §5 · p. 9
• The framework is formulated for flow-based VLAs, and action-generation latency cannot fall below one action-expert forward pass.Source: §5 · p. 9
• FOP provides current observations to the action expert, but subsequent physical-execution delay may reduce its real-world benefit.Source: §5 · p. 9
• Future work could pursue broader architecture compatibility, lower action-generation cost, and tighter integration with motion planning and execution feedback.Source: §5 · p. 9
• The authors report using generative AI for implementation, literature review, drafting, figures, and analysis, and say they reviewed the work and take responsibility for its content.Source: AI USE STATEMENT · p. 10

A.1 LIBERO AND ROBOTWIN 2.0

• The experiments use official X-VLA releases for LIBERO and RoboTwin 2.0 and π0.5 releases for LIBERO and RoboTwin 2.0, with all settings running PyTorch inference.The π0.5 RoboTwin checkpoint comes from Motus Robotics; the LIBERO weights are converted from OpenPI’s released JAX weights.Source: §A.1 · p. 13
• The four simulation configurations use K = 10 buffer slots, with each slot holding h actions for a horizon H = Kh; each rolling update releases one slot.The first target receives one denoising update, and later actions gain updates as they move through the buffer, reaching K updates at steady state.Source: §A.1 · p. 13
• Temporal offset δp = p/K encodes VLM-context age, using an MLP added to π0.5’s flow-timestep embedding and concatenated input features for X-VLA.All action slots share the same offset and retain their individual flow timesteps.Source: §A.1 · p. 13
• The released checkpoints are fine-tuned with the vision-language backbones frozen, while action modules and temporal-conditioning components are trained.π0.5 trains its action expert, projections, flow-timestep MLP, and added offset MLP; X-VLA trains its action Transformer and heads, including the offset input mapping.Source: §A.1 · p. 13
• Each synchronous policy query uses one VLM prefill and ten action-expert forward passes; the baseline uses ten full-horizon denoising steps while RAVEL uses ten rolling updates.Source: §A.1 · p. 13
• For π0.5 on RoboTwin 2.0, normalized absolute joint targets are stored in the buffer and converted relative to the latest state for each update.Predicted clean relative actions are converted back to absolute targets before storage, preserving intended targets as the robot state changes.Source: §A.1 · p. 13

A.2 DYNAMIC OBJECT MANIPULATION

• 370.7 ms was the median full-model inference latency locally on an RTX 4090, versus 226 ms reported on an RTX A6000; slower local inference likely contributes to the reproduced success-rate gap.Table 5 reports the local latency under the authors’ configuration; the hardware differs from the original paper’s.Source: §A.2 · pp. 13–14, Table 5 · p. 14
• RAVEL adds temporal-offset conditioning to DynamicVLA and uses a ConvNeXt-Base image encoder with projections, embeddings, and gated cross-attention adapters for FOP.The VLM is frozen while the action expert and temporal adapter are trained; FOP components are then trained with the policy and image encoder frozen.Source: §A.2 · p. 14
• DynamicVLA’s relative end-effector pose targets are converted across reference states using the coordinate conversion described for π0.5 on RoboTwin 2.0.Source: §A.2 · p. 14
• VLASH adaptation trains on shifted action chunks relative to a future-state anchor and selects the 20K-step checkpoint; its estimated handoff state may differ from the state reached at execution.The authors say this mismatch may contribute to poor adaptation results.Source: §A.2 · p. 14
• 28.56% success under time compensation came from direct full-chunk inference with the FASTER checkpoint, versus 23.78% with FASTER inference; abrupt output changes were also observed.The authors suggest fewer denoising steps for near-term relative actions may be less suited to prediction and lead to discontinuities.Source: §A.2 · pp. 14–15

B.1 TABLE-TENNIS RETURN

• Trials run at 30 Hz and count as successful when the robot hits the incoming ball and returns it beyond a predefined distance.Source: §B.1 · p. 15
• The setup uses a seven-degree-of-freedom xArm 7 with a paddle, a ball launcher, and three RGB camera views; Figure 6 depicts the setup.Policies use joint-state feedback to predict joint-position targets at a nominal 30 Hz control rate.Source: Figure 6 · p. 15, §B.1 · p. 15
• The policy training set contains 237 of 241 teleoperated trajectories, with four held out for validation; labels are future joint states shifted one timestep ahead.The base π0.5 policy is fine-tuned with rank-64 LoRA adapters and selected at 10,000 steps; several compared methods train a further 10K steps on the same data.Source: §B.1 · p. 15
• Table 6 summarizes training configurations for the table-tennis return experiment.Source: Table 6 · p. 15
• All configurations use AdamW, weight decay 10−10, and gradient clipping norm 1.0, with batch size reported globally.Parameter counts cover trainable modules used in action prediction, including action and temporal-conditioning projections.Source: §B.1 · p. 15
• Deployment uses method-specific execution horizons and delays: RAVEL uses K = 10 buffer slots, releases one action per update, and sets δ0 = 0.3.Sync executes a 10-action chunk before requesting another prediction, while Naive Async requests 10-action chunks whenever the inference server is idle.Source: §B.1 · pp. 15–16

• Each method was evaluated over 20 trials with a 30-second timeout; success required grasping the ball and placing it in the basket.Source: §B.2 · p. 16
• The policy used 66 demonstrations collected at 30 Hz, with 64 for training and two for validation.It used two camera views and predicted joint targets and gripper commands.Source: §B.2 · p. 16
• RAVEL started from the pretrained π0.5 10K-step checkpoint and trained for 5K additional steps, while Sync, Naive Async, and RTC used the checkpoint directly.Table 7 summarizes the training configurations; all configurations used AdamW, weight decay 10−10, and gradient clipping norm 1.0.Source: §B.2 · p. 16, Table 7 · p. 16

• Observation age measures elapsed time from receipt of an action’s source images to dispatch, using timestamps from the same host clock.Images are timestamped when the last of three camera images arrives; reused actions retain their original source-observation timestamp.Source: §B.3 · p. 16
• EHF is computed over non-overlapping 3-second windows of TCP positions resampled at 30 Hz, after linear detrending and summing spectral power at frequencies of 1 Hz and above.The median window score is used for each trajectory.Source: §B.3 · p. 16
• TCP jerk is estimated from resampled Cartesian velocity using a local cubic fit, and trajectory-level jerk and EHF scores include successful and failed attempts.Table 3 reports each method’s median trajectory-level scores.Source: §B.3 · p. 17
• Both smoothness metrics depend on motion speed and range, so their scores should be interpreted alongside task success.Source: §B.3 · p. 17

• Measured joint states visibly lag dispatched actions, showing that sending a target does not mean the arm has reached it.Observation age ends at action dispatch and excludes subsequent tracking delay.Source: §B.4 · p. 17
• The observed action–state delay motivates temporal-offset bias: targeting further along the predicted trajectory gives the arm additional tracking lead time.With K = 10 at 30 Hz, δ0 = 0.3 is three control steps, or 100 ms, of lookahead; this setting was selected empirically rather than measured as a fixed delay.Source: §B.4 · p. 17
• Figure 7 plots joints 2, 4, and 6 from four trials, with blue dispatched actions, dashed green controller targets, and orange measured states.The trials use δ0 = 0.3; the traces begin 2 seconds after the first command and share a vertical scale across trials.Source: §B.4 · p. 17, Figure 7 · p. 18