PaperlayerView exact revision on arXiv

Source-linked AI summary

DriftingVLA: Native One-Step Vision-Language-Action Generation via Per-Dimension Temporal Drifting

Yuxuan Gao, Shiqi Zhang, Yedong Shen, Yifan Duan, Wenhao Yu, Xin Zhang, Siyuan Cao, Jiajun Deng, Yanyong Zhang

arXiv:2608.29749v1cs.RO

TL;DR

Flow-based VLAs require iterative action refinement, increasing online control latency. DriftingVLA learns direct one-step action generation with Per-Dimension Temporal Drifting, achieving strong control performance while reducing action-chunk latency.

  • Problem

    Flow-based VLAs rely on inference-time iterative refinement for each action chunk, increasing latency in online robot control.

  • Method

    DriftingVLA preserves pretrained multimodal representations, relearns the action interface as a direct conditional generator, and applies PDTD over complete temporal trajectories of individual action channels while generating chunks jointly.

  • Results

    DriftingVLA outperforms the original 10-NFE π0.5 policy and representative one-step baselines across LIBERO, RoboTwin 2.0, and real-world tasks, while reducing latency from 227.61 ms to 67.67 ms for a 3.36× speedup.

  • Takeaways & Limitations

    Native one-step VLA generation can remove iterative action refinement from deployment without sacrificing control performance.

  • Takeaways & Limitations

    The reported evaluation covers LIBERO, RoboTwin 2.0, and six real-world single- and dual-arm manipulation tasks.

Abstract

from arXiv · show

Conventional flow-based vision-language-action (VLA) models support expressive continuous action generation but rely on multi-step refinement to produce each action chunk, increasing latency in online robot control. To address this issue, we introduce DriftingVLA, a native one-step VLA that generates a complete action chunk with a single action-expert forward pass. Rather than learning a flow field that requires iterative integration at inference, DriftingVLA uses a distribution-drifting objective to learn a direct noise-to-action-chunk mapping for one-step deployment. Since robot action dimensions carry distinct control semantics and distributional characteristics, we further introduce Per-Dimension Temporal Drifting (PDTD). PDTD treats the complete temporal trajectory of each action dimension as a separate drifting unit, enabling finer-grained modeling and shaping of dimension-specific action distributions. This per-dimension decomposition applies only to the training objective; the shared VLA model still generates the complete action chunk jointly, thereby preserving cross-dimensional dependencies. DriftingVLA achieves 98.32% success on LIBERO, 81.09% on RoboTwin 2.0, and 77.67% across six real-world single- and dual-arm tasks, outperforming the evaluated multi-step flow policy and one-step VLA baselines. Native one-step deployment also delivers a 3.36-fold speedup in action-chunk generation, eliminating iterative refinement without sacrificing control performance.

I. INTRODUCTION

DriftingVLA addresses the latency of iterative flow-based VLA inference by learning direct one-step action generation and introducing PDTD for heterogeneous action dimensions. It outperforms evaluated baselines across simulation and real-world settings while reducing action-generation latency.

  • Motivation: Flow-based VLAs model expressive continuous actions but require repeated action-expert evaluations during inference, increasing online control latency.The motivation is to remove iterative refinement from deployment rather than merely reduce its cost.
  • Contributions: PDTD organizes each action channel’s complete temporal trajectory as a separate drifting unit, avoiding shared geometry across heterogeneous channels while retaining temporal completeness.The decomposition applies to the drifting objective, not to the jointly generated action chunk.
  • Contributions: DriftingVLA preserves the pretrained multimodal backbone, freshly initializes the action branch, and jointly adapts both components for direct one-step generation.The action expert learns a direct conditional generator instead of inheriting a flow-velocity parameterization.
  • Evaluation: 98.32% overall success on LIBERO, 81.09% on RoboTwin 2.0, and 77.67% real-world success demonstrate strong performance across simulated and real-world tasks.The real-world evaluation spans six single- and dual-arm manipulation tasks.
  • Evaluation: 3.36× speedup reduces action-chunk latency from 227.61 ms to 67.67 ms under controlled A800 profiling.The reported efficiency improvement accompanies the removal of iterative action refinement at deployment.

A. Vision-Language-Action Models

VLA systems combine pretrained multimodal representations with continuous action-generation interfaces, including π0.5’s flow-matching action expert. These models generate actions by numerically integrating a learned vector field, motivating direct one-step alternatives.

  • VLA Models: VLA models combine visual, language, and proprioceptive context with continuous action generation for robotic manipulation.The action chunk contains future actions indexed by temporal horizon and per-step action dimension.
  • One-Step Alternatives: Recent alternatives accelerate or replace iterative inference through execution-level overlap, distillation, interval-averaged velocity prediction, or alternative one-step decoders.Examples include RTC, SnapFlow, MeanFlowVLA, and π0-EqM.
  • DriftingVLA: DriftingVLA retains pretrained multimodal representations while redesigning the continuous action generator for direct one-step control.It extends distribution drifting from robotic action policies to pretrained flow-based VLAs.
  • Flow-Based Action Generation: π0.5 uses a pretrained vision-language backbone and action expert that predicts a conditional flow-matching vector field from noisy action tokens.The target vector field is formed from Gaussian noise and demonstrated action chunks.
  • Flow-Based Action Generation: At inference, π0.5 starts from Gaussian noise and integrates the learned vector field with K numerical steps to produce an action chunk.The multimodal prefix can be cached, but the action expert is still evaluated K times.

B. Drift-Based Policy Learning

Drift-based policy learning moves distribution refinement from inference-time trajectories into training-time generator optimization. It uses sibling hypotheses and target demonstrations to learn a direct mapping that requires one action-generation evaluation at deployment.

  • Drifting Principle: Distribution drifting applies distribution-level corrections during optimization instead of repeatedly denoising or transporting samples at inference.After training, the generator directly maps a latent sample to an output.
  • Drifting Principle: The drifting field uses an anti-symmetric construction whose interactions attract generated samples toward target data and repel them from generated references.At distributional equilibrium, the field is zero when the two distributions match.
  • Training Objective: Training converts the drifting correction into a stop-gradient regression target, progressively absorbing distributional corrections into generator parameters.Inference therefore uses the learned direct mapping rather than explicitly applying the drifting update.
  • Policy Learning: DBP draws multiple independent latent samples during training to generate sibling action hypotheses and shapes them using demonstrated actions as positive references and generated hypotheses as negative references.The sibling interactions estimate the conditional drifting signal.
  • Action-Chunk Organization: Chunk-wise drifting preserves the full action chunk but mixes all channels, whereas Step-wise drifting separates temporal slices but fragments each channel’s trajectory.These groupings motivate organizing drifting along complete per-channel temporal trajectories.

IV. METHOD

DriftingVLA extends drift-based learning to pretrained flow-based VLAs by jointly adapting the multimodal backbone and a direct action generator. Its training pipeline uses sibling action chunks, reorganizes them with PDTD, and constructs detached attraction–repulsion targets.

  • Direct Generation: DriftingVLA reconfigures the π0.5 action branch as a direct noise-to-action generator while jointly post-training it with the pretrained VLM.This addresses flow-specific action-interface adaptation without discarding pretrained multimodal representations.
  • Training Pipeline: The pretrained multimodal backbone encodes each condition once, and its shared prefix is reused across G sibling generations from independent Gaussian latents.A freshly initialized one-step action expert generates the sibling chunks in parallel.
  • PDTD: PDTD reorganizes sibling action chunks into complete temporal trajectories for each action dimension before constructing drifting targets.The shared VLA still generates the complete action chunk jointly, preserving cross-dimensional dependencies.
  • Training Objective: Detached attraction–repulsion targets jointly post-train the VLM backbone and action expert.The training objective uses generated siblings and demonstrations to shape the direct generator.

A. From Flow-Based VLA Models to Direct Action Generation in DriftingVLA

DriftingVLA reconfigures a pretrained flow-based VLA into a direct noise-to-action generator for native one-step deployment. It preserves the pretrained multimodal backbone while jointly retraining the action interface and using shared-context sibling generation during distribution-drifting training.

  • Direct action generation: DriftingVLA generates a complete H × D action chunk directly from an action-shaped latent and multimodal context.The output is interpreted as the final action chunk rather than a local update along a generative trajectory.
  • Direct action generation: One-step behavior is learned directly, unlike evaluating a conventionally trained flow policy with a single Euler step.DriftingVLA changes the learned function itself rather than only changing the inference-time numerical solver.
  • Pretrained VLA representation: The model retains the pretrained π0.5 multimodal backbone while freshly initializing and jointly optimizing the continuous action-generation branch.This preserves pretrained semantic and perceptual representations without constraining the new generator to the previous velocity-field parameterization.
  • Shared-context sibling generation: Distribution drifting trains the generator using multiple sibling samples produced under the same multimodal condition.The context encoder is shared across siblings, while independently sampled action latents produce sibling-specific action chunks.
  • Drifting geometry: The action-chunk grouping determines which coordinates share distances, scale statistics, affinities, and normalized drifting forces.This grouping is a design choice in the drifting objective rather than a change to the shared VLA generator.

B. From Action-Chunk Geometry to Per-Dimension Temporal Drifting

PDTD reorganizes drifting around each action channel’s full temporal trajectory. It separates heterogeneous channel geometry while retaining joint generation of the complete action chunk and its cross-dimensional dependencies.

  • Action-chunk structure: An action chunk A ∈ R^H×D contains distinct temporal and action-dimension axes, with each column representing one channel’s temporal evolution.Temporal slices mix heterogeneous control variables, whereas channel columns preserve complete trajectories.
  • Limitations of existing groupings: Chunk-wise drifting preserves the full horizon but forces all heterogeneous channels to share one distance, scale, and neighborhood geometry.This creates cross-channel geometric interference in the drifting signal, distinct from desirable policy-level coupling.
  • Limitations of existing groupings: Step-wise drifting separates timesteps but fragments each channel’s full trajectory and still mixes all heterogeneous channels within each timestep.The objective no longer directly compares samples using a channel’s complete temporal evolution.
  • Per-Dimension Temporal Drifting: PDTD treats the full temporal trajectory of each action dimension as one drifting unit, yielding D groups of dimension H.This preserves temporal completeness while defining geometry separately for each action channel.
  • Factorized geometry, joint action generation: PDTD factorizes drifting geometry only after the complete action chunk has been generated jointly by the shared VLA model.All per-dimension losses update the same generator, preserving cross-dimensional dependencies for coordinated control.

C. Per-Dimension Drifting Objective

The PDTD objective constructs channel-specific drifting fields from generated sibling trajectories and demonstrations. It normalizes distances per channel, combines multi-scale attraction and repulsion, and regresses the generator toward frozen targets.

  • Reference construction: For each condition and action channel, PDTD forms a reference set containing generated sibling trajectories and the demonstrated trajectory.The channel-wise trajectories provide the samples used to construct the distribution-drifting training signal.
  • Channel-specific normalization: PDTD estimates a separate distance ruler for every action channel before constructing its drifting geometry.The normalized distance reflects trajectory variation within that channel rather than the aggregate scale of the full action tensor.
  • Multi-scale drifting field: Multi-scale interactions combine local and broader neighborhood information while per-channel normalization prevents raw magnitude from dominating the drifting field.The construction applies channel-specific geometry across bandwidths ρ.
  • Attraction and repulsion: The objective balances attraction toward the demonstrated reference with repulsion among generated siblings using signed affinity coefficients.The construction is translation invariant and balances coefficient mass across the attraction–repulsion interaction.
  • Regression objective: PDTD minimizes mean squared regression error against a frozen drifting target recomputed from the current generator’s samples.Repeated optimization absorbs distribution-level corrections into the generator parameters rather than evaluating the drifting field at deployment.

D. Training and Native One-Step Inference

Training applies PDTD to valid action coordinates and jointly adapts the pretrained VLM and action-generation branch. Inference removes the distribution-level machinery and performs one Gaussian-latent evaluation.

  • Padding-aware action space: PDTD excludes padded action dimensions and future timesteps from distance statistics, affinities, and drifting forces.Only valid physical action dimensions and temporal coordinates contribute to the training geometry.
  • Post-training: During post-training, each condition produces G sibling chunks, which PDTD reorganizes into channel trajectories and uses to update the policy.The shared multimodal context supports joint adaptation of the pretrained VLM and freshly initialized action-generation branch.
  • Native one-step inference: At deployment, DriftingVLA draws one Gaussian latent chunk and evaluates the direct generator once.No sibling set, pairwise interaction, drifting field, or numerical integration is required.
  • Native one-step inference: DriftingVLA moves iterative refinement from robot-time inference to optimization-time distribution learning.The deployed policy retains joint action-generation capacity while reducing execution to a single direct noise-to-action evaluation.

V. EXPERIMENTS

The experiments assess native one-step generation across simulation, real-world control, geometry, adaptation, and deployment efficiency.

  • The evaluation tests native one-step action generation against iterative flow-based control in simulation.
  • The evaluation also measures transfer to real-world single- and dual-arm manipulation.
  • Controlled studies examine PDTD geometry and pretrained-VLA adaptation choices.
  • Deployment experiments assess whether one-step generation reduces latency without excessive training overhead.

A. Experimental Setup

Experiments cover LIBERO, RoboTwin 2.0, and six real-world manipulation tasks, using repeated evaluations across training seeds and comparing iterative, truncated, and native one-step policies.

  • LIBERO contains four suites—Spatial, Object, Goal, and Long—with 10 tasks per suite.Each trained policy is evaluated for 100 episodes per task.
  • RoboTwin 2.0 evaluates all 50 tasks under Easy and domain-randomized Hard settings.Each task and difficulty level receives 100 evaluation episodes.
  • Simulation models use three training seeds and report mean success rate with sample standard deviation.Training uses 50k optimization steps unless otherwise stated.
  • Real-world evaluation spans two single-arm and four dual-arm tasks on a platform with two UR5 manipulators and three cameras.The tasks range from object placement and alignment to cleanup, pouring, and stacking.
  • Comparisons include 10-NFE π0.5, one-step π0.5-1step solver truncation, SnapFlow, and other one-step VLA approaches.π0.5-1step is used as a diagnostic control rather than a natively trained one-step baseline.

B. Simulation Benchmark Results

DriftingVLA matches or exceeds evaluated baselines across LIBERO, RoboTwin 2.0, and real-world tasks, while PDTD consistently outperforms alternative drifting geometries.

  • Simulation benchmark results: 98.32% overall success on LIBERO exceeds 10-NFE π0.5 by 1.22 points and SnapFlow by 0.84 points.On LIBERO-Long, DriftingVLA reaches 94.33%, compared with 92.40% for π0.5 and 93.07% for SnapFlow.
  • Simulation benchmark results: 81.09% overall success on RoboTwin 2.0 exceeds 10-NFE π0.5 by 1.51 points and SnapFlow by 2.56 points.On the Hard split, DriftingVLA obtains 77.56%, while π0.5-1step reaches 53.29%.
  • Real-world evaluation: 77.67% overall real-world success improves over 10-NFE π0.5’s 74.22% by 3.45 points.DriftingVLA achieves the best mean success rate on five of six tasks; π0.5 is slightly higher on Tabletop Cleanup.
  • Real-world evaluation: 66.11% for DriftingVLA-Step and 67.78% for DriftingVLA-Chunk leave PDTD ahead by 11.56 and 9.89 points, respectively, in physical evaluation.The Step < Chunk < PDTD ordering persists from simulation ablations to physical manipulation.
  • Real-world evaluation: DriftingVLA improves average success on both single-arm and dual-arm subsets.Single-arm success rises from 86.00% to 89.67%, while dual-arm success rises from 68.34% to 71.67%.
  • Ablation studies: PDTD improves over Chunk-wise drifting by 3.04 points on LIBERO and 6.92 points on RoboTwin 2.0.It exceeds Step-wise drifting by 5.22 and 10.66 points, respectively.

E. Computational Efficiency

DriftingVLA removes repeated action-expert evaluations from deployment through native one-step generation, substantially reducing action-chunk latency while retaining control performance with moderate training overhead.

  • Inference latency: 3.36× speedup reduces mean action-chunk latency from 227.61 ms to 67.67 ms by replacing 10-NFE iterative inference with one generator evaluation.The measured reduction is 70.3% under the reported LIBERO-Spatial profiling setup.
  • Training cost: Increasing sibling samples from G = 2 to G = 8 raises training cost by 3.78% and peak memory by 9.54%.Training cost increases from 87.92 to 91.24 GPU-hours, while peak memory grows from 45.99 to 50.38 GiB.
  • Training cost: At G = 8, DriftingVLA requires 91.24 GPU-hours, below the corresponding π0.5 reference of 108.49 GPU-hours.The comparison uses the controlled profiling setup described for the training-cost experiment.
  • Summary: The experiments support native one-step generation as preserving or improving control performance while PDTD outperforms DBP geometries and training overhead remains moderate.The conclusion also reports that native one-step generation removes iterative action refinement without sacrificing control performance.
  • Limitations: The evaluation focuses on manipulation tasks and uses fixed per-dimension temporal grouping, leaving adaptive geometries and broader robotic settings for future work.The authors specifically identify larger multi-embodiment VLAs, heterogeneous action spaces, and broader long-horizon interactive settings as future directions.
Loading 2608.29749v1…