PaperlayerView exact revision on arXiv

Source-linked AI summary

ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation

Hongyu Yan, Qiwei Li, Jiaolong Yang, Yadong Mu

arXiv:2603.27670v1cs.ROcs.AI

TL;DR

Existing VLA policies often lack explicit progress awareness, making long-horizon manipulation dependent on brittle termination heuristics. ProgressVLA combines a pretrained progress estimator with an action-conditioned latent world model and differentiable diffusion guidance, and reports improved benchmark and real-robot performance over baselines.

  • Problem

    Existing VLA and generative policies commonly lack explicit task-progress signals and rely on brittle termination heuristics for long-horizon manipulation.

  • Method

    ProgressVLA predicts progress from language and visual observations, projects candidate actions into future latent visual states, and backpropagates progress gradients to guide diffusion action generation.

  • Results

    ProgressVLA improves performance across CALVIN, LIBERO, and real-robot evaluations, including 76% success with 0.81 m/53.3 steps under classifier guidance in real-robot tasks.

  • Takeaways & Limitations

    Explicit progress guidance supports more goal-directed action generation, improved execution efficiency, and more reliable task termination across simulated and real settings.

Abstract

from arXiv · show

Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving cascaded sub-goals. In this work, we investigate the estimation and integration of task progress, proposing a novel model named {\textbf \vla}. Our technical contributions are twofold: (1) \emph{robust progress estimation}: We pre-train a progress estimator on large-scale, unsupervised video-text robotic datasets. This estimator achieves a low prediction residual (0.07 on a scale of $[0, 1]$) in simulation and demonstrates zero-shot generalization to unseen real-world samples, and (2) \emph{differentiable progress guidance}: We introduce an inverse dynamics world model that maps predicted action tokens into future latent visual states. These latents are then processed by the progress estimator; by applying a maximal progress regularization, we establish a differentiable pipeline that provides progress-piloted guidance to refine action tokens. Extensive experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates and generalization over strong baselines.

I. ABSTRACT

ProgressVLA addresses the lack of explicit progress monitoring in vision-language-action policies by estimating task completion and using that signal to guide diffusion-based action generation. The framework combines latent world-model prediction, progress-guided sampling, and empirical evaluation across simulated and real robotic manipulation settings.

  • Progress-guided sampling is presented as improving progress alignment and reducing completion steps on CALVIN relative to a standard diffusion policy.
  • Existing VLA and generative policies often lack explicit task-progress signals, leaving long-horizon execution dependent on brittle termination heuristics.
  • ProgressVLA estimates normalized task completion from language instructions and visual observations, using progress as a dense signal for task evaluation.
  • The framework predicts future latent visual states from candidate latent actions and backpropagates progress gradients through the world model to guide diffusion sampling.
  • The paper reports empirical validation on CALVIN, LIBERO, and real-robot deployments, with gains in long-horizon success and task termination reliability.
  • Normalized timestep labels provide progress supervision without additional annotations because curated expert demonstrations are assumed to advance approximately monotonically toward completion.

B. World Model

The world model learns compact latent representations that connect action dynamics with future visual states. These representations support progress prediction and joint fine-tuning so generated actions can be evaluated for task advancement.

  • The world model uses an encoder as an inverse dynamics model and a decoder as a forward dynamics model for latent-action representation and future-state prediction.
  • Its training objective combines latent-dynamics reconstruction with a KL-divergence term that regularizes the latent action distribution.
  • The learned latents are intended to decouple visual nuisances from task-relevant features and provide a shared state representation for action generation and progress estimation.
  • During joint fine-tuning, predicted future latent states are assessed by the progress estimator to compute task-advancement scores for candidate action chunks.
  • The joint objective combines world-model, progress, and joint losses to make latent dynamics informative for downstream progress estimation and guidance.

D. Progress-Guided Diffusion Policy

ProgressVLA generates embodiment-agnostic latent action chunks, predicts their future visual outcomes, and uses differentiable progress guidance to refine diffusion sampling. Online experience further aligns progress estimates with completion and improves robustness to execution noise.

  • Two-stage generation: The two-stage policy first generates latent action chunks for high-level strategy, then decodes them into executable low-level robot actions.The latent space is intended to support cross-embodiment flexibility.
  • Progress-conditioned prediction: The world model predicts future images from current observations and latent action samples, which the progress estimator scores.This creates a differentiable path from candidate actions to predicted task progress.
  • Differentiable guidance: Progress gradients backpropagate through the world model to guide latent-action diffusion sampling toward higher predicted progress.The guidance modifies the reverse diffusion update through a strength parameter s.
  • Inference behavior: Progress-guided sampling shifts generated actions toward higher predicted progress, reducing extensive resampling and enabling threshold-based runtime termination.These effects are reported empirically for the guided sampler.
  • Online finetuning: Online rollouts provide edge cases such as recovery behaviors, near-failure states, and out-of-distribution visual perturbations for subsequent finetuning.The online buffer is used to finetune both the progress estimator and diffusion policy.
  • Progress estimator finetuning: Successful episodes are expected to have approximately monotonic progress, so violations are mined as anomalies for a margin-based monotonicity loss.The progress estimator is finetuned by minimizing Lprog + Lmono on the online buffer.
  • Policy finetuning: The policy’s task-aware score is the progress predicted after applying a latent action through the world model, and optimization increases that score while remaining KL-constrained.This formulation treats progress maximization as KL-regularized policy improvement.

A. Pretraining Data

The section describes OXE-based pretraining, progress-estimator architecture, world-model regularization, action-generation modules, CALVIN evaluation, and ablations isolating pretraining and classifier guidance. Pretraining improves task completion, especially for longer-horizon sequences, even without guidance.

  • Pretraining setup: All components are pretrained on Open X-Embodiment datasets using specified mixture, filtering, preprocessing, batch-size, hardware, and learning-rate protocols.Training uses batch size 2048 on 8 NVIDIA H20 GPUs with a base learning rate of 1 × 10^-4.
  • Progress estimator: The progress estimator combines DINOv2 start/current patch features with projected language and visual tokens using role embeddings and cross-attention.The architecture encodes start-to-current visual changes while aligning instructions with observations.
  • World model: The UniVLA world model predicts future visual features from latent actions and uses KL regularization to normalize their distribution for latent-action prediction.The regularization improves compatibility with the Latent Action Expert.
  • Action modules: The Latent Action Expert and Action Decoder use DiTA-style causal Transformers for latent-action prediction and decoding.The expert autoregressively predicts latent actions from multimodal context.
  • CALVIN benchmark: CALVIN evaluates language-conditioned long-horizon manipulation across an ABC→D split using sequences of five subtasks and reports consecutive completion lengths from 1 to 5.It also reports the average number of tasks completed per episode.
  • Ablations: The ablations compare unguided, pretrained, classifier-guided, pretrained-guided, and reinforcement-learning-finetuned ProgressVLA variants.These variants isolate the effects of diffusion pretraining, classifier guidance, pretrained evaluators, and RL finetuning.
  • Pretraining result: Pretraining the diffusion policy yields large and consistent task-completion improvements, particularly on longer-horizon sequences, even without guidance.The reported interpretation is that pretraining supplies a strong latent-action prior and reduces compounding errors.

3) Classifier guidance relies on a reliable evaluator:

Classifier guidance improves diffusion-policy performance across simulated and real-robot evaluations, with larger gains when the evaluator is pretrained. The evaluation spans CALVIN, LIBERO, and five real-robot manipulation tasks.

  • CALVIN: Pretrained classifier guidance raises CALVIN 5-in-a-row success from 52.8% to 56.4% and 4-in-a-row success from 60.8% to 63.6%.Average completed length also reaches 3.68.
  • LIBERO: Progress guidance improves LIBERO average success from 81.5 to 83.3, while the full model reaches 84.5.On LIBERO-LONG, the full method achieves 66.2 versus 53.7 for OpenVLA.

4) Evaluation protocol and baselines.:

Evaluation compares guided and unguided ProgressVLA variants against established baselines across real-robot scenarios and progress-estimation settings. The results show stronger progress prediction under scene shifts and more goal-directed guided trajectories.

  • Evaluation protocol: The real-robot trials use 20 evaluations per task and compare Octo with unguided and guided ProgressVLA variants.The progress estimator is evaluated independently using controlled original, lighting-shift, and novel-object scenes.
  • Real-robot protocol: ProgressVLA with classifier guidance reaches 76% average real-robot success, compared with 66% without guidance and 23% for Octo.It also reduces average distance to 0.81 m and steps to 53.3.
  • Progress-estimator evaluation: Pretraining improves progress estimation under original, lighting-shift, and novel-object scenes after finetuning on small real-robot datasets.In the original scene, Pearson correlation improves from 0.912 to 0.977 and stop reliability from 53.8 to 82.1.
  • Progress-estimator evaluation: Under lighting shift, the pretrained and finetuned estimator achieves Pearson 0.953 and stop reliability 80.8, while the from-scratch model reaches 0.809 and 3.6.The reported comparison uses Pearson correlation and stop reliability.
  • Guidance visualization: Figure 7 compares trajectories and progress curves with and without guidance, showing more plausible, goal-directed paths for ProgressVLA.The visualization includes examples from simulation and realistic scenarios.

APPENDIX A IMPLEMENTATION DETAILS

The implementation combines a cross-attention progress estimator with a latent-space inverse-dynamics world model and a two-stage diffusion policy. The evaluator supplies differentiable, noise-aware guidance during action generation.

  • Progress estimator: The progress estimator maps language, start observations, and current observations to a normalized scalar progress score in [0, 1].It uses frozen DINOv2 visual features, CLIP text features, cross-attention, and an MLP prediction head.
  • World model: The world model encodes latent actions from current and future visual features and decodes predicted future features in latent space.Operating on DINOv2 features rather than pixels is intended to improve robustness to appearance changes.
  • Progress guidance: Candidate latent-action chunks are scored by applying the progress estimator to world-model-imagined futures, producing a differentiable task-aware evaluator.The evaluator provides classifier-guidance gradients with respect to the noisy latent action.
  • Noise-aware guidance: A noise-conditioned evaluator uses timestep embeddings during guidance to stabilize gradients across diffusion noise levels without changing the base world model.The world model and progress estimator are jointly finetuned under the same noise conditioning before distillation.

2) Action Decoder:

The action decoder converts latent action plans into executable chunks through a coupled diffusion process. Guidance affects the latent plan and thereby the decoded actions, improving progress and reducing unnecessary motion in the illustrated rollout.

  • Action decoder: The action decoder generates executable action chunks conditioned on observations, instructions, and the noisy latent-action variable.The latent chunk represents a visual-space plan that is translated into embodiment-specific low-level actions.
  • Coupled diffusion: Inference couples a 1000-step latent-action diffusion with a 100-step action-chunk diffusion.The latent process uses a longer schedule for coherent planning, while the action process uses a shorter schedule to reduce inference cost.
  • Coupled diffusion: The latent-action process is first warm-started to the action diffusion noise scale, after which both latent and executable action variables are jointly updated.This lets latent improvements influence action updates within the same diffusion trajectory.
  • Action generation: The final denoised action chunk is executed after the coupled denoising loop, with classifier guidance providing the progress-directed update.CG denotes classifier guidance in the update procedure.
  • Rollout comparison: With guidance, the orange-object rollout successfully grasps and lifts the object while the measured progress segment increases monotonically; without guidance, grasping fails and progress oscillates.The comparison is shown for the same instruction under the two inference settings.

C. Progress Estimator generalization

Pretraining and finetuning improve progress estimation under lighting and object shifts, while the evaluator also supports KL-regularized progress maximization for action selection.

  • Evaluation settings: The evaluated settings compare pretrained+finetuned and from-scratch estimators under lighting changes and novel objects using expert-collected rollouts.
  • Appearance-shift generalization: Pretrained+finetuned estimation remains smooth and near-monotonic under lighting shifts, reaching high progress in fewer steps than training from scratch.
  • Appearance-shift generalization: Under novel objects, pretrained+finetuned estimation approaches high progress near completion, whereas from-scratch estimation fluctuates and is less separable.
  • Progress-guided selection: The task-aware score represents predicted progress after applying a latent action through the world model.
  • Progress-guided selection: Progress maximization is formulated as a KL-regularized policy improvement problem.
  • Progress-guided selection: The KL-constrained solution introduces a Lagrange multiplier and yields an optimum whose log-density differs by an additive energy term.

C. Instantiating π0 as a VP (Variance-Preserving) diffusion policy over latent actions

The latent-action policy uses a variance-preserving diffusion formulation with an epsilon predictor, then incorporates evaluator-based guidance during denoising and distills the guided target into the denoiser.

  • VP diffusion parameterization: The latent action is identified with x_0, while x_τ denotes the denoising variable at diffusion step τ.
  • VP diffusion parameterization: The diffusion policy is parameterized by an epsilon predictor ε_θ(x_τ, τ, s), whose score approximation is exact when it matches the conditional noise mean.
  • Guided sampling: A noise-aware evaluator Q_τ(s, x_τ) is used during sampling to construct a guided denoising score.
  • Guided sampling: The guided direction is converted into a noise target and distilled into the denoiser through the standard denoising objective.
Loading 2603.27670v1…