PaperlayerView exact revision on arXiv

Source-linked AI summary

ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation

Hongyu Yan, Qiwei Li, Jiaolong Yang, Yadong Mu

arXiv:2603.27670v1cs.ROcs.AI

TL;DR

Many generative VLA policies lack explicit dense task-progress signals, and long-horizon execution often relies on termination heuristics. ProgressVLA guides diffusion action generation with predicted progress, reporting benchmark gains and 76% average real-robot success versus Octo’s 23%.

  • Problem

    Many generative VLA policies lack explicit dense task-progress signals, while long-horizon execution often relies on termination heuristics.

    Source: §II · p. 1

  • Method

    ProgressVLA integrates a learned progress estimator and inverse-dynamics world model into diffusion sampling to guide action generation toward predicted progress.

    Source: §II · p. 2

  • Results

    ProgressVLA reports gains on CALVIN and LIBERO and achieves 76% average real-robot success versus Octo’s 23%.

    Source: §II · p. 2, 4) Evaluation protocol and baselines · p. 7

  • Takeaways & Limitations

    On evaluated real-robot tasks, classifier guidance improves reliability and execution efficiency.

    Source: 4) Evaluation protocol and baselines · p. 7

Abstract

from arXiv · show

Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving cascaded sub-goals. In this work, we investigate the estimation and integration of task progress, proposing a novel model named {\textbf \vla}. Our technical contributions are twofold: (1) \emph{robust progress estimation}: We pre-train a progress estimator on large-scale, unsupervised video-text robotic datasets. This estimator achieves a low prediction residual (0.07 on a scale of $[0, 1]$) in simulation and demonstrates zero-shot generalization to unseen real-world samples, and (2) \emph{differentiable progress guidance}: We introduce an inverse dynamics world model that maps predicted action tokens into future latent visual states. These latents are then processed by the progress estimator; by applying a maximal progress regularization, we establish a differentiable pipeline that provides progress-piloted guidance to refine action tokens. Extensive experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates and generalization over strong baselines.

II. INTRODUCTION … A. Progress Estimator

ProgressVLA integrates a language-conditioned progress estimator with an action-conditioned world model and diffusion policy to guide long-horizon manipulation toward task completion. Its progress signal is learned from normalized trajectory time and supports principled termination, with reported gains on CALVIN and LIBERO and real-robot deployments.

II. INTRODUCTION

  • ProgressVLA reports significant gains in long-horizon success rates and more reliable task termination across CALVIN, LIBERO, and real-robot deployments.The introduction also reports that progress-guided sampling improves progress alignment, reduces completion steps, and consistently increases success on CALVIN.Source: §II · pp. 1–2, Table I · p. 1

A. Vision-Language-Action Models

  • ProgressVLA addresses the lack of explicit task-progress monitoring in VLA and diffusion policies by integrating a learned progress estimator directly into diffusion sampling.Unlike passive conditioning or post-hoc trajectory evaluation, progress gradients steer generation toward task advancement and support threshold-based termination.Source: A. Vision-Language-Action Models · p. 2, §II · p. 2

B. World Model

  • The world model predicts latent future observations and task progress, providing dense differentiable guidance and threshold-based termination rather than sparse binary success supervision.Progress gradients are backpropagated through the model to guide diffusion sampling, reducing dependence on costly, sparse, or task-specific supervision.Source: B. World Model · pp. 2–3, Figure 2 · p. 3

IV. THE PROPOSED METHOD

  • ProgressVLA combines a normalized task-completion estimator, a bidirectional action-conditioned world model, and a diffusion policy guided by differentiable progress gradients.The world model can project future visual states from latent actions or infer latent actions from visual state transitions.Source: §IV · p. 3

A. Progress Estimator

  • The progress estimator evaluates task advancement from the language instruction, initial observation, and current image, outputting a normalized scalar score.The initial observation provides a global task anchor.Source: A. Progress Estimator · p. 3
  • Progress labels use normalized timestep p*=t/T as an annotation-free proxy because curated expert demonstrations advance steadily and approximately monotonically toward completion.This timestep-based target is presented as a robust and effective surrogate for task progress.Source: A. Progress Estimator · p. 3

B. World Model · C. Joint Finetuning of World Model and Progress Estimator

The method uses a compact latent world model to represent visual states and dynamics, then jointly fine-tunes it with a progress estimator so predicted outcomes reflect task advancement.

B. World Model

  • The world model’s encoder maps transitions between observations into a compressed latent-action space, while its decoder predicts future images from observations and latent actions.The encoder acts as an inverse dynamics model; the decoder acts as a forward dynamics model.Source: B. World Model · p. 3
  • Its training objective combines latent-dynamics reconstruction with KL-divergence regularization of the latent-action distribution.Source: B. World Model · p. 3
  • The model learns compact, transferable latents that separate visual nuisances from task-relevant features and provide a shared state representation for action generation and progress evaluation.Source: B. World Model · p. 4

C. Joint Finetuning of World Model and Progress Estimator

D. Progress-Guided Diffusion Policy · E. RL Finetuning with Progress

ProgressVLA guides latent-action diffusion toward higher predicted task progress, then uses online experience to refine both progress estimation and policy robustness. Its two-stage policy generates embodiment-agnostic latent action chunks and decodes them into executable actions.

D. Progress-Guided Diffusion Policy

  • A Latent Action Expert generates embodiment-agnostic action chunks for high-level strategy, and an Action Decoder maps them to low-level robot actions.This two-stage pipeline is designed for cross-embodiment flexibility.Source: D. Progress-Guided Diffusion Policy · p. 4
  • The world model predicts a future image from the observation and latent sample; the progress estimator scores it, and gradients guide diffusion updates toward increased progress.The predicted progress is differentiable with respect to the latent action through the world-model decoder; guidance strength controls the gradient contribution.Source: D. Progress-Guided Diffusion Policy · p. 4
  • Progress-guided sampling shifts generated actions toward higher predicted progress, reducing extensive resampling and enabling robust threshold-based runtime termination.The sampled latent-action chunk is decoded into an executable action sequence for deployment.Source: D. Progress-Guided Diffusion Policy · p. 4

E. RL Finetuning with Progress

  • Online rollouts finetune the progress estimator and diffusion policy for alignment with task completion and robustness to execution noise, capturing recovery, near-failure, and visual-perturbation cases.The online buffer includes edge cases underrepresented in static offline datasets.Source: E. RL Finetuning with Progress · p. 4
  • For successful episodes, estimator finetuning mines violations of expected progress monotonicity and applies a margin-based monotonicity loss on the online buffer.The estimator is finetuned by minimizing the progress loss together with the monotonicity loss.Source: E. RL Finetuning with Progress · pp. 4–5
  • KL-regularized policy improvement uses predicted post-action progress to adjust the denoiser’s noise target, favoring higher-progress actions while staying close to the current policy.The updated denoiser is trained with a standard denoising objective.Source: E. RL Finetuning with Progress · p. 5

V. EXPERIMENTS … 2) Pretraining contributes to diffusion policy performance:

The experiments report that OXE pretraining improves diffusion-policy task completion, particularly on longer CALVIN sequences, and describe the pretraining setup, model components, evaluation protocol, and compared variants.

A. Pretraining Data

  • All components are pretrained on Open X-Embodiment datasets using established dataset-mixture protocols, normalized and filtered actions, and shared image preprocessing unless specified otherwise.Training uses batch size 2048 across eight NVIDIA H20 GPUs and a base learning rate of 1 × 10−4.Source: A. Pretraining Data · p. 5

1) Progress estimator pretraining:

  • The progress estimator combines DINOv2 features from starting and current frames with instruction tokens to output a scalar progress score.Role embeddings distinguish start, current, and text tokens; cross-attention aligns instructions with observations before mean pooling, MLP fusion, and a sigmoid head.Source: 1) Progress estimator pretraining · p. 5

2) World model pretraining:

  • The world model adopts UniVLA [6] to predict future visual features from candidate latent actions and uses KL regularization to normalize latent-action distributions.The regularization is intended to stabilize latent-action prediction and improve compatibility with the Latent Action Expert.Source: 2) World model pretraining · p. 5

3) Latent action expert pretraining:

  • The Latent Action Expert uses a DiTA-style [16] causal Transformer to autoregressively predict latent actions, and the Action Decoder shares its architecture and training recipe.Additional hyperparameters are deferred to the supplemental material.Source: 3) Latent action expert pretraining · p. 5

C. CALVIN

  • CALVIN evaluation trains on environments A, B, and C and tests on D, measuring consecutive subtasks completed and average tasks completed per episode.Trials comprise five subtasks sampled from language-specified goals across up to 1,000 unique sequences.Source: C. CALVIN · p. 5

1) Baselines and our variants:

  • The compared variants isolate diffusion-policy pretraining, classifier guidance, pretrained evaluation components, and RL finetuning.They include from-scratch and OXE-pretrained policies without guidance; pretrained policies with guidance using a scratch-trained evaluator or pretrained world model and progress predictor; and the full pretrained-evaluator-plus-RL-finetuning variant.Source: 1) Baselines and our variants · p. 5

2) Pretraining contributes to diffusion policy performance:

  • Diffusion-policy pretraining yields large, consistent task-completion improvements, especially on longer-horizon CALVIN sequences, even without guidance.The comparison attributes this benefit to a strong latent-action prior and reduced compounding errors.Source: 2) Pretraining contributes to… · p. 5

3) Classifier guidance relies on a reliable evaluator: … 4) Evaluation protocol and baselines.:

Progress-guided classifier guidance improves benchmark and real-robot performance, with pretrained evaluation and RL finetuning strengthening guidance and robustness. The evaluation spans LIBERO and five real-world tasks, comparing against unguided ProgressVLA and Octo [41].

3) Classifier guidance relies on a reliable evaluator:

  • Pretraining the evaluator raises 5-in-a-row success from 52.8% to 56.4% and 4-in-a-row success from 60.8% to 63.6%, with average completed length reaching 3.68.The authors attribute the stronger gains to more reliable progress signals under distribution shift and execution noise, yielding higher-quality guidance gradients.Source: 3) Classifier guidance relies on a… · pp. 5–6

4) RL finetuning further improves robustness:

  • RL finetuning uses 1,000 trajectories from environments A/B/C; the full model achieves the best average completed length and improves 1–4 subtask success.Online experience improves evaluator progress–completion alignment and guidance, while refining the diffusion policy against execution noise.Source: 4) RL finetuning further improves… · p. 6

D. LIBERO

  • ProgressVLA achieves an 84.5 average on LIBERO and 66.2 on LIBERO-LONG versus OpenVLA’s 53.7; guidance raises average from 81.5 to 83.3.The full method achieves the best overall average, with progress guidance improving performance across all subsets.Source: D. LIBERO · p. 6, Table III · p. 6

1) Experiment setup.:

  • Real-world trials use an ARX AC-One dual-arm robot with two D405 RGB-D cameras, while RGB images serve as primary policy inputs.The tabletop setup uses fixed object initializations and consistent camera perspectives.Source: 1) Experiment setup · p. 6

2) Task setup.:

  • The five real-robot tasks range from single-step to long-horizon behaviors: Toy→Drawer, Pick Peach, Open Drawer, Orange→Plate, and Stack Bowls.Target objects and goal receptacles are marked with blue boxes in Fig. 5.Source: 2) Task setup · p. 7, Figure 5 · p. 7

3) Data collection and finetuning.:

4) Evaluation protocol and baselines.:

  • Quantitative real-robot evaluation runs 20 trials per task and compares full guided ProgressVLA with unguided ProgressVLA and pretrained VLA baseline Octo [41].Each reported step is one predicted and executed action chunk; lower distance and step counts indicate more efficient execution.Source: 4) Evaluation protocol and baselines · p. 7

E. Real-World Evaluation

  • 76% success with 0.81 m/53.3 steps, versus 66% with 0.96 m/100.8 steps unguided and Octo’s 23% with 1.30 m/187.9 steps, across real-robot tasks.Classifier guidance improves both reliability and execution efficiency; reductions are especially clear on tasks prone to redundant motion.Source: 4) Evaluation protocol and baselines · p. 7, Table IV · p. 7

F. Progress Estimator Generalization on Real-Robot Expert Trajectories … VI. CONCLUSION

ProgressVLA evaluates progress prediction on real-robot expert trajectories under original, lighting-shift, and novel-object conditions, finding that pretraining followed by real-data finetuning improves generalization. The paper also visualizes progress-guided actions and concludes that classifier guidance improves success and execution efficiency, with RL finetuning further improving robustness.

1) Offline, policy-agnostic evaluation protocol.:

  • The policy-agnostic evaluation uses real-robot expert demonstrations in original, lighting-shift, and novel-object scene settings.The lighting shift adds a desk lamp; the novel-object setting swaps in an unseen target while keeping layout and cameras fixed.Source: 1) Offline, policy-agnostic evaluation… · p. 7

2) Metrics:

  • Progress alignment measures Pearson correlation with a reference ramp, stop reliability measures late-trajectory predictions above 0.9, and progress error is ramp MAE.Higher alignment and stop reliability are better; lower progress error is better.Source: 2) Metrics · p. 7

3) From-scratch vs. pretrained and finetuned progress predictor:

  • From Scratch trains only on the target real-robot dataset, whereas Finetuned starts from a large-scale-pretrained progress predictor and finetunes on real demonstrations.The reported comparison evaluates both regimes on the same real-robot data.Source: 3) From-scratch vs. pretrained and… · pp. 7–8

G. Visualization

  • Fig. 7 contrasts baseline diffusion-policy trajectories with ProgressVLA trajectories and shows progress scores after equal numbers of action tokens.The visualization includes representative simulation and realistic scenarios, with progress-score curves across operations.Source: Figure 7 · p. 8, G. Visualization · p. 8

4) Results and discussion:

  • In original scenes, From Scratch→Finetuned improves Pearson 0.912→0.977, Stop 53.8→82.1, and MAE 0.14→0.10.Under lighting shift, From Scratch scores Pearson 0.809 and Stop 3.6, while Finetuned scores Pearson 0.953, Stop 80.8, and MAE 0.12. For novel objects, From Scratch→Finetuned yields Pearson 0.810→0.972, Stop 37.5→81.2, and MAE 0.15→0.11.Source: Table V · p. 8, 4) Results and discussion · p. 8

VI. CONCLUSION

  • The conclusion states that classifier guidance steers diffusion sampling with progress gradients, improving success rates and execution efficiency, while RL finetuning further improves evaluator and policy robustness.The progress estimator uses pretrained visual features and is described as robust across original, lighting-shift, and novel-object settings.Source: §VI · p. 8

APPENDIX A IMPLEMENTATION DETAILS … D. Latent Action Expert and Action Decoder

Appendix A specifies the progress estimator, latent-action world model, noise-conditioned evaluator, and two-stage diffusion policy used to implement ProgressVLA.

A. Progress Estimator

  • The progress estimator cross-attends instruction, start-observation, and current-observation features to predict a scalar progress score in [0, 1].It uses frozen DINOv2 visual features [34] and OpenAI CLIP ViT-L/14 text features [36], projected to a shared space and fused through a mean-pooling MLP with a sigmoid head.Source: A. Progress Estimator · p. 12
  • The estimator uses 768-dimensional DINO and CLIP features projected to width 1024, an 8-head, 6-block Spatio-Temporal Transformer [46], and a 512-hidden-unit sigmoid head.Pretraining uses at most 400 H20-hours and provides a reliable downstream guidance signal.Source: A. Progress Estimator · p. 12

B. World Model; A VQ bottleneck is used for latent-action discretization before passing az to downstream policy modules.

  • Following UniVLA [6], the world model extracts frozen DINOv2 features [34] from observations ot and ot+N, maps them to a feature-space latent action, and predicts future features conditioned on Ft and that action.The encoder and decoder use Spatio-Temporal Transformers [46] with width 768, 12 layers, and 12 attention heads; the latent action has dimension 128. Training uses L2 feature reconstruction, VQ commitment/codebook terms, and a KL regularizer.Source: B. World Model · p. 12, A VQ bottleneck is used for… · p. 12

C. Noise-conditioned evaluator

  • Candidate latent-action chunks are evaluated by applying the progress estimator to world-model-imagined futures, with a noise-conditioned form differentiable with respect to xτ for classifier-guidance gradients.The base world model is τ-agnostic; the guidance-time variant conditions on noisy latent actions and current DINO features.Source: C. Noise-conditioned evaluator · p. 12
  • A sinusoidal τ embedding is concatenated with the noisy latent-action token and current DINO features, and a Transformer decoder predicts future DINO features to stabilize guidance gradients across noise levels.This guidance-time conditioning does not change the base world-model formulation.Source: C. Noise-conditioned evaluator · p. 12
  • The world model and progress estimator are jointly finetuned on expert demonstrations under noise conditioning, then distilled into a noise-aware evaluator using 160 H20-hours for classifier guidance.Source: C. Noise-conditioned evaluator · p. 12

D. Latent Action Expert and Action Decoder

  • The policy factorizes execution into denoising a latent-action chunk and decoding it into an executable action chunk, using LLaMA2-style diffusion transformers with epsilon prediction [16].Each network processes compact visual tokens concatenated with language conditioning in the standard variance-preserving diffusion setting.Source: D. Latent Action Expert and Action… · p. 12

1) Latent Action Expert: … B. Policy execution and evaluation

The method uses coupled latent-action and action diffusion processes, then applies progress guidance during real-robot execution. In the orange-pickup rollout, guidance accompanies successful grasping and lifting with steadily rising predicted progress, unlike the unguided rollout.

1) Latent Action Expert:

  • The latent-action expert denoises a noisy latent chunk conditioned on the observation and instruction, using 1000 training timesteps to represent task-relevant high-level intent.The fine-grained schedule supports smoother intermediate noise levels for guidance and refinement.Source: 1) Latent Action Expert · p. 12

2) Action Decoder:

  • The action decoder conditions on the observation, instruction, and noisy latent plan to generate embodiment-consistent low-level actions using 100 training timesteps.The shorter schedule reduces inference cost while retaining sufficient fidelity.Source: 2) Action Decoder · pp. 12–13

E. Two-stage inference with latent warm-start

  • Inference first runs the latent denoiser from its Tz=1000 schedule tail to produce a partially denoised latent at the action diffusion’s Ta=100 noise scale.This warm-start aligns the latent and action processes despite their different denoising schedules.Source: E. Two-stage inference with latent… · p. 13

2) Stage 2: joint denoising of latent and action chunks:

  • A Ta=100-step loop then jointly denoises the latent and action chunks, with each action update conditioned on the updated latent; the final predicted action chunk is executed.Classifier guidance is used in the coupled updates.Source: 2) Stage 2: joint denoising of latent… · p. 13

A. Robot platform and sensors

  • Real-world trials use an ARX AC-One dual-arm platform with two X5 arms, parallel grippers, and wrist-mounted and fixed RGB-D cameras; policy input uses RGB only.Trials manipulate with the right arm while the left arm remains stationary.Source: A. Robot platform and sensors · p. 13, Figure 8 · p. 13

B. Policy execution and evaluation

  • With progress guidance, the robot grasps and lifts the orange as predicted progress rises monotonically; without it, grasping fails and progress oscillates.The comparison is shown for the instruction “pick up the orange and put it on the plate.”Source: B. Policy execution and evaluation · p. 13, Figure 9 · p. 14
  • During diffusion sampling, the evaluator-score gradient favors latent-action chunks predicted to make greater progress, helping select advancing actions while reducing steps and unnecessary motion.The reported qualitative comparison associates guidance with stable progress and successful orange pickup, versus unguided dithering.Source: B. Policy execution and evaluation · p. 13

C. Progress Estimator generalization … B. KL-constrained improvement

The pretrained-and-finetuned progress estimator maintains more usable progress traces than a from-scratch estimator under lighting and novel-object shifts. The appendix also defines an evaluator-based task-aware score and derives a KL-regularized policy improvement rule.

C. Progress Estimator generalization

1) Lighting shift:

  • Under lighting shift, the pretrained+finetuned estimator stays smooth and near-monotonic, reaching high progress in fewer steps, while the from-scratch estimator rises slowly and plateaus.The traces are for an expert-collected “stack the bowls” trajectory.Source: Figure 10 · p. 14, 1) Lighting shift · p. 14

2) Novel-object shift:

  • With novel objects, the pretrained+finetuned estimator remains smooth and near-monotonic, while the from-scratch estimator fluctuates more and gives a less separable near-completion region.The traces evaluate an expert-collected “pick up the apple and put it on the plate” trajectory.Source: Figure 11 · p. 14, 2) Novel-object shift · p. 14

D. Additional video results

  • Supplementary videos provide progress-guidance rollouts and progress-estimator examples organized by instruction and shift type.Guidance videos are in videos/Guidance/; estimator videos are in videos/Estimator/ and named novel_object and light_shifting.Source: D. Additional video results · p. 14

A. Task-aware score from the evaluator

  • The task-aware score uses state s = (ℓ, o0, ot), latent action a, and a learned evaluator comprising the world model and progress estimator.The score is the predicted progress after applying a through world model D.Source: A. Task-aware score from the evaluator · p. 14

B. KL-constrained improvement

  • Progress maximization is formulated as KL-regularized policy improvement, whose Lagrangian yields a unique Boltzmann-form optimum with a log-density differing by an additive energy term.The normalization Z(s) does not depend on a.Source: B. KL-constrained improvement · pp. 14–15

C. Instantiating π0 as a VP (Variance-Preserving) diffusion policy over latent actions · D. Classifier guidance on the denoising variable xτ · E. Converting guided score to a guided ϵ target (Eq. (19))

The sections instantiate a VP diffusion policy over latent actions, apply evaluator-based guidance to its denoising variable, and convert the guided score into a noise target for denoiser training.

C. Instantiating π0 as a VP (Variance-Preserving) diffusion policy over latent actions

D. Classifier guidance on the denoising variable xτ

E. Converting guided score to a guided ϵ target (Eq. (19))

Related papers

Loading 2603.27670v1…