PaperlayerView exact revision on arXiv

Source-linked AI summary

PermVLA: Factorization Order as a Regularizer for VLA Learning

Yanqiao Chen, Yuhan Rui, Dongsheng Hou, Zijie Nie, Yutong Wan, Qi Hao

arXiv:2610.04659v1cs.AIcs.RO

TL;DR

VLA policies commonly train action chunks with a fixed chronological factorization, although other valid reveal orders exist. PERMVLA adds a sampled-order auxiliary objective with a tunable chronological prefix while retaining LTR deployment; experiments report gains across benchmarks and greater agreement across reveal orders.

  • Problem

    VLA action-chunk training usually fixes chronological LTR factorization, leaving the regularization role of alternative valid chain-rule orders unexplored.

    Source: §1 · p. 1

  • Method

    CAP combines the LTR objective with sampled reveal-order likelihoods, training one shared policy on different known subsets while preserving chronological deployment.

    Source: §1 · pp. 1–2, §3.2 · p. 4

  • Results

    CAP improves matched-control closed-loop success across LIBERO, LIBERO-Plus, and CALVIN, with greater reveal-order agreement while retaining LTR fit.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The findings motivate studying alternative chain-rule factorizations and routinely evaluating out-of-distribution robustness in VLA policies.

    Source: §5 · p. 9

    The controlled comparisons center on Gaussian autoregressive policies in simulation; real-robot transfer and interaction with large-scale robot pretraining remain open.

    Source: §5 · p. 9

Abstract

from arXiv · show

Vision-language-action (VLA) policies commonly learn action chunks through a fixed left-to-right (LTR) factorization, although the same expert trajectory distribution admits many valid chain-rule factorizations. We identify factorization order as an overlooked regularization choice and introduce causally anchored permutation (CAP), which samples action reveal orders with a tunable chronological prefix. Its auxiliary objective trains one shared policy to predict actions from different known subsets of the same expert chunk, while deployment retains deterministic LTR control. We call this conditional-set augmentation: it creates multiple conditional prediction problems from one expert chunk without adding demonstrations. This discourages reliance on the single chronological prefix used by ordinary teacher forcing. Controlled experiments show that CAP consistently outperforms standard LTR training on LIBERO and LIBERO-Plus, with the same advantage appearing in cross-dataset CALVIN evaluation. A diagnostic that measures the expected squared difference between a chunk's joint log likelihood under two reveal orders verifies that CAP training internalizes agreement across reveal orders. These findings position sampled subset-conditioned auxiliary objectives as a general recipe for constructing VLA regularizers, illustrated by an extension to diffusion action generators.

1 INTRODUCTION

PERMVLA treats action-chunk factorization order as a training regularizer, using sampled conditioning subsets while preserving chronological deployment. Matched experiments report improved control over LTR baselines and greater agreement across reveal orders.

1 INTRODUCTION

  • Because LTR always conditions each slot on preceding physical actions, the authors hypothesize that predicting actions from varied known subsets favors dependencies useful across conditioning patterns.They suggest this may improve closed-loop control under shifts in visual, linguistic, or embodiment context.Source: §1 · p. 1
  • CAP adds an auxiliary permutation likelihood with a tunable chronological prefix, training one shared policy on different known-action subsets while retaining deterministic LTR deployment.The same expert chunk supplies multiple conditional prediction problems without changing its action identities or deployment procedure.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • CAP improves closed-loop success over both matched LTR controls across LIBERO, LIBERO-Plus, and CALVIN, while likelihood diagnostics show greater reveal-order agreement with LTR fit retained.The matched controls and diagnostic connect the training objective to changes in the learned policy.Source: §1 · p. 2
  • PERMDIFFUSION illustrates extending the subset-conditioned auxiliary-objective idea to diffusion action generators by denoising slots not in the clean conditioning subset.This extension is presented as an illustration of a broader design principle, not as a matched benefit comparison.Source: §1 · p. 2
  • The paper identifies factorization order as an overlooked VLA regularization axis and frames training across conditioning subsets as conditional-set augmentation.The authors distinguish this training-only regularizer from the chronological objective used for deployment.Source: §1 · p. 2

2 RELATED WORK

The paper distinguishes CAP from permutation and masking methods that use variable orders or masks as their primary objectives. It instead uses permuted likelihood as an auxiliary VLA regularizer alongside chronological training and deployment.

Order-agnostic and masked sequence modeling

  • Prior work spans orderless density estimation, permutation language modeling, autoregressive diffusion, multi-agent trajectory factorization, and variable-mask trajectory completion.The paper positions CAP as importing permutation modeling into VLA learning in the distinct role of a training-only regularizer.Source: §2 · p. 3

Permutation training as a VLA regularizer

  • CAP uses an auxiliary permuted likelihood on top of the chronological density objective, rather than making permutation or masking the primary training objective.It fixes the backbone and continuous action generator to isolate factorization order as a training intervention.Source: §2 · p. 3
  • Unlike completion-style masked objectives, CAP assigns an ordered, normalized likelihood to each sampled reveal order within one future action chunk.The tunable prefix K controls the order family, including unrestricted orders at K = 0.Source: §2 · p. 3

3 METHOD

PERMVLA models future action chunks conditioned on current language, visual history, and proprioception, then trains a shared policy with chronological and sampled-order objectives. A tunable anchored prefix controls the training order family, while the architecture and deployment preserve chronological action identity and execution.

3.1 PROBLEM FORMULATION

  • The policy predicts a length-H action chunk from language, visual history, and proprioception, with conditioning values available from the control interface or previously revealed actions.Each action retains its physical slot index when reveal order changes.Source: §3.1 · p. 3

3.2 PERMUTATION MODELING WITH A TUNABLE ANCHOR

  • K-anchored orders fix the first K reveal positions to chronological slots and permute the remainder; K = 0 permits all orders and K = H recovers LTR.CAP uses revealed subsets to create missing-action prediction problems while preserving physical indices and chronological deployment.Source: §3.2 · p. 4

3.3 REFERENCE ARCHITECTURE

  • The reference policy uses a pretrained VLM and Perceiver-style resampler to condition a two-stream action decoder, whose attention masks implement the sampled reveal order.Physical-time embeddings preserve slot identity, and the Gaussian head reads query states whose targets are masked.Source: §3.3 · p. 4, Figure 2 · p. 5

3.4 TRAINING OBJECTIVE

  • The training loss combines LTR and Monte Carlo CAP likelihoods as (1 − λ)LLTR + λLCAP; the reference uses K = 1, λ = 0.25, and M = 1.Both objectives reuse one context encoding and share decoder parameters; increasing M adds training-time decoder work while leaving parameter count and LTR inference cost fixed.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments test CAP against matched LTR controls, then assess its performance across LIBERO, LIBERO-Plus, and CALVIN and measure agreement across reveal orders.

4.1 EXPERIMENTAL SETUP

  • Matched controls hold the model and training protocol fixed while isolating the CAP objective; evaluation spans LIBERO, zero-shot LIBERO-Plus perturbations, and five-instruction CALVIN chains.The experiments also measure how joint likelihood changes across reveal orders.Source: §4 · p. 5, §4.1 · p. 6

4.2 LIBERO EVALUATION OF THE REFERENCE POLICY

  • The compact Gaussian policy reaches the range of established VLA systems, while shorter horizons give the best overall balance and temporal settings redistribute suite performance.Long-horizon control remains the main gap.Source: §4.2 · p. 6

4 EXPERIMENTS

  • CAP exceeds both LTR controls on every LIBERO suite, improves all seven LIBERO-Plus perturbation aggregates, and leads both controls at every CALVIN chain depth.The CALVIN ordering reproduces the LIBERO ordering.Source: §4.3 · p. 6, §4.4 · p. 8, §4.5 · p. 8

4.3 CONTROLLED EVALUATION OF THE CAP REGULARIZER

  • Prefix length is task-dependent: K = 0 keeps the four-suite mean close to K = 1, while temporal settings and offline fit can diverge from closed-loop success.The largest CAP–LTR gap occurs on Long, where CAP’s stronger control contrasts with LTR’s slightly closer expert-action fit.Source: §4.3 · pp. 6–7

4.4 ZERO-SHOT ROBUSTNESS ON LIBERO-PLUS

  • 20 of 28 suite–perturbation combinations favor CAP over LTR-1pass, and 25 of 28 favor CAP over compute-matched LTR-2pass.CAP also raises the zero-shot mean in every suite; gains are broadest for Language and Background, while Long improves under every perturbation family.Source: §4.4 · p. 8

4.5 CROSS-BENCHMARK TRANSFER ON CALVIN

  • The CAP-to-LTR-1pass Success@k ratio rises from 1.04 at Success@1 to 2.65 at Success@5, and CAP matches Diffusion Policy-CNN’s average completed chain length.LTR-2pass has the lowest logged pointwise action error but the weakest control.Source: §4.5 · p. 8

4.6 FACTORIZATION SENSITIVITY DIAGNOSTIC

  • CAP lowers factorization sensitivity by several orders of magnitude across suites while preserving chronological LTR likelihood, but the diagnostic does not by itself rank control performance.Conditional independence can minimize sensitivity trivially, and K = 8 raises sensitivity under Π1 while improving Long success.Source: §4.6 · pp. 8–9

5 DISCUSSION

The discussion frames factorization order as a VLA regularization choice, while identifying adaptive factorization, broader action generators, and simulation-focused evidence as open directions.

5 DISCUSSION

  • CAP exposes factorization order as a controllable VLA supervision choice, and LIBERO-Plus and CALVIN show that the resulting action dependencies remain useful beyond training environments.The diagnostic indicates less reveal-order dependence in a chunk’s joint likelihood.Source: §5 · p. 9

Temporal scope and adaptive factorization

  • Suite-dependent effects motivate learning reveal-order distributions, selecting K from predictive uncertainty, and using curricula that move from LTR toward more permuted orders.Rollouts without teacher forcing and perturb-and-recover tests could distinguish continuation, calibration, and recovery.Source: §5 · p. 9

Action generators and scaling

  • PERMDIFFUSION extends the idea by treating sampled action slots as clean inputs while denoising the others.Matched diffusion, flow, token-autoregressive, multi-effector, and pretrained-VLA studies remain proposed tests of scaling and regularization.Source: §5 · p. 9

Limitations

  • The controlled comparisons center on Gaussian autoregressive policies in simulation; real-robot transfer and interaction with large-scale robot pretraining remain open.Factorization sensitivity measures agreement over sampled reveal orders, and identifying which learned dependencies drive OOD gains requires targeted interventions.Source: §5 · p. 9

6 CONCLUSION

The conclusion presents action-chunk factorization order as a VLA regularization axis and points to broader policy types and evaluations for future study.

6 CONCLUSION

  • CAP trains over sampled reveal orders with a tunable prefix but retains chronological deployment, and matched controls separate its gains from computation and pointwise fit.The conclusion reports consistent generalization across environments and datasets and agreement across reveal orders.Source: §6 · p. 9

A EXPERIMENTAL DETAILS AND SUITE-RESOLVED RESULTS

  • The appendix consolidates configurations, suite-resolved LIBERO-Plus results, training diagnostics, parameter semantics, and reproducibility settings.Appendix D adds full robot rollouts and an audit of demonstration endings and success criteria.Source: A EXPERIMENTAL DETAILS AND… · p. 14

A.1 BACKBONE REPLACEMENT ABLATION

The backbone replacement ablation tests whether CAP remains effective with a different visual-language representation, holding the policy and training protocol fixed. The reported PaliGemma results retain strong performance, though the four-suite mean remains slightly below the Qwen3-VL policy.

A.1 BACKBONE REPLACEMENT ABLATION

  • The ablation fixes the action decoder, CAP objective, data, optimization budget, and evaluation protocol while replacing the multimodal backbone.It tests whether the factorization regularizer and Gaussian action policy remain effective with a different visual-language representation.Source: §A.1 · p. 14
  • Table 5 reports 500-episode LIBERO success rates for the backbone replacement under the fixed CAP M = 1, H = 16, O = 1, e = 8 protocol.The supplied passages do not provide the table’s individual scores.Source: Table 5 · p. 14
  • PaliGemma 2–3B improves Goal by 9.2 points while retaining strong Spatial and Object performance under the fixed policy and training protocol.Its four-suite mean remains within 2.9 points of the Qwen3-VL policy despite a lower Long score.Source: §A.1 · p. 14

A.2 FOUR-SUITE LIBERO CONFIGURATIONS

  • Table 6 provides absolute scores for the four-suite LIBERO configurations, while Table 7 reports signed changes for one-factor interventions and coupled configurations.The supplied passages describe these tables’ scope but not their individual values.Source: §A.2 · p. 14, §A.3 · p. 14

A.4 TASK-RESOLVED PARAMETER COMPARISONS

Task-resolved comparisons show that aggregate effects can conceal reversals across tasks, motivating parameter choices tailored to the control problem. Targeted diagnostics also examine execution length and CAP gradient routing.

A.4 TASK-RESOLVED PARAMETER COMPARISONS

  • Tables 8–11 expand the completed rows of Table 2 into task-resolved comparisons for LIBERO suites.The supplied table captions identify Spatial, Object, Goal, and Long task layouts, but do not provide their cell values.Source: §A.4 · p. 14, Table 8 · p. 16, Table 9 · p. 16
  • Task-level changes vary by suite and parameter: O2 Object gains are broad, while its Spatial regression concentrates in stove and cabinet placements.The stronger CAP weight improves six of ten Long tasks and several weak Goal tasks, but its Spatial and Object losses concentrate in a few placements and object identities.Source: §A.4 · p. 14
  • Similar suite means can conceal large task-level reversals, motivating adaptation of prefix length and regularization strength to the control problem.The reported examples include K0 Object losses and Goal gains, and a Long loss on the stove-and-moka-pot task despite gains on basket and placement tasks.Source: §A.4 · pp. 14–15

A.5 TARGETED SINGLE-SUITE DIAGNOSTICS

  • For the H = 8 reference, executing the full predicted chunk gives the strongest tested Long result, while preferred execution length remains task- and model-dependent.CAP is nearly unchanged from e = 8 to e = 16 in the O = 1 family; the O = 2 policy degrades at e = 16.Source: §A.5 · p. 15
  • Blocking CAP gradients from semantic context lowers Goal under the matched M = 4, H = 16, O = 2, e = 8 configuration.The result supports jointly shaping shared context and action decoder in this experiment.Source: §A.5 · p. 16, Table 13 · p. 17

A.6 SUITE-RESOLVED LIBERO-PLUS ROBUSTNESS

The appendix reports suite-resolved LIBERO-Plus robustness, training diagnostics, implementation details, and a diffusion extension. CAP’s gains span perturbation families, while the diffusion construction preserves standard DDIM deployment.

A.6 SUITE-RESOLVED LIBERO-PLUS ROBUSTNESS

  • Robustness depends on task structure: Long degrades broadly under visual shifts, while Spatial and Object retain language-conditioned behavior well.The passages caution that interpreting robustness requires both perturbation factor and task structure.Source: §A.6 · p. 17
  • CAP improves 25 of 28 suite–factor cells against compute-matched LTR-2pass, raises the Plus mean in every suite, and improves all seven pooled perturbation families.The matched H = 16 CAP comparison improves 20 of 28 cells against LTR-1pass, including every Language block, every Background block, and all seven Long blocks.Source: §A.6 · pp. 17, 19, Table 17 · p. 18

A.8 PARAMETER MEANINGS AND SWEEP SEMANTICS

  • The appendix identifies Table 20 as defining principal PermVLA parameter meanings and distinguishing the reference from measured settings.The supplied passage does not enumerate those individual meanings or settings.Source: Table 20 · p. 19

A.7 TRAINING DIAGNOSTICS

  • In matched Long diagnostics, LTR-2pass improves closed-loop control over one-pass LTR, and CAP improves it further after the same update and sample budget.This ordering isolates a behavioral contribution from diverse conditioning sets beyond repeated decoder computation.Source: §A.7 · p. 19, Table 19 · p. 19

A.9 CONDITIONAL TWO-STREAM ATTENTION IMPLEMENTATION

  • The conditional two-stream implementation uses physical-time embeddings and reveal ranks to determine which action slots each query can see.Content states can read their own action and previously revealed actions; query states read the strict revealed prefix.Source: §A.9 · p. 20

A.10 EXPERIMENTAL PARAMETERS

  • The CALVIN transfer protocol trains matched objectives on 17,870 ABC play-data episodes and evaluates fixed five-instruction chains in unseen Scene D.Evaluation uses 1,000 chains, with at most 360 environment steps per instruction.Source: §A.10 · p. 20

B PERMUTATION-REGULARIZED DIFFUSION POLICIES

  • PERMDIFFUSION extends subset-conditioned regularization to diffusion policies, using clean revealed slots and noisy prediction slots in its auxiliary training objective.The construction keeps physical action positions fixed and uses the permutation to select revealed subsets; m = 0 gives fully noisy unconditional denoising.Source: B PERMUTATION-REGULARIZED DIFFUSION… · p. 20, §B.1 · p. 21

B.2 ARCHITECTURE, TRAINING, AND SAMPLING

  • PERMDIFFUSION uses a shared denoiser and retains standard synchronous DDIM deployment, beginning from Gaussian noise and updating the full physical-time chunk.Training uses per-slot diffusion times and revealed-mask embeddings; the permutation subset construction is training-only.Source: §B.2 · pp. 21–22, Figure 3 · p. 22

B.3 LIBERO SUITE RESULTS

The PERMDIFFUSION suite matrix averages 74.6% and retains standard DDIM sampling. The accompanying theory separates the exact population result from a proposed finite-capacity regularization mechanism based on conditional-set augmentation.

B.3 LIBERO SUITE RESULTS

  • 74.6% is the average of the complete H = 8 PERMDIFFUSION LIBERO matrix, which spans spatial, object, goal-conditioned, and multi-stage control while retaining standard DDIM sampling.Relative to the temporally matched CAP reference, the diffusion prototype is close on Spatial, lower on Object and Goal, and higher on Long.Source: §B.3 · p. 22

C.1 EXACT CHAIN-RULE RESULT ACROSS REVEAL ORDERS

  • Under realizability, one shared parameter setting represents the expert distribution under every reveal order, so CAP preserves the expert population target while changing finite-model optimization geometry.The result holds for every prefix length K, while the retained LTR objective and chronological deployment specify the control path.Source: §C.1 · p. 24

C.2 FINITE-CAPACITY REGULARIZATION THROUGH CONDITIONAL SETS

  • Each demonstrated chunk creates multiple conditional prediction problems under sampled subsets, while preserving its physical target trajectory.Finite demonstrations, restricted capacity, shared features, and stochastic optimization couple these factorization tasks.Source: §C.2 · p. 24
  • Conditional-set augmentation gives repeated support to features predictive across several revealed subsets, proposing a finite-capacity regularization preference among models fitting the same LTR examples.The passage identifies this preference as the proposed regularization mechanism.Source: §C.2 · p. 24

C.3 FACTORIZATION SENSITIVITY AS A DIAGNOSTIC

The diagnostic links order-diverse CAP supervision to agreement across reveal orders, while later analyses delimit that agreement’s support and examine temporal, scaling, rollout, and supervision boundaries.

C.3 FACTORIZATION SENSITIVITY AS A DIAGNOSTIC

  • CAP reduces median reveal-order sensitivity by several orders of magnitude and the arbitrary-order NLL increase from tens of nats per slot to approximately one, while retaining closely matched LTR NLL.A second LTR pass leaves cross-order agreement and robustness to unseen factorization orders unchanged.Source: §C.3 · p. 25, Table 23 · p. 25
  • Stronger CAP weight reduces disagreement and raises success on both suites, but more sampled orders improve Long without changing the global sensitivity statistic, and invariance follows the trained order family.The narrower K = 8 policy becomes more sensitive on the broader unseen Π1 support, whereas the broadly trained Π1 policy remains consistent on nested Π8.Source: Table 24 · p. 25, §C.3 · p. 26, Table 25 · p. 26

C.4 CONNECTION TO OOD GENERALIZATION

  • CAP is predicted to help most when shifts disrupt context correlations but preserve within-chunk action dependencies; gains may shrink when action semantics or information needed by conditional views change.The discussion recommends controlled camera, language, embodiment, and scene shifts alongside cross-benchmark transfer and suite-resolved reporting.Source: §C.4 · p. 26

C.5 TEMPORAL SCOPE, SCALING, AND EXTENSIONS

  • CAP regularizes length-H expert chunks, while closed-loop behavior also depends on rollout shift, partial observability, latency, recovery, and the executed prefix.The K = 0 ablation retains average LIBERO performance while trading Object success for gains on Goal and Long; longer anchors preserve more deployment order, and shorter execution prefixes increase feedback.Source: §C.5 · p. 27
  • The decoder-only Goal experiment weakens the result, and extending CAP to other generators requires matched base-objective controls and explicit definitions of the permuted variables.The discussion proposes layer-selective gradient routing and extensions to diffusion, flow, token, and multi-effector policies.Source: §C.5 · p. 27

D.1 MATCHED ROLLOUTS THROUGH THE TIME LIMIT

  • In the selected table-appearance-shift rollout, CAP retains success and leaves the pot closer to upright, but it remains tilted and in gripper contact.LTR-1pass eventually leaves the pot on its side; LTR-2pass contacts the stove without satisfying the placement predicate.Source: §D.1 · pp. 27–28, Table 26 · p. 27, Figure 6 · p. 28

D.2 WHAT THE SUCCESS PREDICATE MEASURES

  • The stove-placement predicate omits upright orientation, base support, gripper release, and settling, so a sideways pot can count as success and first-success stopping hides later goal loss.Binary success therefore does not fully capture placement quality or post-success behavior.Source: §D.2 · p. 28

D.3 DEMONSTRATION ENDINGS AND SUPERVISION QUALITY

  • Among 41 audited demonstrations, 38 end with an opening command and three with a closing command; most opening endings provide only a short release-and-withdrawal segment.The audit suggests heterogeneous terminal supervision can blur goal-region arrival and clean placement, while improving the expert targets remains complementary to CAP.Source: §D.3 · pp. 28–29

D.4 ADDITIONAL TASKS IN ORIGINAL LIBERO

Matched full-budget rollouts on two original LIBERO tasks show differences in completing multi-stage instructions and maintaining the official goal predicate.

D.4 ADDITIONAL TASKS IN ORIGINAL LIBERO

  • Both tasks use the first evaluation initial state, matched LTR controls, and aligned physical action-step columns through the full-budget protocol.A green or red final border marks whether the official goal predicate holds at the time limit.Source: §D.4 · p. 29, Figure 8 · p. 30, Figure 9 · p. 31
  • CAP completes both bowl placement and drawer closure, whereas LTR-1pass places the bowl but leaves the drawer open.The policies start from the same initial state and execute through all 520 actions; the final frame indicates whether the official predicate holds.Source: §D.4 · p. 29, Figure 8 · p. 30
  • In the two-object basket task, CAP and LTR-2pass place both objects and maintain the goal through the remaining actions, while LTR-1pass places only the soup.The first joint-placement actions are 230 for CAP and 338 for LTR-2pass.Source: §D.4 · p. 29, Figure 9 · p. 31
Loading 2610.04659v1…