PaperlayerView exact revision on arXiv

Source-linked AI summary

MixVLA: Adaptive Mixing of Non-Invariant Information for Generalizable Vision-Language-Action Models

Pingrui Zhang, Yu Zhang, Pengyuan Wu, Bin Wang, Haoming Song, Xianqiang Gao, ZhaxiZhuoma, Zhigang Wang, Dong Wang, Bin Zhao, Xuelong Li

arXiv:2610.02898v1cs.RO

TL;DR

VLA models remain vulnerable to OOD observation shifts, and existing robustness routes can require costly data collection or model-specific changes. MixVLA uses AMI to regularize non-invariant cues while fusing them with invariant features, and reports improved zero-shot robustness with strong in-domain performance. On LIBERO-Plus, it achieves the highest overall success rate among compared methods.

  • Problem

    VLA models struggle with zero-shot generalization under OOD shifts, while common approaches rely on costly diverse-data collection or model-specific modifications.

    Source: §1 · p. 1

  • Method

    MixVLA uses an information-theoretic objective to obtain invariant representations, stochastically mixes non-invariant features, and fuses both for action prediction without extra data or architectural changes.

    Source: §1 · p. 2, §3.1 · p. 4, §3.3 · p. 5

  • Results

    MixVLA improves overall zero-shot robustness while retaining strong in-domain performance; on LIBERO-Plus it achieves 76.2% overall success, 6.6% above OpenVLA-OFT.

    Source: §4.1 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    The findings suggest non-invariant information can provide useful predictive cues when appropriately regularized, offering a practical direction for generalizable embodied policies without additional OOD training data.

    Source: §5 · p. 10

    Improvements are less pronounced under large geometric shifts such as camera-viewpoint changes than under appearance-related perturbations.

    Source: D Discussion · p. 22

Abstract

from arXiv · show

Vision-Language-Action (VLA) models have achieved remarkable advances in robotic manipulation, yet their zero-shot generalization under out-of-distribution (OOD) conditions remains limited. These models often entangle task-relevant invariant structure with environment-specific non-invariant factors, causing policies to rely on spurious appearance cues during action prediction. In this work, we propose \textbf{MixVLA}, a model-agnostic training framework that improves the generalization of VLA models without requiring additional OOD data or architectural modifications. The key component of MixVLA is \textbf{Adaptive Mixing of Non-Invariant Information (AMI)}. AMI stochastically mixes non-invariant representations to regularize distribution-specific variability while preserving complementary predictive cues. The mixed non-invariant features are then fused with invariant representations for final action prediction, resulting in improved robustness without sacrificing policy expressiveness. Extensive experiments across challenging manipulation settings, including LIBERO, LIBERO-Plus, the RoboTwin perturbation suite, and real-world tasks, demonstrate that MixVLA improves overall zero-shot robustness while retaining strong in-domain performance.

1 Introduction

VLA models perform well in-distribution but remain vulnerable to OOD observation shifts, while common robustness approaches can require costly data collection or model-specific changes. MixVLA instead regularizes non-invariant signals and reports improved zero-shot robustness across manipulation settings.

1 Introduction

  • VLA policies can degrade under mild lighting, appearance, or camera-viewpoint shifts, while data scaling is costly and model-specific modifications limit general applicability.The paper therefore targets a model-agnostic strategy that requires no additional OOD data.Source: §1 · p. 1, Figure 1 · p. 2
  • ERM can capture stable task cues alongside environment-specific signals, which may create spurious dependencies under shifts; strict invariance can discard useful predictive information.This motivates regularizing non-invariant variability rather than relying only on invariant features.Source: §1 · p. 2
  • MixVLA improves zero-shot robustness across LIBERO, LIBERO-Plus, RoboTwin, and real-world tasks without additional data or architectural modifications.Its core mechanism stochastically mixes non-invariant representations and fuses them with invariant features for action prediction.Source: §1 · pp. 2–3
  • AMI adaptively mixes non-invariant representations and fuses them with invariant features to improve robustness under distribution shifts.The contribution is framed as retaining complementary predictive cues while regularizing environment-specific variability.Source: §1 · pp. 2–3

2 Related Work

Prior VLA work advances capability through scaling, richer representations, supervision, and post-training, while robust generalization to unseen observation variations remains an open challenge. MixVLA addresses environmental variability with a model-agnostic training strategy.

2.1 Vision-Language-Action Model

  • VLA research spans scaled multimodal pretraining and robot data, spatial representations, intermediate supervision, and post-training optimization, yet robustness to unseen observation variations remains open.MixVLA focuses on intrinsic generalization through a model-agnostic training strategy.Source: §2.1 · p. 3

2.2 Generalization in Robotic Manipulation

  • Data scaling can be costly, while specialized modules may add complexity and fail to cover multifaceted shifts; object-centric methods also leave environmental nuisance variation unaddressed.MixVLA instead regularizes non-invariant environmental variability without extra data or specialized modules.Source: §2.2 · pp. 3–4
  • Figure 2 depicts MixVLA’s decomposition and regularization stages: deriving a non-invariant branch, mixing its embeddings within a mini-batch, then fusing them with invariant embeddings.The fused representation is passed to the policy head for training.Source: Figure 2 · p. 4

3 Method

MixVLA separates invariant and complementary non-invariant representations, regularizes the latter through stochastic mixing, and fuses both for action prediction using the original policy head.

3.1 Preliminaries

  • A VLA policy maps an observation-instruction pair to actions through a latent representation and policy head; MixVLA operates at that latent level without changing the backbone or head.It distinguishes invariant embeddings Ie from non-invariant embeddings NIe and fuses them as h = [Ie ∥ ˜NIe].Source: §3.1 · p. 4

3.2 Invariant and Non-Invariant Decomposition

  • The Invariant Net uses an information bottleneck objective to compress observation information while preserving action-predictive information.The non-invariant branch is derived by weight subtraction, WNonInv = WNormal − WInv, as a candidate complementary branch, not an assumed exact separation.Source: §3.2 · pp. 4–5

3.3 Adaptive Mixing of Non-Invariant Information

  • AMI couples non-invariant embeddings from different samples to smooth environment-conditioned variability; soft mixing interpolates feature statistics, while hard mixing interpolates embeddings.The two variants provide complementary regularization effects.Source: §3.3 · p. 5

3.4 Training Objective and Integration with VLA

  • The mixed non-invariant embedding is concatenated with the unchanged invariant embedding and fed to the original policy head under the base VLA training objective.This preserves the base policy interface without architectural changes.Source: §3.4 · p. 6

4 Experiment

The experiments test zero-shot robustness, the role of non-invariant information, generality across policies and real-world deployment, and reduced training requirements. Across unseen shifts, MixVLA improves robustness while retaining strong in-domain performance.

4 Experiment

  • The evaluation spans LIBERO and LIBERO-Plus, clean-to-clean and clean-to-randomized RoboTwin tests, and real-robot robustness, using OpenVLA-OFT and π0.5.Main simulation training used 100K steps for the Normal and Invariant Nets, followed by 1K AMI steps on LIBERO or 10K on RoboTwin.Source: §4 · pp. 6–7

4.1 Robustness to Unseen Distribution Shifts

  • 76.2% overall success on LIBERO-Plus surpassed OpenVLA-OFT by 6.6%; on RoboTwin, MixVLA raised C2R success from 46.0% to 62.4% (+16.4%).MixVLA also raised RoboTwin C2C success from 70.7% to 75.6% (+4.9%).Source: §4.1 · p. 7, Table 3 · p. 7

4.2 Understanding the Role of Non-Invariant Information

  • Across LIBERO-Plus and RoboTwin, AMI-regularized non-invariant features improved over IB-only by 4.0% and 2.8% on OOD performance, respectively.MixVLA outperformed Invariant Net across all seven LIBERO-Plus perturbation categories while largely preserving clean-environment performance on RoboTwin.Source: §4.2 · p. 8, Table 4 · p. 8
  • Qualitative analyses show MixVLA retaining task-associated feature neighborhoods with local clean–randomized interleaving, while succeeding on a task where the Normal and Invariant Nets fail.The t-SNE projections illustrate feature organization rather than establishing complete domain alignment.Source: §4.2 · p. 8, Figure 4 · p. 8

4.3 Generality across Policies and Real-World Deployment

  • MixVLA improved robustness over corresponding baselines for both OpenVLA-OFT and π0.5, despite their different action-generation mechanisms.OpenVLA-OFT uses parallel action decoding, whereas π0.5 uses flow matching.Source: §4.3 · p. 9
  • In real-world bread placement under lighting perturbations, MixVLA achieved 60% success over 10 trials, compared with 20% for π0.5.The policies were deployed on a Franka robot.Source: §4.3 · p. 9, Figure 5 · p. 9

4.4 Robustness from a Single Training Run

  • Self-MixVLA reuses intermediate checkpoints from one Invariant Net training run, eliminating separate Normal Net training while retaining robustness benefits with a modest performance reduction.It forms a Self Non-Invariant Net from the 10K- and 100K-step checkpoints.Source: §4.4 · p. 9, Table 5 · p. 9

5 Conclusion

MixVLA improves VLA generalization by regularizing rather than discarding non-invariant information. Experiments show improved zero-shot robustness while retaining strong in-domain performance.

5 Conclusion

  • MixVLA learns invariant representations through an information bottleneck objective and derives a complementary branch through weight arithmetic.The framework improves VLA generalization by regularizing rather than discarding non-invariant information.Source: §5 · p. 9
  • AMI stochastically mixes non-invariant features before fusing them with invariant representations for action prediction.The approach regularizes non-invariant features.Source: §5 · pp. 9–10
  • Experiments demonstrate improved overall zero-shot robustness while retaining strong in-domain performance.The findings suggest non-invariant information can provide useful predictive cues when appropriately regularized.Source: §5 · p. 10

Abstract

The supplementary material outlines theoretical analyses, implementation details, additional experiments, and limitations; its theory examines information compression and when adaptive mixing can improve OOD prediction.

Abstract

  • The supplement adds theoretical analyses, implementation details, experiments, and a discussion of limitations and future directions.Its analyses address information compression, adaptive-mixing statistics, and an illustrative regression model.Source: Abstract · p. 15

A Theoretical Analysis

  • The theoretical analysis characterizes information compression and adaptive-mixing statistics, and presents a local-linearization interpretation and an illustrative regression model with explicit conditions for improved OOD prediction.These analyses do not establish unconditional guarantees for nonlinear VLA policies.Source: A Theoretical Analysis · p. 15

A.1 Invariant Representation via Information Bottleneck

  • The Information Bottleneck objective bounds conditional information about environment-dependent factors beyond task-relevant factors, but practical encoders need not eliminate nuisance dependence or generalize to arbitrary unseen environments.A complementary representation may retain predictive information missing from the learned invariant representation.Source: §A.1 · p. 16

A.2 Generalization Benefit of Adaptive Mixing

The analysis studies adaptive mixing in an illustrative regression setting and distinguishes hard and soft feature regularization. It identifies conditions for lower target error, not a universal advantage across models.

A.2 Generalization Benefit of Adaptive Mixing

  • Under its stated regression conditions, AMI has strictly lower target squared error than ERM and invariant-only prediction.The result is explicitly conditional, not a universal ranking of the three predictors.Source: §A.2 · p. 17
  • The regression model assumes independent pairing, population optimization, a linear predictor, and a particular noise-variance shift.Its benefit is not established for real VLA features satisfying none of these assumptions.Source: §A.2 · p. 17

A.3 Special Cases of AMI

  • Hard mixing interpolates complete embeddings, whereas soft mixing interpolates feature statistics; their effectiveness depends on the learned representations and is evaluated empirically.Soft mixing contracts statistic-vector covariance in an idealized setting while retaining the normalized source pattern, which alone does not guarantee preserved task semantics.Source: §A.3 · p. 18

B Implementation Details

The implementation applies MixVLA to OpenVLA-OFT and π0.5 with different fine-tuning strategies, inputs, and information-bottleneck critics.

B Implementation Details

  • OpenVLA-OFT uses LoRA adapters while freezing base weights, and predicts eight 7-dimensional actions from camera images, language, and proprioception.The continuous action head and proprioceptive projector are trainable.Source: B Implementation Details · p. 18
  • The π0.5 implementation fully fine-tunes the policy, including its visual encoder, using three RGB views, language, and robot state with an action horizon of 50.It uses a flow-matching objective.Source: B Implementation Details · p. 18
  • OpenVLA uses separate schedules for its baseline and invariant nets versus its short AMI stage, while π0.5 uses warmup followed by cosine learning-rate decay.The OpenVLA AMI learning rate remains constant during its short training stage.Source: B Implementation Details · p. 19
  • The implementations use different information-bottleneck critics: Donsker–Varadhan MINE for OpenVLA and a Jensen–Shannon critic for π0.5.Their critic architectures, coefficients, and optimization settings also differ.Source: B Implementation Details · p. 19

C More Experiments

Additional experiments compare AMI mixing strategies, mixing probabilities, and real-world performance under lighting changes. Soft mixing has the best average performance across perturbation types, while success varies with perturbation and mixing probability.

C More Experiments

  • Both mixing strategies improve LIBERO-Plus robustness, with soft mixing achieving the best average performance across perturbation types.Hard mixing interpolates embeddings; soft mixing mixes feature statistics through adaptive normalization.Source: Table 6 · p. 19, C More Experiments · p. 20
  • Figures 7 and 9 present representative trajectories in standard LIBERO scenes and under LIBERO-Plus observation perturbations, respectively.The LIBERO-Plus scenes include changes in background texture, lighting, and visual noise.Source: Figure 7 · p. 20, C More Experiments · p. 21, Figure 9 · p. 22
  • On 2,318 LIBERO-Plus evaluation episodes, both mixing strategies peak at probability 0.1 and show non-monotonic success rates as probability increases.Soft mixing drops at 0.4 and hard mixing at 0.5; soft mixing fluctuates less from 0.6 to 0.9.Source: C More Experiments · p. 21
  • MixVLA achieves 60% aggregate success across two real-world lighting conditions, compared with 20% for π0.5.It succeeds in five of five low-frequency flashing trials but only one of five high-frequency multicolored flashing trials; π0.5 succeeds in two and zero, respectively.Source: C More Experiments · p. 21
  • The supplementary video includes qualitative MixDP executions on an xArm 6, illustrating application of the framework to Diffusion Policy.The work's investigation was motivated by observed failures when ambient lighting differed from demonstration collection conditions.Source: C More Experiments · pp. 21–22

D Discussion

MixVLA’s discussion identifies large geometric shifts, especially camera-viewpoint changes, as a remaining robustness limitation. It outlines spatially informed approaches and future directions involving finer mixing, online adaptation, and checkpoint-trajectory reuse.

Limitations under Large Geometric Shifts

  • Camera perturbations yield less pronounced improvements than appearance-related perturbations, and large viewpoint shifts may make latent-space vicinal smoothing less effective.Addressing such geometric shifts may require explicit spatial reasoning or multi-view representations.Source: D Discussion · pp. 22–23

Future Research Directions

  • Structure-aware or attention-guided mixing may provide finer control over regularizing non-invariant information.These are proposed as more advanced mixing strategies.Source: D Discussion · p. 23
  • Integrating AMI with test-time adaptation or online representation updates may further improve robustness in highly dynamic environments.Source: D Discussion · p. 23
  • Self-MixVLA motivates exploring optimization trajectories to reuse parameter-update directions across checkpoints and narrow the robustness gap to full MixVLA without a separate Normal Net training run.Adaptive checkpoint selection and trajectory-based weight extrapolation are proposed for this direction.Source: D Discussion · p. 23

Related papers

Loading 2610.02898v1…