PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization

Ye Wang, Sipeng Zheng, Hao Luo, Wanpeng Zhang, Haoqi Yuan, Chaoyi Xu, Haiweng Xu, Yicheng Feng, Mingyang Yu, Zhiyu Kang, Zongqing Lu, Qin Jin

arXiv:2602.09722v1cs.RO

TL;DR

The paper asks whether the standard data-scaling recipe reliably transfers to robotics despite heterogeneous embodiments, sensors, and action spaces. It conducts controlled VLA ablations with flow-matching control and blinded real-robot evaluation, finding that EEF-relative alignment is strongest while naive data mixing and intuitive regularization often fail to improve performance.

  • Problem

    It remains unclear whether scaling heterogeneous robotic data reliably improves VLA generalization across incompatible embodiments, sensors, and action spaces.

  • Method

    The study performs controlled ablations of physical alignment, embodiment mixtures, and training regularization in a VLA framework combining a vision-language backbone with flow matching, alongside blinded real-robot evaluation.

  • Results

    EEF-relative actions are the most effective alignment choice, heterogeneous data mixing frequently degrades performance, and sensory dropout or multi-stage fine-tuning do not consistently improve results at scale.

  • Takeaways & Limitations

    Large-scale VLA training benefits from careful physical action alignment and data curation rather than indiscriminate data or training-recipe expansion.

Abstract

from arXiv · show

While Vision-Language-Action (VLA) models show strong promise for generalist robot control, it remains unclear whether -- and under what conditions -- the standard "scale data" recipe translates to robotics, where training data is inherently heterogeneous across embodiments, sensors, and action spaces. We present a systematic, controlled study of VLA scaling that revisits core training choices for pretraining across diverse robots. Using a representative VLA framework that combines a vision-language backbone with flow-matching, we ablate key design decisions under matched conditions and evaluate in extensive simulation and real-robot experiments. To improve the reliability of real-world results, we introduce a Grouped Blind Ensemble protocol that blinds operators to model identity and separates policy execution from outcome judgment, reducing experimenter bias. Our analysis targets three dimensions of VLA scaling. (1) Physical alignment: we show that a unified end-effector (EEF)-relative action representation is critical for robust cross-embodiment transfer. (2) Embodiment mixture: we find that naively pooling heterogeneous robot datasets often induces negative transfer rather than gains, underscoring the fragility of indiscriminate data scaling. (3) Training regularization: we observe that intuitive strategies, such as sensory dropout and multi-stage fine-tuning, do not consistently improve performance at scale. Together, this study challenge some common assumptions about embodied scaling and provide practical guidance for training large-scale VLA policies from diverse robotic data. Project website: https://research.beingbeyond.com/rethink_vla

I. INTRODUCTION

This study examines whether scaling heterogeneous robotic data reliably improves VLA models, focusing on physical alignment, embodiment mixtures, and training regularization. It uses controlled ablations and a blinded real-robot evaluation protocol to assess these choices.

  • Heterogeneous robotic data differs in kinematics, joint limits, control frequencies, sensing modalities, and action spaces, making positive transfer from naive scaling uncertain.
  • The study asks which action representations align embodiments, when cross-robot data mixing helps or hurts, and whether regularization remains effective at scale.
  • The controlled study ablates physical alignment, embodiment mixture, and training regularization within a representative VLA framework combining a VLM backbone with flow matching.
  • The Grouped Blind Ensemble protocol blinds operators to model identity and separates policy execution from outcome judgment to reduce human bias in real-robot evaluation.
  • EEF-relative actions consistently outperform alternatives, indiscriminate cross-embodiment mixing often degrades performance, and sensory dropout or multi-stage curricula do not reliably improve scaling.

III. METHODOLOGY AND EVALUATION PROTOCOL

The methodology combines a Mixture-of-Transformers VLA with flow-matching control, a unified physically grounded action space, and a grouped blind evaluation protocol. These components support cross-embodiment training while preserving robot-specific control dimensions.

  • The testbed studies VLA scaling with a generalist architecture and a bias-resistant physical evaluation procedure.
  • The Mixture-of-Transformers uses parallel Semantic and Action Experts that exchange information through shared layerwise attention.
  • Flow matching learns a conditional vector field that transports Gaussian noise to short-horizon action chunks using multimodal context.
  • The unified action space is a superset of supported physical degrees of freedom, partitioned into semantically aligned end-effector, joint, gripper, hand, and auxiliary subspaces.
  • Robot-specific actions are embedded into shared semantic slots, while unused dimensions are disabled with binary masks.
  • The Grouped Blind Ensemble randomly partitions models into groups and anonymizes execution trials before returning aggregate statistics.

B. Grouped Blind Ensemble Evaluation

The Grouped Blind Ensemble protocol structures real-robot comparisons to reduce operator bias and fatigue. Its evaluation uses blinded policy execution with binary success/failure recording, supported by grouped trials.

  • Operators execute anonymized policies without access to model identities or versions and record binary success or failure.
  • Grouping reduces human preferences and provides structured breaks that help maintain consistent evaluation quality during large-scale studies.
  • Figure 2 presents the composition of the balanced pre-training data used in the study.

IV. PRE-TRAINING DATA AND IMPLEMENTATION

The paper constructs a heterogeneous corpus for VLA pre-training and summarizes its implementation settings.

  • The section describes the construction of the heterogeneous pre-training corpus and its key implementation settings.

A. Large-scale Heterogeneous Robot Data

The study constructs heterogeneous pretraining mixtures across real and simulated data, end-effector and joint-space control, and applies implementation choices designed for scalable VLA training.

  • Data composition: The pretraining collection spans real and simulated trajectories organized by domain and control space.The four source categories are real-world end-effector, real-world joint-space, simulation end-effector, and simulation joint-space data.
  • Sampling: Dynamic downsampling reduces the dominance of dense simulation streams and high-frequency real-world logs during training.Dataset-specific frame-step sizes are used to balance sampling across sources.
  • Model configuration: The model combines a 2B-parameter semantic expert initialized from InternVL-3.5-2B with a 0.7B action expert trained from scratch.The two experts use hidden sizes of 2048 and 1024, respectively.
  • Regularization: Regularization experiments mask proprioception and camera views with probability p = 0.2 and evaluate a two-stage training curriculum.The first curriculum stage freezes the VLM backbone while optimizing the action expert for 40k steps.

V. EXPERIMENTS

The experiments systematically test physical alignment, embodiment mixture, and training regularization through simulation benchmarks and real-robot deployments.

  • Experimental scope: The study evaluates whether coordinate frames, action representations, heterogeneous data aggregation, and regularization affect VLA control.These dimensions form the paper's three-part analysis of physical alignment, embodiment mixture, and training regularization.
  • Simulation benchmarks: Simulation evaluation uses LIBERO and RoboCasa with a single unified policy fine-tuned jointly across tasks.The protocol measures downstream transfer on standardized multi-task benchmarks.
  • Real-robot evaluation: Real-robot experiments use a Franka Panda equipped with two RGB cameras and evaluate four tasks probing complementary capabilities.The tasks include precision and long-horizon planning components.

A. Exploration of Physical Alignment

Physical alignment strongly affects cross-embodiment transfer: EEF-relative actions provide the most reliable gains, while world-frame and delta representations can transfer poorly or fail in real-world execution.

  • Frozen-backbone results: 75.1% average success and +8.2% improvement make EEF-Relative the strongest choice when the VLM backbone is frozen.This regime relies heavily on representations initialized during pretraining.
  • Real-world results: Real-world Delta actions achieve 0% success because they jitter in place, whereas Relative actions execute effectively.Among the two Relative representations, the real-world evaluation finds no significant performance difference.

B. Exploration of Embodiment Mixture

The study tests progressively richer heterogeneous pre-training mixtures and finds that adding diverse robot data often reduces transfer rather than improving it. Regularization and staged-training choices likewise provide no consistent gains at scale.

  • Embodiment Mixture: 77.3% average success on LIBERO is achieved by D1, while D2 and D3 fall to 73.8% and 72.1%, respectively.D4 recovers 3.0% but remains below the OXE-only baseline.
  • Embodiment Mixture: 54.7% on RoboCasa for D1 falls to 48.8% after adding heterogeneous real-robot EEF data in D2.Later mixtures only marginally recover performance, leaving D4 nearly 5% below D1 overall and 10.0% worse on Doors/Drawers.
  • Embodiment Mixture: Naively pooling structurally disparate robot datasets induces destructive interference rather than improved transfer.The cumulative inclusion protocol isolates the marginal contribution of OXE, real EEF, simulated EEF, and projected joint-space data.
  • Training Regularization: Disabling visual dropout reaches 85.6% success versus 84.5% for the balanced baseline, while heavy proprioceptive masking reduces performance.The results do not support a consistent benefit from stochastic modality dropout.
  • Training Regularization: Direct end-to-end fine-tuning in Stage 2 Only achieves the best average success at 85.8%, outperforming the multi-stage schedule.The authors describe this as evidence that the initialized action expert can co-adapt with the VLM backbone under joint training.

D. Comparison with Representative Generalist Policies.

The base model is competitive with representative generalist VLA policies under matched 50-shot fine-tuning, while qualitative rollouts cover four diverse real-world manipulation tasks. The evaluation emphasizes precision, long-horizon planning, dynamic contact, and tool manipulation.

  • Comparison with Representative Generalist Policies: 97.9% average success on LIBERO and 50.0% on RoboCasa are achieved by the base model under matched 50-shot fine-tuning.The benchmark uses representative generalist policies including π0, π0.5, and GR00T-N1.
  • Comparison with Representative Generalist Policies: The base model is competitive without task-specific tuning or specialized optimizations.These results support using the implementation as a representative foundation for the study’s experiments.
  • Qualitative Analysis: The real-world rollout set includes Stack Bowls, Pick-to-Drawer, Wipe Board, and Water Plant.The tasks probe precision, long-horizon planning, dynamic surface contact, and fine-grained tool manipulation, respectively.
  • Qualitative Analysis: Pick-to-Drawer is vulnerable to intermediate-stage failures because it requires opening, placing, and closing subtasks.The other tasks expose failures in consecutive stacking, incomplete wiping, and inaccurate spray-trigger pressing.
  • Conclusion: The study concludes that EEF-relative actions are effective for diverse kinematics, while indiscriminate data mixing and complex regularization do not reliably improve performance.The conclusion presents simple training recipes and careful data alignment as practical guidance within the evaluated scope.

APPENDIX

The appendix describes how the balanced pre-training corpus is counted, subsampled, and optimized before downstream fine-tuning. It reports approximately 182.4 million effective frames and specifies the main training schedules.

  • Data Mixture Statistics: The effective frame count is computed as Neff = ⌊Nraw/S⌋ from raw frame count Nraw and sampling step size S.This calculation is used to characterize the balanced pre-training corpus.
  • Data Mixture Statistics: The final balanced dataset contains approximately 182.4 million frames.Larger step sizes are assigned to massive Agibot and InternData datasets to balance them with OXE subsets.
  • Training Setup: Pre-training uses two stages: 40,000 frozen-backbone steps followed by 200,000 full-model steps.The learning rates are 1 × 10^-4 and 2 × 10^-5, respectively, with global batch size 256 throughout.
  • Training Setup: Downstream fine-tuning updates the full model with global batch size 128 and learning rate 1 × 10^-4.Training lasts 30,000 steps for LIBERO 5-shot and 60,000 steps for full LIBERO and RoboCasa.

C. Real-World Task Details

The real-world evaluation uses four multi-stage manipulation tasks on a Franka Panda with two RGB cameras. Each task targets a distinct capability and defines success through sequential subgoals.

  • Evaluation Setup: The evaluation uses a Franka Panda with two RGB cameras, and each task is tested in 10 trials with randomized object initialization.Success is recorded using multi-stage criteria.
  • Stack Bowls: Stack Bowls requires sequentially grasping and stacking three bowls into a target plate.Its main difficulty is precise consecutive grasping and stacking.
  • Pick-to-Drawer: Pick-to-Drawer requires opening a drawer, placing three objects inside, and closing it.Because it is long-horizon and involves articulated-object manipulation, failures often occur during intermediate stages.
  • Wipe Board: Wipe Board requires grasping a sponge, contacting the board, executing continuous wiping, and returning the sponge.A common failure is leaving the surface incompletely clean.
  • Water Plant: Water Plant requires grasping and orienting a spray bottle, with failures including knocking it over or missing the trigger.The task probes fine-grained tool manipulation and functional orientation.
Loading 2602.09722v1…