Source-linked AI summary
ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulation
Mi Yan, Wenhao Zhang, Zhiqi Zhang, Yu Peng, Tangxinyu Wang, Lingfei Zhai, Jiayi Su, Shengliang Deng, Lin Peng, Yaowei Liu, Yuxing Chen, Zhiyuan Wei, Jilong Wang, Jiayi Chen, Jiangran Lyu, Zhizheng Zhang, He Wang
TL;DR
The paper addresses limited and inconsistently evaluated evidence about zero-shot VLA transfer across robot embodiments. It defines separate strict and pretrain-exposed protocols, builds a controlled benchmark, and studies four transfer factors. Local EEF representations, source diversity, auxiliary co-training, and small amounts of target pretraining exposure improve transfer within stationary tabletop manipulation.
Problem
Systematic understanding of zero-shot cross-embodiment VLA transfer is limited because definitions and evaluations do not consistently isolate embodiment changes.
Method
The paper separates strict and pretrain-exposed protocols and evaluates four transfer factors in controlled simulation and real-world tabletop manipulation benchmarks.
Results
Local EEF representations, source diversity, and auxiliary co-training improve transfer by around 15, 18, and 7 percentage points, while 5% target exposure improves average progress by 13.4 percentage points.
Takeaways & Limitations
Strict and pretrain-exposed zero-shot transfer should be reported separately, with local EEF representations, controlled source diversity, and auxiliary co-training providing practical guidance in the studied setting.
Takeaways & Limitations
The study is limited to stationary tabletop manipulation with representative two-finger grippers and relatively short-horizon tasks, excluding mobile manipulation, dexterous hands, and long-horizon settings.
Abstract
from arXiv · showhide
Zero-shot generalization to unseen embodiments is important for generalizable vision-language-action (VLA) models as robot hardware evolves and task-specific data collection remains costly. However, a systematic understanding of this problem remains limited, in part because the literature lacks a unified zero-shot transfer definition and controlled evaluation settings that isolate embodiment changes from differences in tasks, scenes, or protocols. To address this gap, we first distinguish strict zero-shot transfer, where the target embodiment is absent from all training data, from pretrain-exposed zero-shot transfer, where it appears only during pretraining. We then introduce a controlled benchmark spanning 14 held-out target embodiments across simulation and real-world validation. Within this framework, we conduct a controlled analysis of four factors: state-action representations, pretraining embodiment diversity, auxiliary co-training objectives, and target-embodiment exposure. Experimental results show that local end-effector (EEF) state-action representations, the source embodiment diversity, and auxiliary co-training improve cross-embodiment transfer by around 15, 18, and 7 percentage points, respectively. We further find that adding only 5% target-embodiment data during pretraining improves average target-embodiment progress by 13.4 percentage points, showing that strict and pretrain-exposed zero-shot transfer are distinct and should be reported separately. Together, these findings provide practical guidance for evaluating and improving cross-embodiment VLA transfer in stationary tabletop manipulation with two-finger grippers, while motivating future investigation of broader settings including mobile-base control, dexterous hands, and long-horizon tasks.
1 Introduction
The paper clarifies zero-shot cross-embodiment protocols and evaluates transfer in a controlled tabletop setting. Its experiments identify representation, source diversity, co-training, and target exposure as important factors.
- Motivation: Zero-shot transfer is inconsistently defined because studies differ in target-embodiment exposure and often confound embodiment shifts with task, scene, camera, data, or protocol changes.The paper focuses on stationary tabletop manipulation with two-finger grippers to isolate embodiment differences.
- Protocol clarification: Strict zero-shot transfer excludes the target embodiment from all training, whereas pretrain-exposed zero-shot transfer includes it during pretraining but not post-training.The distinction represents different deployment scenarios and is treated as an explicit experimental variable.
- Controlled benchmark: The benchmark evaluates seven held-out target embodiments in simulation across appearance-only, gripper-only, arm-only, and full-embodiment shifts, with matched real-world validation.Each simulation comparison uses 6,300 rollouts per model under experiment-specific budget controls.
- Factorized study: Around 15 percentage points of average strict-transfer improvement comes from local EEF-centered state-action representations.The study evaluates this factor among four research questions spanning representations, source diversity, co-training, and target exposure.
- Factorized study: Around 18 percentage points separate the 512-source and single-source models under a fixed 640K-trajectory pretraining budget, while auxiliary co-training raises average progress from 75.7% to 82.3%.These results quantify effects of source-embodiment diversity and auxiliary objectives.
- Target exposure: Adding 5% target-embodiment data during pretraining improves average target-embodiment progress by 13.4 percentage points, supporting separate reporting of the two protocols.Real-world validation shows the same overall trends for representation and source-diversity questions.
2 Related Work
Related work expands multi-embodiment data, shared behavior representations, and cross-embodiment benchmarks. The paper positions its contribution around inconsistent zero-shot definitions and protocol-aware reclassification.
- Multi-embodiment data: Large robot foundation models use heterogeneous robot data from federated, in-the-wild, skill-rich, standardized, dual-arm, unified, and synthetic sources.These datasets broaden embodiment and manipulation coverage for pretraining.
- Representations and adaptation: Cross-embodiment policies represent behavior through shared skill or action spaces, including human videos, image-space tracks, universal action codes, and task-centric latent actions.Another line conditions shared backbones on embodiment-specific information through interfaces or adaptation modules.
- Benchmarks: Existing benchmarks study cross-embodiment manipulation and navigation, while RoboMIND and AnyBody broaden multi-embodiment data and benchmark coverage.Recent studies also analyze how embodiment diversity and scaling affect robotic learning.
- Protocol taxonomy: Table 1 reclassifies representative evaluations described as zero-shot or out-of-the-box according to target-embodiment exposure.The classification distinguishes evaluations with unseen embodiments from those with unseen task–embodiment pairs.
- Protocol taxonomy: Prior evaluations use “zero-shot” for different protocols, including cases where targets appear during pretraining or post-training, limiting direct comparison.The paper argues that these settings should be interpreted separately.
3 Problem Formulation and Preliminaries
The paper formalizes two-stage cross-embodiment transfer and separates strict from pretrain-exposed zero-shot evaluation. It also categorizes embodiment shifts by which robot factors change.
- Notation: Embodiments encode robot-dependent appearance, morphology, and kinematics, while D(e, t) denotes demonstrations for an embodiment–task pair.The notation aggregates demonstrations across embodiment and task sets using D(E′, T′).
- Transfer setup: A policy is pretrained on Dpre, post-trained using only source embodiment es, and evaluated on a different target embodiment et for the same downstream tasks.This setup isolates transfer across embodiments after source-specific post-training.
- Zero-shot protocols: Strict zero-shot transfer requires et not to appear in pretraining, whereas pretrain-exposed transfer permits pretraining exposure but excludes et from post-training.The distinction is necessary because prior “zero-shot” evaluations span different exposure protocols.
- Embodiment shift taxonomy: Appearance-only shifts change visual appearance, gripper-only shifts change end-effector geometry, arm-only shifts change arm morphology or kinematics, and full-embodiment shifts change both.These categories are instantiated in simulation and real-world settings.
4 Study Design
The study controls embodiment transfer by using fixed-dimensional Cartesian end-effector representations, separated pretraining and post-training, and procedurally generated source embodiments with held-out targets. It evaluates representation choices and transfer across controlled embodiment shifts while matching non-embodiment factors.
- Representation: Fixed-dimensional Cartesian end-effector representations are used because joint-indexed interfaces may differ in dimensionality, limits, and kinematic semantics across unseen robots.
- Representation: The study compares two action frames and two state encodings, producing four state-action representation combinations.
- Training protocol: Pretraining and post-training use disjoint task sets, with primary comparisons fixing pretraining at 640K trajectories and post-training on source-embodiment data.
- Data construction: The simulation dataset contains 512 procedurally generated Franka-family source embodiments, while seven commercial test robots are held out from the source pool.
- Evaluation: Evaluation covers appearance-only, gripper-only, arm-only, and full-embodiment shifts, with matched cameras, environments, and tasks to isolate embodiment changes.
- Evaluation: Table 3 reports that EEF-Delta states improve arm-only and full-embodiment transfer, while EEF-Delta actions improve average score.
5 Controlled Experiments
The controlled experiments isolate representation, source diversity, co-training, and target exposure while aligning other evaluation factors. Results show that local EEF representations, broader source coverage, auxiliary objectives, and limited target exposure improve transfer, though budget allocation affects diversity gains.
- RQ1: State-Action Representations: Local EEF-centered state-action representations improve strict zero-shot transfer by around 15 percentage points on average.EEF-Delta state with EEF-Delta actions achieves the strongest real-world average score, reaching 89.9%.
- RQ2: Source-Embodiment Diversity: The 512-source setting outperforms the single-source setting under a fixed 640K-trajectory pretraining budget.The largest gains occur for gripper-only, arm-only, and full-embodiment shifts, while attention becomes more concentrated on task-relevant objects and the gripper.
- RQ2: Source-Embodiment Diversity: Source diversity improves transfer over the evaluated 32–512 range when the per-embodiment budget is fixed.Under a fixed total budget, increasing source count reduces per-embodiment data density and produces a non-monotonic real-world trend.
- RQ3: Auxiliary Co-training: Auxiliary co-training raises average progress from 75.7% to 77.7–82.3%, with gains concentrated on larger embodiment shifts.The EEF-only language-action objective outperforms the mixed-frame LAP variant.
- RQ4: Target-Embodiment Exposure: Adding only 5% target-embodiment data during pretraining improves average target-embodiment progress by 13.4 percentage points.Target exposure reduces but does not eliminate the transfer gap and narrows representation-performance differences.
- Practical Guidance: The study recommends reporting target exposure explicitly and treating representation, source diversity, budget allocation, and co-training as transfer-design variables.The recommendations apply within the studied stationary tabletop manipulation setting.
6 Limitations and Conclusion
The paper concludes that controlled protocol separation and factorized evaluation clarify cross-embodiment VLA transfer in stationary tabletop manipulation. Its scope is limited to two-finger-gripper manipulators and relatively short-horizon tasks, motivating broader validation.
- Limitations: The evaluation focuses on stationary tabletop manipulation with representative two-finger-gripper manipulators and relatively short-horizon tasks.Mobile manipulation, dexterous hands, long-horizon tasks, and broader deployment components remain outside the study’s scope.
- Conclusion: The paper separates strict and pretrain-exposed zero-shot transfer and evaluates appearance, gripper, arm, and full-embodiment shifts.Table 7 formalizes the protocols by whether the target embodiment appears in pretraining.
- Conclusion: The reported guidance supports local EEF-centered representations, controlled source-diversity evaluation, auxiliary co-training, and explicit target-exposure reporting.These recommendations are stated for the controlled setting studied in the paper.
B Per-RQ Experimental Design Matrix
The experimental design matrix makes explicit which protocol, pretraining data, representation, and evaluation platform isolate each research question. Across comparisons, downstream data and core training conditions are held consistent unless the factor under study changes.
- Per-RQ Design Matrix: The matrix specifies the isolated factor, zero-shot protocol, pretraining data, and evaluation platform for each research question.It is intended to make the controlled comparisons explicit.
- Per-RQ Design Matrix: All research questions use the same source-embodiment downstream data while keeping the backbone, visual inputs, action horizon, and optimization schedule fixed.The design therefore distinguishes the experimental variable from shared training conditions.
C Detailed Experimental Results
The detailed results tables expand the aggregate simulation findings into per-task and per-embodiment progress scores. They report the held-out targets used in simulation and organize real-world results by task, target embodiment, and controlled variable.
- Detailed Simulation Results: Simulation tables report per-task, per-embodiment progress scores across the source embodiment and seven held-out target embodiments.Unless otherwise stated, entries contain three simulation repeats as run-1/run-2/run-3 progress.
- Detailed Real-World Results: Real-world tables report completed RQ1 and RQ2 runs for open the fryer and water the flower.Each task has a separate table, with target embodiments as rows and the controlled variable as columns.
D Data-Construction Details
The study constructs procedurally varied tabletop embodiments and controls visual, geometric, camera, and evaluation factors to support cross-embodiment analysis.
- Simulation embodiment generation: 512 Franka-style source embodiments vary arm geometry, end-effector morphology, and visual appearance through procedural generation.Arm-link scale factors span approximately 0.7–1.3, with corresponding visual and collision-mesh deformation.
- Evaluation coverage: The evaluation reports task-progress results for three simulation tasks and two real-world tasks across representation, diversity, co-training, and exposure studies.Detailed tables cover serving sausages, stacking bowls, stacking cubes, opening an air fryer, and watering a plant.
- Target embodiment taxonomy: The benchmark includes appearance-only, gripper-only, arm-only, and full-embodiment target shifts.Examples include altered robot textures, UMI grippers, UR5e arms, and combined UR5e–UMI changes.
- Evaluation alignment: Camera calibration and a shared tabletop task frame reduce embodiment-dependent visual and coordinate discrepancies.Object poses, goal poses, and task-relevant spatial quantities are expressed in an environment-anchored frame fixed to the tabletop workspace.
F.4 Details of Metrics
Metrics assign partial or full credit to rollouts and aggregate results so tasks and embodiment-shift categories contribute uniformly.
- Progress scoring: Progress scores are s ∈ {0, 0.5, 1}, representing failure, interaction-subgoal completion, and full task completion.Reported progress is 100s averaged over rollouts.
- Aggregation: Macro-averaging gives each downstream task and embodiment-shift category equal weight in the aggregate score.Per-robot task means are averaged within categories, then the four category scores are averaged.
- Data protocol: Real-world evaluation uses two tasks with 50 demonstrations each, while simulation uses three tasks with 40K trajectories per task.Downstream post-training data is collected on the same source Franka arm for fair comparison.
G Auxiliary Co-Training Objective Specifications
Auxiliary co-training augments imitation learning with language-action, subgoal, and task-conditioned visual objectives using shared observations and controlled sampling.
- Training mixture: Auxiliary variants combine imitation-action examples and auxiliary-query examples with sampling weights of 0.75 and 0.25.The imitation branch uses flow-matching action loss, while auxiliary responses use next-token cross entropy.
- Language-action objective: Language-action targets are generated from delta action vectors and rendered as clauses for translation, rotation, and gripper commands.EEF-frame actions use centimeter and degree units, integer rounding, and ordered directional clauses.
- Subgoal prediction: The subgoal-prediction objective encodes the next manipulation-relevant waypoint in the current EEF coordinate frame.Translation values use 0.001 precision and rotations use 0.01 precision.
- Bounding-box objective: The task-conditioned bounding-box objective predicts one relevant object box per input view using normalized XYXY coordinates and quantized tokens.The task-conditioned wording targets the object relevant to the instruction rather than generic object boxes.
- Target exposure: Target-embodiment exposure replaces a controlled fraction of non-target source data in separate 640K-trajectory pretraining pools.UR5eUMI and GoogleRobot represent the exposed-target setting, while target post-training oracles provide upper references rather than zero-shot results.
I Model Training Details
The policy receives language, dual-view images, and EEF state history, then predicts 12-step action chunks under controlled state-action interfaces.
- Policy architecture: The Mixture-of-Transformers backbone couples a pretrained vision-language stream with a separate action expert through layer-wise attention.The policy uses 4-step proprioceptive history and predicts a 12-step action chunk.
- Inference inputs: Each inference request contains a language instruction, wrist and third-person RGB observations, and a four-entry EEF state history.Images are resized to 224 × 224.
- Action outputs: The response is a 12-step action chunk whose per-step action combines a six-dimensional pose delta with a gripper command.The gripper output is quantized to close, no-op, or open.
- Serving details: The server batches up to 10 requests, with one warmed-up query taking 168 ms on a single NVIDIA GeForce RTX 4090 GPU.The serving setup uses 6168 MiB of GPU memory.
- Representation variants: The controlled representation study crosses two action frames with two state encodings to form four state-action variants.The compared frames are World-Delta and EEF-Delta, while state uses absolute EEF or EEF-Delta encodings.