Source-linked AI summary
SkillVLA: Tackling Combinatorial Diversity in Dual-Arm Manipulation via Skill Reuse
Xuanran Zhai, Zekai Huang, Longyan Wu, Qianyou Zhao, Qiaojun Yu, Jieji Ren, Ce Hao, Harold Soh
TL;DR
Bimanual VLAs face combinatorial diversity because distinct left-right skill pairings create different tasks, yet common designs do not explicitly support recombination. SkillVLA separates per-arm skill invocation while retaining controlled inter-arm communication, reaching 51% success on unseen combinations where strong baselines achieve near-zero success.
Problem
Common bimanual VLA formulations overlook combinatorial diversity: distinct left-right skill pairings create different tasks, but concatenated-action policies are confined to pairings observed in demonstrations.
Source: §I · p. 2Method
SkillVLA identifies skills in demonstrations, invokes single-arm skills independently for recomposition, and captures inter-arm dependence when behaviors require collaboration.
Source: §I · p. 2Results
SkillVLA achieves 51% success on unseen combinations, while strong bimanual VLA baselines achieve near-zero success on unseen tasks composed of learned skills.
Source: §I · p. 2, A. Single-Arm Skill Reuse · p. 7Takeaways & Limitations
Across real-world evaluations, SkillVLA supports more reliable reuse of single-arm and cooperative dual-arm behaviors and improves data efficiency in continual skill acquisition.
Source: §VII · p. 8SkillVLA depends on pretrained VLMs for high-level reasoning and cooperation priors, tying performance to their capability, latency, and cost.
Source: §VII · p. 9
Abstract
from arXiv · showhide
Recent progress in vision-language-action (VLA) models has demonstrated strong potential for dual-arm manipulation, enabling complex behaviors and generalization to unseen environments. However, mainstream bimanual VLA formulations largely overlook the critical challenge of combinatorial diversity. Different pairings of single-arm behaviors can induce qualitatively distinct task behaviors, yet existing models do not explicitly account for this structure. We argue that effective bimanual VLAs should support skill reuse - the ability to recombine previously learned single-arm skills across novel left-right pairings - thereby avoiding the need to separately learn every possible combination. Current VLA designs entangle skills across arms, preventing such recomposition and limiting scalability. To address this limitation, we propose SkillVLA, a framework explicitly designed to enable skill reuse in dual-arm manipulation. Extensive experiments demonstrate that SkillVLA substantially improves skill composition, increasing overall success rate from 0% to 51%, and achieves strong performance on cooperative and long-horizon tasks.
I. INTRODUCTION
Bimanual VLAs face combinatorial diversity because different left-right skill pairings create distinct tasks, while common designs limit recomposition to pairings seen in demonstrations. SkillVLA targets skill reuse, with real-robot results showing improved generalization and strong performance on cooperative and long-horizon tasks.
I. INTRODUCTION
- Different left-right skill pairings create distinct bimanual tasks, and their number grows quadratically as the underlying skill set expands.Concatenated-action VLA formulations learn joint distributions from paired demonstrations, limiting their outputs to pairings observed during training.Source: §I · p. 1
- Skill reuse means discovering single-arm skills in demonstrations and recombining them across left-right pairings not explicitly observed during training.The paper presents this flexibility as a foundation for broader generalization in bimanual settings.Source: §I · p. 2
- SkillVLA reaches 51% success on unseen tasks composed of learned skills, while strong bimanual VLA baselines achieve near-zero success.The real-robot evaluations also report baseline-matching performance on three highly cooperative tasks and a 21% execution-time reduction on two long-horizon tasks.Source: §I · p. 2
II. RELATED WORK
- Prior dual-arm work includes methods specialized to task families, richer-actuation platforms, and policies trained from human demonstration videos.The introduction also situates recent VLA models as enabling generalist bimanual policies.Source: §I · p. 1, §II · p. 2
III. PROBLEM FORMULATION
- The problem formulation seeks a bimanual policy that reuses learned single-arm skills and skill pairings alongside genuinely dual-arm skills.The formulation distinguishes the single-arm skill sets and the paired single-arm skills present in demonstrations.Source: §III · p. 2
IV. SKILL REUSE IN DUAL-ARM MANIPULATION
The paper defines skill reuse around selecting appropriate skills for a scene and generating their actions, including novel pairings of single-arm skills. It distinguishes those compositions from genuinely dual-arm skills that require inter-arm coordination.
A. Definitions
- A single-arm skill is a conditional action distribution for one arm, while a dual-arm skill is a joint distribution with dependent arm actions.The paper characterizes dual-arm dependence using positive conditional mutual information and describes an inter-arm message pathway for coordination.Source: A. Definitions · p. 3
- Skill reuse requires realizing appropriate single-arm skill pairings even when that pairing never appeared in demonstrations.The set of correct single-arm pairings is defined over the full product of left- and right-arm skill sets, not only demonstrated pairs.Source: A. Definitions · p. 3
IV. SKILL REUSE IN DUAL-ARM MANIPULATION
- When a scene calls for a genuinely dual-arm skill, the policy must reproduce that skill while maintaining inter-arm coordination.The paper treats skill selection and correct action generation as separate requirements for reuse.Source: A. Definitions · p. 3, §IV · p. 3
A. Definitions
- Reusing per-arm behaviors can make acquiring a new bimanual skill require mainly learning additional coupling information rather than relearning both arms.The remark presents this as a way to improve efficiency in continual or large-scale learning.Source: A. Definitions · p. 3
B. Skill Selector
The skill selector chooses a single-arm skill pair or a dual-arm skill for a scene. The resulting action distribution mixes the selected skills’ action distributions.
B. Skill Selector
- Given a finite skill library, action generation first selects a skill or skill pair for the scene, then samples actions from the corresponding distributions.The ideal high-level selector defines p*(Y | x), and the resulting action distribution is a mixture.Source: B. Skill Selector · p. 3
- The selector maps a scene to a skill index, choosing either a dual-arm skill or a concurrent left-right single-arm skill pair.The ideal selector should also choose appropriate configurations for novel inputs where the correct configuration was never observed.Source: B. Skill Selector · p. 3
C. Skill Entanglement in Current VLAs
Conventional bimanual VLAs can entangle arm skills in joint actions and shared latent representations, limiting recomposition of unseen left–right pairings. SkillVLA uses per-arm skill descriptions and action streams, with gated cross-attention for behaviors requiring cooperation.
C. Skill Entanglement in Current VLAs
- Joint-action supervision can entangle arms in correlations from paired demonstrations, limiting recomposition beyond trained left–right pairings.Shared latent representations can also mix cross-arm dependencies before action generation.Source: C. Skill Entanglement in Current VLAs · p. 4
A. Method Pipeline
- SkillVLA uses a two-level pipeline in which a high-level VLM produces per-arm natural-language skill descriptors for flexible recomposition.The high-level VLM is frozen during low-level skill learning to preserve its vision–language generalization.Source: A. Method Pipeline · p. 4, §V · p. 4
- Separate low-level VLM streams generate per-arm actions, while adaptive cross-attention between action experts captures inter-arm dependence when needed.The cooperation signal gates the inter-arm message to support skill-adaptive action generation.Source: A. Method Pipeline · p. 4, Figure 2 · p. 5
- A cooperation estimator uses the high-level VLM representation to identify whether behavior is a composition of single-arm skills or a cooperative dual-arm skill.Its scalar α indicates cooperation degree and gates message passing between arms.Source: A. Method Pipeline · pp. 4–5
V. METHOD: SKILLVLA
- The cooperation objective raises α when cross-arm communication reduces behavioral-cloning error and lowers it otherwise.The method also introduces training strategies to help infer inter-arm cooperation.Source: §V · p. 4, A. Method Pipeline · p. 5
B. Additional Cooperation-Level Learning
SkillVLA regularizes its cooperation gate using VLM-derived priors and encourages temporally sticky, conservative communication; an optional binary gate is used to improve stability.
B. Additional Cooperation-Level Learning
- A VLM prior estimates task-dependent coordination strength as α_vlm in [0, 1], or as {0, 1} for a discrete gate.The prior is produced for the current scene and task.Source: B. Additional Cooperation-Level Learning · p. 5
- The communication gate is regularized toward the VLM prior, with additional penalties encouraging temporal smoothness and conservative communication.The gate is kept small unless interaction is beneficial.Source: B. Additional Cooperation-Level Learning · p. 5
- Optional gate discretization restricts α_t to {0, 1} to address small persistent fluctuations that can destabilize action generation.The model predicts a communication probability, binarizes it at inference, and this tokenized formulation improved stability in preliminary experiments.Source: B. Additional Cooperation-Level Learning · p. 5
VI. EXPERIMENTS
The experiments are designed to assess skill recomposition, cooperative coordination, cooperation-level estimation on long-horizon tasks, and continual learning. The supplied setup describes recomposition tests and evaluation baselines and metrics.
VI. EXPERIMENTS
- The evaluation targets skill reuse, inter-arm coordination, cooperation estimation across multi-stage tasks, and continual learning.The experiments ask whether learned single-arm skills recombine on unseen pairings and whether the model supports cooperative and long-horizon behaviors.Source: §VI · pp. 5–6
- Recomposition training uses three skills per arm, followed by zero-shot tests over possible left–right skill combinations.The task setup specifies that test combinations never appear in training.Source: Figure 3 · p. 6
- SkillVLA is compared with π0.5, with π0-FAST additionally included for recomposition and TwinVLA as a factorized baseline.Success rate is the primary metric for most tasks; long-horizon tasks also report progress score and average completion time.Source: §VI · p. 6
A. Single-Arm Skill Reuse
SkillVLA recomposes unseen left–right skill pairs more successfully than the baselines, while cooperative-task ablations show communication matters for close inter-arm coordination.
A. Single-Arm Skill Reuse
- Training uses three single-arm skills per arm, then tests zero-shot on unseen simultaneous left–right pairings.All methods perform comparably on seen single-arm skills, but diverge on unseen combinations.Source: A. Single-Arm Skill Reuse · p. 6
- 51% success on unseen combinations for SkillVLA contrasts with 0% for π0.5 and π0-FAST.The test pairs left- and right-arm skills that never appeared together in training.Source: A. Single-Arm Skill Reuse · pp. 6–7
- SkillVLA separates per-arm skill invocation, adapts action generation, and controls inter-arm communication to reduce spurious interference on novel pairings.The baselines’ failures are associated with entangled bimanual control; π0.5 often reverts to one active arm and one idle arm.Source: A. Single-Arm Skill Reuse · pp. 6–7
B. Dual-Arm (Cooperative) Skill Reproduction
- Cooperative tasks test tight low-level alignment in SHAKE and BALL, and both low-level alignment and high-level coordination in ALIGN.ALIGN randomizes arm-object assignment across trials.Source: B. Dual-Arm (Cooperative) Skill… · p. 7, Figure 4 · p. 7
- Disabling cross-attention particularly hurts SHAKE and BALL, while ALIGN shows a smaller performance drop.The ablation indicates that communication is important for tight low-level alignment, whereas ALIGN is comparatively less sensitive to continuous low-level coupling.Source: B. Dual-Arm (Cooperative) Skill… · p. 7
C. Applications in Long-Horizon Settings
In long-horizon tasks, SkillVLA switches between independent and cooperative modes and reduces completion time; continual-learning experiments also show rapid adaptation from learned skills. Its high-level reasoning depends on pretrained VLMs.
C. Applications in Long-Horizon Settings
- Long-horizon tasks interleave independent per-arm actions with cooperative phases; TUBES combines tube transfers with moving a loaded rack using both arms.The policy is evaluated zero-shot on the composed task after training on the three component skills.Source: C. Applications in Long-Horizon Settings · p. 7
- ≈21% lower average completion time than baselines, with comparable progress scores, on long-horizon tasks.In TUBES, SkillVLA concurrently transfers tubes with both arms during the independent collection phase.Source: C. Applications in Long-Horizon Settings · p. 7, Figure 5 · p. 8
- The cooperation gate uses α = 0 for independent execution and α = 1 for cooperation, switching with task phase.In TUBES it activates cooperation to move the loaded rack; continuous, non-discretized α fluctuates more and is associated with lower success and higher performance variance.Source: C. Applications in Long-Horizon Settings · pp. 7–8, Figure 5 · p. 8
D. Reuse of Learned Skills in Continual Learning
- In LIFT BAG, SkillVLA reaches 20% zero-shot success and near-peak performance after only 5 fine-tuning demonstrations.π0.5 and TwinVLA remain far from convergence after 30 demonstrations.Source: D. Reuse of Learned Skills in… · p. 8, Figure 6 · p. 8
- For SHAKE CUP, SkillVLA transfers related single-arm behaviors to a cooperative policy despite the new stabilization requirement.One hand must stabilize the cap rather than simply shake the cup body.Source: D. Reuse of Learned Skills in… · p. 8
VII. CONCLUSION, LIMITATIONS, AND FUTURE WORK
- Across real-world evaluations, SkillVLA improves combinatorial generalization and continual-learning data efficiency, supporting reuse of single-arm and cooperative dual-arm behaviors.Its high-level reasoning and cooperation priors depend on pretrained VLMs, tying performance to their capability, latency, and cost.Source: §VII · pp. 8–9
APPENDIX A METHOD ANALYSIS
The appendix analyzes why monolithic bimanual policies struggle with unseen skill combinations: both joint-action modeling and shared latent reasoning can entangle the arms.
APPENDIX A METHOD ANALYSIS
- The appendix uses effective support to distinguish demonstration-covered actions from actions needed for novel recomposition.A learned policy generalizes to a new context only if its predicted support covers the target support.Source: Appendix A · p. 13
A. Action Entanglement in Monolithic VLAs
- Action Entanglement names the structural disconnect between a monolithic joint-action policy’s training manifold and the target space for skill recomposition.Demonstration data can encode arm correlations, such as a left-arm lift occurring only while the right arm idles.Source: A. Action Entanglement in Monolithic… · p. 13
- When target actions lie outside the training support, a monolithic policy tends to return to high-density training modes, producing cross-arm interference.This can make one arm’s execution pull the other toward its training-time partner behavior.Source: A. Action Entanglement in Monolithic… · p. 13
B. Latent Entanglement in Action-Expert VLAs
- Separate action experts remain vulnerable because a shared VLM latent can place unseen recompositions outside the experts’ training support.The experts are optimized for latents from the training distribution and may behave unpredictably on out-of-distribution conditioning latents.Source: B. Latent Entanglement in… · pp. 13–14
- Output factorization alone is insufficient; reliable combinatorial generalization also requires reasoning to decompose context into per-arm skill descriptors.A single shared latent can encode observed cross-arm dependencies before signals reach otherwise separate action experts.Source: B. Latent Entanglement in… · pp. 13–14
C. Analysis of SkillVLA
SkillVLA addresses action and latent entanglement by factoring reasoning across arms, supporting recomposition of known single-arm skills into unseen pairs. Its product of per-arm action supports covers the target support when the experts recover the component skills.
C. Analysis of SkillVLA
- Action and latent entanglement are identified as barriers because the training manifold mismatches the target space of unseen tasks.The analysis examines how SkillVLA’s generative process supports single-arm skill recomposition where monolithic baselines fail.Source: C. Analysis of SkillVLA · p. 14
- With the collaboration gate closed, the joint action distribution factors into independent per-arm action experts, per-arm VLMs, and a high-level VLM.The high-level VLM supplies the skill descriptors; the per-arm components process them independently.Source: C. Analysis of SkillVLA · p. 14
- The high-level planner decomposes a novel context into left- and right-arm skill descriptors that individually match skills in the training repertoire.The novel pairing is decomposed into recognizable semantic units.Source: C. Analysis of SkillVLA · p. 14
- Independent latent generation maps each known skill descriptor to an in-distribution latent, avoiding the undefined regions associated with latent entanglement.The analysis contrasts this with shared-latent models encoding the novel combination in a single vector.Source: C. Analysis of SkillVLA · p. 14
- SkillVLA’s product of per-arm action supports covers the target support for unseen skill pairs when the experts recover the known component skills.The analysis describes this as a mechanism for zero-shot skill recomposition.Source: C. Analysis of SkillVLA · p. 14
APPENDIX B IMPLEMENTATION DETAILS
The appendix describes SkillVLA’s implementation, its comparison baselines, and the protocol for evaluating zero-shot skill recomposition.
APPENDIX B IMPLEMENTATION DETAILS
- The implementation makes minimal structural changes to π0.5 [23] to isolate structural limitations while keeping components comparable.Pretrained PaliGemma checkpoints initialize the components; the high-level VLM is lightly fine-tuned for subtask generation and then frozen, while low-level VLMs receive independent LoRA adapters [20].Source: Appendix B · p. 14
- A cross-attention module connects the action experts, while a Transformer decoder estimates cooperation from the high-level VLM’s KV cache.The cooperation labels used as learning priors are annotated from training data with frozen Qwen3-VL-32B [3].Source: Appendix B · p. 14
A. Baselines
- π0-FAST [34] and π0.5 [23] represent autoregressive and diffusion-based baselines, respectively; TwinVLA adds a factorized baseline with cross-attention between parallel VLM streams.The appendix also identifies Table IV as cooperative task details and Fig. 7 as a COLLECT ITEMS rollout.Source: A. Baselines · p. 15, Table IV · p. 15, Figure 7 · p. 15
1) Skill Recomposition Tasks:
- Skill recomposition evaluation trains on independent single-arm skills with 50 demonstrations each, then tests unseen pairings in 10 episodes per pairing.The protocol also checks each source skill in 10 rollouts before testing zero-shot generalization.Source: 1) Skill Recomposition Tasks · p. 15
2) Cooperative Tasks:
The evaluation protocols cover cooperative manipulation, long-horizon multi-stage tasks, and continual learning, using task-specific training and repeated rollouts.
2) Cooperative Tasks:
- Three cooperative tasks—SHAKE CUP, LIFT BALL, and ALIGN BLOCKS—probe inter-arm communication, with 50 demonstrations per task and 20 evaluation rollouts per model.The tasks require strict bimanual coupling, from synchronized low-level actuation to high-level planning.Source: 2) Cooperative Tasks · p. 15
3) Long-Horizon Tasks:
- For TUBES and COLLECT ITEMS, task-specific policies use 50 demonstrations per required skill and are evaluated over 10 episodes each with randomized initial object placements.The evaluation uses the same items and environment across trials; Fig. 8 and Fig. 9 show skills learned for TUBES and COLLECT ITEMS.Source: 3) Long-Horizon Tasks · p. 15, Figure 8 · p. 16, Figure 9 · p. 16
- Long-horizon performance is measured by average progress rate, with progress defined as episode score divided by the task’s maximum score.TUBES involves transferring tubes and placing a loaded rack; COLLECT ITEMS involves opening and closing a drawer, loading items, and relocating an initially unreachable item between arms.Source: 3) Long-Horizon Tasks · pp. 15–16
- Progress-normalized completion time is averaged only over episodes with non-zero progress, using execution time and episode progress rate.The protocol defines the eligible episode set as those achieving non-zero progress.Source: 3) Long-Horizon Tasks · p. 16
4) Continual-Learning Tasks:
- Continual-learning evaluation first pretrains on fundamental single-arm skills with 50 demonstrations per skill, then fine-tunes on dual-arm tasks with limited demonstrations and evaluates over 20 rollouts after each stage.Source: 4) Continual-Learning Tasks · p. 16