PaperlayerView exact revision on arXiv

Source-linked AI summary

Arm-wise Compositional Generalization in Dual-Arm Vision-Language-Action Models

Zaibin Zhang, Binghao Ran, Yuhan Wu, Zhongbo Zhang, Yifan Wang, Junwei Jiang, Junlan Xiao, Wangcheng Shi, Li Kang, Yiran Qin, Zhenfei Yin, Lijun Wang, Huchuan Lu

arXiv:2610.06184v1cs.RO

TL;DR

Dual-arm models can perform familiar tasks, but this does not establish whether they can recombine component skills under unseen coordination requirements. The paper introduces ACG-Bench and compares training and architectural strategies with a shared π0.5 backbone; AE-VLA achieves higher reported success than the strongest baselines in simulation and on physical robots.

  • Problem

    Existing evaluations offer limited insight into which training and architectural choices support recombining familiar arm-level skills under different coordination requirements.

    Source: §1 · p. 2

  • Method

    ACG-Bench evaluates dual-arm policies on unseen compositions with shared per-arm atomic prompts and task-goal, milestone, order, and timing requirements, while the study compares strategies using a common π0.5 backbone.

    Source: §1 · p. 2

  • Results

    21.53% generalization CCSR in simulation and 39.00% mean success across five unseen SO101 conditions for AE-VLA, versus 5.53% and 10.00% for Dual π0.5, respectively.

    Source: §6 · p. 9

  • Takeaways & Limitations

    The study highlights complementary benefits from skill-conditioned parameters and arm-wise attention for dual-arm skill reuse under unseen compositions.

    Source: §1 · p. 2, §6 · p. 9

    The study focuses on execution under supplied skill structures; a high-level planner for decomposing novel tasks, assigning skills to arms, and revising plans remains a next step.

    Source: §6 · p. 10

Abstract

from arXiv · show

Generalization in multi-arm collaboration can be studied as composing familiar atomic skills in new ways across arms. However, existing evaluations offer limited insight into which training and architectural choices support this ability under different coordination requirements. We introduce \textbf{ACG-Bench}, a benchmark for \emph{Arm-wise Compositional Generalization} that provides a common testbed for studying skill recomposition in dual-arm policies. It contains 23 task--condition pairs across 8 task families, with 6 in-domain conditions and 17 unseen compositions covering reordering, synchronization, their combination, and cross-task composition. All methods receive the same per-arm atomic prompts, and success requires achieving the task goal while satisfying physical milestones and specified order or timing constraints. Using $\pi_{0.5}$ as a common vision-language-action backbone, we compare representative data-augmentation and architectural strategies with shared source data and a common evaluation protocol. Our architectural study examines arm-token grouping, skill-specific LoRA adapters (SkillLoRA), and arm-wise attention (AWA), highlighting the complementarity of skill-conditioned parameters and attention structure. Combining these choices yields \textbf{AE-VLA}, which achieves 21.53\% generalization success in simulation, compared with 2.94\% for Single $\pi_{0.5}$, 3.06\% for MA-VLA, and 5.53\% for two independently controlled $\pi_{0.5}$ policies. On physical SO101 robots, AE-VLA reaches 39.00\% mean success across five unseen conditions, compared with 10.00\% for the strongest baseline. These findings provide empirical guidance for designing dual-arm policies that generalize beyond fixed training routines.

1 Introduction

The paper studies whether dual-arm policies can recombine familiar arm-level skills under unseen coordination requirements, and introduces ACG-Bench to evaluate this ability. Using a shared π0.5 backbone and controlled comparisons, it finds that combining skill-conditioned parameters with arm-wise attention improves generalization.

1 Introduction

  • ACG asks whether familiar arm-level skills can be reused when their order, synchronization, or task partners change.The paper motivates this as a way to move beyond fixed collaboration routines without collecting demonstrations for every possible combination.Source: §1 · p. 2
  • ACG-Bench contains 23 task–condition pairs across eight task families, including six in-domain conditions and 17 unseen compositions.Unseen groups cover reordering, synchronization, their combination, and cross-task composition; success requires the task goal plus physical milestones and order or timing constraints.Source: §1 · p. 2
  • 21.53% generalization success in simulation and 39.00% mean success across five unseen SO101 conditions: AE-VLA exceeds strongest-baseline means of 5.53% and 10.00%, respectively.AE-VLA combines skill-specific LoRA adapters and arm-wise attention.Source: §1 · p. 2
  • The architectural study examines arm-token grouping, SkillLoRA for skill-conditioned parameters, and AWA for regulating cross-arm information flow.Their combined configuration is AE-VLA, and the study reports complementary benefits from skill-conditioned parameters and arm-wise attention.Source: §1 · p. 2
  • The study compares representative augmentation and architectural strategies using a common π0.5 backbone, source dataset, and execution protocol.The baseline comparisons include shared-policy π0.5, MA-VLA with Arm Shuffle, and two independently controlled π0.5 policies; all show limited generalization to unseen compositions.Source: §1 · p. 2

2 Related Work

Prior work studies task composition and bimanual coordination, but the paper focuses on a distinct shift: familiar local behaviors must operate under new cross-arm order or timing relations. It uses a controlled setup to examine arm-wise representations, skill-specific weights, and attention access.

Vision-language-action policies

  • The paper studies shifts in cross-arm skill order or timing while local behaviors remain familiar, rather than task, scene, or embodiment transfer.This shift is evaluated on π0.5.Source: §2 · p. 3

Bimanual learning and coordination

  • ACG tests whether learned bimanual coordination supports relations absent from training, using shared scene context and restricted attention between local arm streams.Source: §2 · p. 3

Arm-wise representation and skill reuse

  • TwinVLA, SkillVLA, and MA-VLA explore related arm-specific representations, hierarchical skill decisions, communication, or arm-role permutation; this study compares related design ideas in one setup.The compared ideas are per-arm token groups, skill-specific weights, and attention access.Source: §2 · p. 3

Manipulation benchmarks and skill composition

  • Existing manipulation benchmarks vary objects, layouts, goals, appearance, sequences, or skill combinations, while recent bimanual suites expand task and scene diversity.ACG-Bench instead controls the low-level skill vocabulary and changes the arm-level event graph; its metric rejects executions with the wrong order or timing.Source: §2 · p. 3

3 Arm-wise Compositional Generalization

The paper frames arm-wise compositional generalization as executing familiar skills in combinations absent from training, with order and synchronization expressed as task constraints. ACG-Bench tests this using held-out compositions, common per-arm prompts, and scoring that checks both task goals and coordination requirements.

3.1 Problem Formulation

  • The task is represented by an event graph whose nodes are skill executions with specified arms, objects, and targets.Order constraints specify which events precede others, while synchronization constraints require designated completion milestones within a task-specific time window.Source: §3.1 · p. 3
  • A test condition uses skill types seen in training but a joint event graph not seen in training, and the policy must satisfy the task goal, order, and timing.Arm-specific phase instructions provide the target structure, so the evaluation tests execution of a new composition rather than plan inference.Source: §3.1 · p. 3, §3.2 · p. 4

3.2 ACG-Bench

  • ACG-Bench has six source task families and two held-out cross-task families, with 100 clean expert demonstrations collected per source task.The source demonstrations are pooled into one multitask training set; held-out compositions contribute no training demonstrations, and methods use the same policy across conditions without task-specific fine-tuning.Source: §3.2 · p. 4
  • A common rule-based scheduler gives every method identical per-arm prompts, and shared state checks record milestone times for scoring.The protocol evaluates execution of supplied skill plans, not plan inference.Source: §3.2 · p. 4
  • The benchmark has 23 task–condition pairs: six in-domain and 17 generalization pairs across Reorder, Sync, Sync+Reorder, and Cross-task Combination.Groups may change more than one aspect of difficulty: cross-task tests change task context, while Sync+Reorder changes both order and timing.Source: §3.2 · p. 4

4 Design Choices on 𝜋0.5

The section presents arm-token grouping, SkillLoRA, and AWA as design choices for recomposing familiar skills across arms while retaining shared scene context. It describes their shared policy setup and how each choice changes action representations, parameters, or attention.

4 Design Choices on 𝜋0.5

  • The study tests token grouping, skill-specific weights, and attention changes on pretrained π0.5, with AE-VLA combining SkillLoRA and AWA on Token Group.The four SkillLoRA/AWA settings are tested on Token Group.Source: §4 · p. 4

4.1 Shared Backbone and Arm-token Grouping

  • Token Group gives each arm its own H-token action range within one shared action expert, unlike Single π0.5’s joint action tokens.Both groups still embed the full noisy joint action, so their inputs are not independent.Source: §4.1 · p. 5

4.2 Skill-conditioned Action Parameters

  • SkillLoRA selects skill-specific low-rank updates for each arm’s action tokens, with adapter banks shared across arms and tasks.Separate arm routers select an adapter from pooled prompt/state features; predicted selections are used in both training and inference, while skill labels supervise the routers.Source: §4.2 · pp. 5–6

4.3 Arm-wise Attention Connectivity

  • AWA restricts each arm’s local prefix and action attention while global queries can still read both local prefixes, preserving a route for shared observation information.The mask also makes action blocks lower triangular, changing temporal attention as well as blocking cross-arm action directions.Source: §4.3 · p. 6

4.4 Training and Comparison Protocol

  • All variants retain flow matching; SkillLoRA variants also optimize router cross-entropy over valid per-arm skill labels, ignoring missing labels.The hard router choice blocks action-loss gradients through the router; variants without SkillLoRA use only the action loss.Source: §4.4 · p. 6

5 Experiments

The experiments compare dual-arm policies under shared evaluation conditions, measuring task success together with milestone and coordination-constraint compliance. AE-VLA improves compositional success in simulation and on SO101 robots, while the architectural comparison also reveals an in-domain cost and limits on isolating design effects.

5.1 Experimental Setup

  • All policies use the same pretrained π0.5 initialization and per-arm evaluation prompts; each condition receives 100 episodes, and generalization averages 17 unseen conditions.CCSR requires reaching the goal and meeting milestone, order, and timing constraints; TwinVLA was omitted from quantitative comparisons because its pretrained weights differ.Source: §5.1 · pp. 6–7

5.2 Main Results

  • 21.53% generalization CCSR for AE-VLA, versus 2.94% for Single π0.5, 3.06% for MA-VLA, and 5.53% for Dual π0.5.Figure 3 also reports 8.67% for AE-VLA on Sync and 27.17% in-domain, below all three baselines.Source: §5.2 · p. 7, Figure 3 · p. 8

5.4 Real-world SO101 Results

  • 39.00% mean success on five unseen SO101 conditions for AE-VLA, versus 2.00% for Single, 6.00% for MA-VLA, and 10.00% for Dual.AE-VLA led on four of five conditions; in-domain success was 60.00%, below Dual's 70.00%.Source: §5.4 · p. 8

5.3 Design Choices and Their Interaction

  • 21.53% generalization CCSR with SkillLoRA and AWA, versus 6.00% with SkillLoRA alone and 5.82% with AWA alone.Token Group reaches 3.82% from 2.94%; AWA's gain is 2.00 points without SkillLoRA and 15.53 with it, a 13.53-point interaction. AWA lowers in-domain CCSR, and the comparison does not isolate the bundled changes within SkillLoRA or AWA.Source: §5.3 · pp. 7–8, Table 2 · p. 9

6 Conclusion

The conclusion presents ACG-Bench and AE-VLA’s results, while bounding the study to execution under supplied skill structures and identifying full planning–execution as a next step.

6 Conclusion

  • ACG-Bench evaluates dual-arm skill reuse under unseen ordering, synchronization, and cross-task compositions using a common π0.5 backbone to compare strategies.The conclusion highlights the complementarity of skill-conditioned parameters and arm-wise attention.Source: §6 · p. 9
  • 21.53% generalization CCSR in simulation and 39.00% mean success across five unseen SO101 conditions compare with 5.53% and 10.00% for Dual π0.5, respectively.Table 3 reports the SO101 trials; the paper also notes that synchronization and in-domain performance remain challenging.Source: §6 · p. 9, Table 3 · p. 10
  • The study focuses on execution under supplied skill structures; a high-level planner that decomposes tasks, assigns arms, sets order and synchronization, and revises plans remains a next step.Coupling planning with execution would enable evaluation of the full planning–execution loop and construction of new combinations for unseen tasks.Source: §6 · pp. 9–10

AI Use Statement

The authors used generative AI only to polish manuscript language and presentation, and reviewed the assisted text.

AI Use Statement

  • Generative AI was used for grammar, phrasing, and readability, not experimental data, benchmark or protocol design, evaluation implementation, or replacement of scientific judgment.The authors reviewed all AI-assisted text and take responsibility for the paper’s accuracy, claims, results, and final content.Source: AI Use Statement · p. 11

Ethics Statement

The authors report no specific anticipated ethical concerns beyond standard physical-robot safety risks.

Ethics Statement

  • The authors identify standard physical-robot operation risks and call for appropriate deployment precautions.Source: Ethics Statement · p. 11

Reproducibility Statement

The appendices provide benchmark, implementation, training, evaluation, architectural-results, and real-world setup details for reproduction and comparison.

Reproducibility Statement

  • Appendices A–C describe benchmark conditions and atomic skills, policy implementation and training, evaluation protocols, per-condition architectural results, and the real-world setup.Source: Reproducibility Statement · p. 11

A Benchmark and Atomic Skills

ACG-Bench studies whether familiar arm-level skills can be reused under new coordination requirements. It defines atomic operations, builds demonstrations and per-arm prompts, and evaluates recomposed order and timing constraints.

A.1 Condition Inventory

  • The condition inventory distinguishes execution requirements within the Cross-task group, including Reference and Sync.Cross-task conditions remain in that group even when they also change timing.Source: Table 4 · p. 14, §A.1 · p. 14

A.2 Task-program and Demonstration Generation

  • Task programs are generated from task specifications, tested from varied valid initial states, and revised after failures before accepted programs produce 100 demonstrations per source task.Demonstrations are split into skills using gripper state, object attachment, poses, and joint positions.Source: §A.2 · p. 14

A.3 Atomic Skills and Their Arguments

  • An atomic skill describes one arm’s behavior during a phase, such as Pick or Place, and can include several control steps.Skill executions include an arm, operation, object or articulated part, and optional target or dependency; prompts specify behavior rather than a fixed trajectory.Source: §A.3 · p. 14, Table 5 · p. 15

A.4 Skill Coverage Across Tasks

  • Reorder and synchronization conditions change relations across arms while retaining a family’s operations, whereas cross-task compositions pair source behaviors with new partners or targets.Sharing an operation name alone does not make the full manipulation familiar; objects, targets, and partner arms also matter.Source: §A.4 · p. 15, Table 6 · p. 15

A.5 From Demonstrations to Per-arm Prompts

  • Demonstration segments receive paired per-arm prompts, and a phase index identifies the prompt pair rather than one atomic skill.Segmentation uses gripper transitions for several tasks, but Push Cubes and cabinet pulling also require motion or handle-grasp cues.Source: §A.5 · p. 15, Table 7 · p. 16
  • At evaluation, a shared scheduler updates prompts from task state checks, while success is checked separately against required milestones and the final goal.A demonstration’s gripper closure is an annotation cue, not sufficient evidence of a successful test-time grasp.Source: §A.5 · p. 16

A.6 Examples of Unseen Compositions

  • Unseen compositions can reverse a source routine’s event order or synchronize milestone completion within a task-specific time window.Synchronization is based on recorded event times, not identical trajectories or continuously overlapping arm motions.Source: §A.6 · p. 16, Table 8 · p. 16

B.1 Training Configuration

The training and architecture details hold the shared source data and evaluation prompts in common while varying skill-conditioned weights and attention structure. Arm-specific token groups provide the layout for those choices.

B.1 Training Configuration

  • The grouped AE-VLA variants share the source dataset, batch size, training steps, and learning-rate settings, and use a flow-matching action loss on joint actions.The loss samples demonstration actions, Gaussian noise, and noise levels; inference uses ten Euler denoising steps.Source: §B.1 · pp. 16–17

B.2 Shared Instruction Interface and SkillLoRA

  • All methods receive the same per-arm atomic prompts at evaluation, while SkillLoRA trains skill routers from skill labels and adds a skill-classification loss.SkillLoRA adapters are shared across arms, while router MLPs are arm-specific; its effects are not isolated from changes in parameter count and training loss.Source: §B.2 · pp. 17–18

B.3 Token Layout and Exact Attention Masks

  • Token Group assigns separate action-token blocks to each arm, and AWA uses those group boundaries to restrict attention connections.With AWA, action tokens read their own arm’s prefix group and same-arm action tokens at current or earlier chunk positions; the comparison also changes within-chunk attention.Source: §B.3 · pp. 18–19

B.4 Relationship to Prior Design Principles

  • The study compares selected design choices within a fixed π0.5 backbone rather than ranking full TwinVLA or SkillVLA systems.The variants omit aspects of those systems, including SkillVLA’s high-level skill selector and adaptive cooperation gate; TwinVLA’s different pretrained weights are also excluded from quantitative comparison.Source: §B.4 · p. 19

C.1 Task Results

The supplied results passages describe the architectural comparison, physical evaluation setup, and constraint-compliant scoring, but do not provide condition-level outcome values. They also report a collision-related scoring override for some Dual π0.5 successes.

C.1 Task Results

  • Table 1 lists the 23 conditions evaluated for the baseline policies and AE-VLA.Source: §C.1 · p. 19

C.2 Complete Architectural Results

  • The complete architectural comparison scores four Token Group settings on the same 23 conditions using 100 episodes per condition.Token Group + AWA uses the same conditions, synchronization windows, and scoring rules as the other variants.Source: §C.2 · p. 19, Table 10 · p. 20

C.3 Real-world SO101 Setting

  • The SO101 evaluation trains on 150 teleoperated demonstrations from three tasks and tests eight conditions, including five generalization conditions.Cube in Bowl is held out from training; each method has 160 rollouts, with randomized valid object poses and distractors.Source: §C.3 · p. 20

C.4 Constraint-compliant Scoring

  • Episodes require the final goal and ordered milestones, with synchronized milestones required to finish within task-specific windows.Manual review found collisions in all apparent Dual π0.5 successes on Stack Bowls/Sync and Cube in Bowl/Sync; their CCSR is zero while milestone progress is retained, so progress does not measure collision-free execution.Source: §C.4 · pp. 20–21
Loading 2610.06184v1…