PaperlayerView exact revision on arXiv

Source-linked AI summary

Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface

Zhizhen Zhang, Yuxia Fu, Zijian Wang, Helen Huang, Yadan Luo

arXiv:2609.33125v1cs.CVcs.RO

TL;DR

Co-training can underperform independent task-specific training, while independently trained VLA experts may not retain their capabilities when merged. PolicyWeave preserves a common action interface and selects a sparse merge from the initial context; on RoboCasa365, it reaches 64.7% success with SFT experts and 74.1% after task-specific RL, compared with 60.7% for joint RL.

  • Problem

    Co-training can limit individual-task performance, and strong independently trained VLA experts do not necessarily retain their capabilities after merging.

    Source: §1 · p. 1

  • Method

    PolicyWeave preserves the base policy’s action interface during task-specific post-training and uses initial visual-language context to select and sparsely merge relevant expert updates.

    Source: §1 · p. 2

  • Results

    On 18 RoboCasa365 tasks, PolicyWeave achieves 64.7% success with SFT experts and 74.1% after task-specific RL, compared with 60.7% for joint RL.

    Source: §1 · p. 2

  • Takeaways & Limitations

    RoboCasa365, LIBERO-10, and Piper-arm evaluations support deploying independently learned skills together while retaining long-horizon and real-world manipulation capabilities.

    Source: §1 · p. 2, §5 · p. 9

    PolicyWeave is evaluated with GR00T N1.5 and a shared embodiment and action interface; its merged parameters remain fixed during each episode.

    Source: F LIMITATIONS · p. 26

Abstract

from arXiv · show

Co-training offers a straightforward way to build a multi-task vision-language-action (VLA) policy, but can fall short of the performance achieved by training each task independently. The challenge is to retain these task-specific gains in a multi-task policy without joint post-training. Combining independently trained experts through model merging is a natural approach, yet strong individual experts do not necessarily yield a strong merged policy. We identify one source of this incompatibility: task-specific changes to the action interface, comprising action normalization and the action encoder and decoder. We propose PolicyWeave, combining merge-compatible post-training with context-guided sparse merging. During post-training, all experts retain the common base policy's action interface, while task adaptation is restricted to LoRA updates in the hidden layers of the action model. This makes the experts more compatible with existing model merging methods. However, merging all experts can still introduce interference from unrelated tasks at deployment. PolicyWeave scores each expert's LoRA updates using the initial visual-language context, determines the expert set through leave-one-layer-out ranking stability, and forms a sparse weighted merge of the selected updates that remains fixed for current task. We evaluate PolicyWeave with GR00T N1.5 on 18 RoboCasa365 tasks, using only 10% of the target-task demonstrations for supervised fine-tuning (SFT). Preserving the shared action interface raises the average success rate across four static merging methods from 17.0% to 52.8%. PolicyWeave achieves 64.7% success with these SFT experts and 74.1% after task-specific reinforcement learning (RL), compared with 60.7% for joint RL. Further evaluations on LIBERO-10 and an AgileX Piper arm support the deployment of independently learned skills in long-horizon and real-world manipulation.

1 INTRODUCTION

The paper asks whether independently trained task capabilities can be retained in a multi-task policy without joint post-training. It introduces PolicyWeave to preserve a common action interface and limit interference when merging experts.

1 INTRODUCTION

  • Task-specific changes to action normalization and the action encoder and decoder can break compensations in hidden-layer parameters when experts are merged.This incompatibility can alter merged actions even when the original experts produce identical actions.Source: §1 · p. 1
  • PolicyWeave keeps the base policy’s action interface fixed and restricts task adaptation to hidden-layer LoRA updates in the action model.This lets existing merging methods combine task updates without merging task-specific action interfaces.Source: §1 · p. 2
  • Because static merging can include task-irrelevant updates, PolicyWeave ranks experts by their responses to the initial visual-language context and uses ranking stability to select a sparse set.The method does not require a trained router or calibration trajectories.Source: §1 · p. 2
  • On 18 RoboCasa365 tasks, shared-interface experts raise the average success rate across four static merging methods from 17.0% to 52.8%.PolicyWeave reaches 64.7% with SFT experts and 74.1% after task-specific RL, versus 60.7% for joint RL.Source: §1 · p. 2
  • Evaluations on LIBERO-10 and a Piper arm support long-horizon skill retention and multi-task manipulation on a physical robot.The RoboCasa365 evaluation uses 10% of target-task demonstrations for SFT.Source: §1 · p. 2

2 METHOD: POLICYWEAVE

PolicyWeave makes independently trained VLA experts more compatible by preserving their shared action interface, then uses initial context to select and merge relevant updates. The selected merged parameters remain fixed throughout an episode.

2.1 MERGE-COMPATIBLE VLA POST-TRAINING

  • Task-specific interface changes can be compensated by expert hidden weights, but merging both can break that compensation and alter actions.The action interface comprises normalization statistics and the action head’s encoder and decoder.Source: §2.1 · p. 3
  • With a shared action interface, the average success rate across four static merging methods on 18 RoboCasa365 tasks rises from 17.0% to 52.8%.Paired OpenDrawer rollouts using the same weight-averaging rule show successful opening only with the shared interface.Source: §2.1 · p. 3, Figure 2 · p. 3
  • PolicyWeave retains the base policy’s normalization statistics and freezes its action encoder, decoder, and VLM, restricting adaptation to hidden action-model LoRA parameters.This shared-interface post-training supports both existing static methods and context-guided sparse merging.Source: §2.1 · p. 3

2.2 CONTEXT-GUIDED SPARSE MERGING

  • Even with a shared interface, static merging can introduce interference from updates irrelevant to the current task, so PolicyWeave uses the initial observation and instruction to select relevant experts.The method considers tasks covered by the expert bank without a supplied task identity.Source: §2.2 · p. 3
  • The initial context determines merged parameters that remain fixed throughout the episode while the policy responds to later observations.If only one expert is selected, its weight is one and the deployed parameters equal that expert’s parameters.Source: §2.2 · p. 4
  • PolicyWeave scores each expert’s LoRA response to visual-language context, then uses leave-one-layer-out ranking stability to choose one expert or a sparse set.The selected updates are weighted by their original aggregate scores and merged at each adapted hidden layer.Source: §2.2 · pp. 4–5

3 EXPERIMENTS

The experiments test whether merging preserves independently learned skills, how a shared action interface affects merging, and whether context-guided selection can reduce interference. Evaluation spans RoboCasa365, LIBERO-10, and real-robot manipulation.

3.1 EXPERIMENTAL SETUP

  • The evaluation uses GR00T N1.5 experts and five post-hoc merging baselines; RoboCasa365 uses 50 SFT demonstrations per task, or 10% of available data, followed by RL.LIBERO-10 tests long-horizon control with an SFT-only expert bank.Source: §3.1 · p. 5

3.2 PRESERVING TASK CAPABILITIES THROUGH MERGING

  • 64.7% and 74.1% success: POLICYWEAVE approaches Individual results of 65.6% and 76.2% on RoboCasa365 SFT and SFT-then-RL expert banks.The strongest static method, SSR-Merge, reaches 58.9% and 62.8%, respectively.Source: §3.2 · p. 5, Table 1 · p. 6
  • 75.4% final success and 86.4% partial success: POLICYWEAVE largely preserves LIBERO-10 long-horizon progress and completion, exceeding selected co-trained results including GR00T N1.5 at 72.4%.Without a shared interface, static methods reach 0% final success; with it, they reach 14.0–28.2%.Source: §3.2 · p. 6, Figure 5 · p. 7

3.3 THE ROLE OF A SHARED ACTION INTERFACE

  • Shared normalization differences vary across tasks and are largest in base-motion channels; task-specific WA drifts 54.1 cm by step 160, while shared-interface WA stays within 0.1 cm.Encoder averaging and replacing normalization reduce success more than decoder averaging.Source: §3.3 · pp. 7–8, Figure 6 · p. 7, Table 3 · p. 7

3.4 EXPERT SELECTION AND SPARSE MERGING

  • 75.4% success with an average of 1.076 selected experts: stability-adaptive selection outperforms Top-1 at 73.2% while adapting how many updates are merged.Top-2 lowers success relative to Top-1, and non-shared-interface multi-expert selection yields no successful rollouts.Source: §3.4 · p. 8, Table 4 · p. 8

3.5 REAL ROBOT DEPLOYMENT

  • On four AgileX Piper tasks, POLICYWEAVE outperforms GR00T co-training on every task, with the largest gains on bottle placement and block stacking.It achieves 100% Top-1 expert-selection accuracy without supplied task identity.Source: §3.5 · pp. 8–9, Table 5 · p. 8

4 RELATED WORK

Prior VLA work includes co-training, structured multi-task skill learning, and expert routing, while model-merging work spans static and input-conditioned approaches. PolicyWeave differs by merging independently post-trained updates while preserving a shared action interface.

4.1 MULTI-TASK VISION-LANGUAGE-ACTION MODELS

  • Multi-task VLA methods include skill-guided experts, grouped RL residuals, instruction-selected experts, and kinematic-cluster routing; PolicyWeave merges independently post-trained updates without architectural changes.The cited examples are AtomicVLA, DyGRO-VLA [Lin et al., 2026], DiTEA [Li & Wang, 2026], and KinRT [Yang et al., 2026].Source: §4.1 · p. 9

4.2 MODEL MERGING

  • Model merging includes static conflict-resolution and subspace methods, LoRA-specific methods, and dynamic input-conditioned methods; PolicyWeave preserves a shared action interface for existing static merging.CORAL folds one independently trained LoRA expert into the base, while MergeVLA redesigns the action expert and uses task-specific masks.Source: §4.2 · p. 9

5 CONCLUSION

PolicyWeave preserves the action interface during independent post-training, then uses context-guided sparse merging to limit deployment interference. Results support retaining independently learned skills across RoboCasa365, LIBERO-10, and real-world Piper manipulation.

5 CONCLUSION

  • PolicyWeave preserves the action interface during post-training, improving compatibility with static merging while retaining individual task performance.Context-guided sparse merging addresses remaining interference at deployment.Source: §5 · p. 9
  • RoboCasa365 results show that gains from independent RL carry into the merged policy, while LIBERO-10 and Piper experiments support long-horizon skill retention and real-world multi-task manipulation.Source: §5 · p. 9

APPENDIX OVERVIEW

  • The appendices cover related work, action-interface analysis, context-response scoring, ranking stability, evaluation and training protocols, task-level results, Piper experiments, and limitations.Source: APPENDIX OVERVIEW · p. 15

A.1 MULTI-TASK AND MODULAR VLA LEARNING

This block reviews approaches to multi-task, modular, continual, and post-hoc VLA learning, then develops PolicyWeave’s shared-interface and context-guided merging pipeline. It also describes the expert banks used in evaluation.

A.1 MULTI-TASK AND MODULAR VLA LEARNING

  • Generalist VLA learning shares parameters, while modular and continual approaches build or update skills through prescribed training workflows; PolicyWeave instead merges independently post-trained task policies.Source: §A.1 · p. 15

A.2 VLA POLICY MERGING AND EXPERT SELECTION

  • PolicyWeave handles mergeability during independent post-training and specifies context-response scoring plus stability-based set selection, unlike architecture-oriented MergeVLA or CORAL’s external single-expert selection.Source: §A.2 · p. 16

B.1 SHARED ACTION INTERFACE

  • Keeping the action interface shared makes parameter averaging consistent with action averaging in the scalar linear model; varying interfaces can break that equivalence even when experts initially produce identical actions.Source: §B.1 · pp. 16–17

B.2 CONTEXT-RESPONSE SCORING

  • The context-response score measures directional overlap between an update’s sensitivity and the context’s second moment, and is invariant to scaling either factor.Source: §B.2 · p. 17

B.3 STABILITY OF EXPERT SELECTION

  • The selected expert set is the smallest prefix of the aggregate ranking containing every leave-one-layer-out top-ranked expert; a stable original leader yields a single-expert merge.Source: §B.3 · p. 18

B.4 EPISODE-LEVEL INFERENCE

  • The initial visual-language context selects and weights updates, which are applied to the adapted action-model modules and held fixed for the episode.Hard Top-1 and Hard Top-2 use the same scores and score-normalized weights but fix the selected set size.Source: §B.4 · p. 18

C.1 EXPERT BANKS

  • The RoboCasa365 bank has 18 SFT experts trained with 50 demonstrations each, and its RL bank continues from those SFT checkpoints; LIBERO-10 uses 10 SFT experts.The 50 demonstrations correspond to 10% of available RoboCasa365 training data; LIBERO-10 experts use 29–49 demonstrations per task.Source: §C.1 · pp. 18–19

C.2 TRAINING DETAILS

The training appendix specifies task-specific expert configurations, co-training baselines, and checkpoint-selection procedures. The supplied passages identify settings but do not provide the configuration-table values.

C.2 TRAINING DETAILS

  • Task-specific experts use rank-8 LoRA with scaling factor 16 and zero dropout; SFT uses bfloat16, AdamW, and a cosine schedule with 5% warmup.Source: §C.2 · p. 19, Table 7 · p. 19
  • The RoboCasa365 co-training SFT baseline uses the official checkpoint trained on all 18 target tasks with 10% of training data, and joint RL continues from it.Source: §C.2 · p. 19
  • The selected task-specific RL checkpoints total 610 steps, while the co-trained checkpoint receives 600 joint RL steps and is selected at step 600.Source: §C.2 · pp. 19–20, Table 8 · p. 20, Table 9 · p. 20

C.3 ROLLOUT PROTOCOL AND EVALUATION METRICS

The appendix describes matched evaluation protocols and merging and expert-selection baselines, alongside interface-compatibility results. Without a shared action interface, static merges show no final LIBERO-10 success, while a paired diagnostic reports much larger base displacement under task-specific WA.

C.3 ROLLOUT PROTOCOL AND EVALUATION METRICS

  • RoboCasa365 results average 50 episodes per task, and the specified rollout settings are held fixed across methods within each benchmark.LIBERO-10 comparisons use separate closed-loop rollouts with the same reset schedule and policy-noise seeds.Source: §C.3 · p. 20

C.4 MERGING BASELINES

  • The merging baselines include weight averaging, TIES, TSV-Merge, ISO-CTS, and SSR-Merge; the main static checkpoints retain the shared action interface.Validation rollouts select fixed global coefficients for several methods, using seeds disjoint from final evaluation.Source: §C.4 · p. 20

C.5 EXPERT SELECTION BASELINES

  • The trainable MLP selects one expert from masked mean visual-language context, while Semantic-NN selects by cosine similarity between instruction and task-name embeddings.Source: §C.5 · pp. 20–21

C.6 PUBLISHED LEADERBOARD REFERENCES

  • The leaderboard comparison juxtaposes different settings: published Atomic-Seen scores cover pretraining kitchens, while this study uses target kitchens with SFT followed by RL.The stated 10% refers to SFT demonstrations, separately from base pretraining and RL interaction.Source: §C.6 · p. 21

D.1 ACTION-INTERFACE COMPATIBILITY

  • 0% final success and 8.3–13.2% partial success are reported for all four static methods on LIBERO-10 without a shared action interface.The table reports aggregate success rates; partial success measures the maximum fraction of goal predicates satisfied simultaneously during an episode.Source: Table 11 · p. 21, §D.1 · p. 21
  • Across three matched scenes, maximum base displacement is 88.3–122.0 cm for task-specific WA versus 0.6–1.6 cm for shared-interface WA.Source: §D.1 · p. 22

D.2 LIBERO-10 PER-TASK RESULTS

The LIBERO-10 results section documents per-task and aggregate evaluation, expert-selection behavior across two benchmarks, and robustness to instruction and observation changes. RoboCasa365 routing is nearly diagonal, while LIBERO-10 shows some task confusions; instruction and observation tests show different robustness patterns.

D.2 LIBERO-10 PER-TASK RESULTS

  • Published LIBERO-Long co-training reports provide final success only, while GR00T N1.5 is re-evaluated and MergeVLA is reproduced using its own backbone and fine-tuning pipeline.The passages do not supply the comparative result values.Source: §D.2 · p. 22
  • LIBERO-10 reports final and partial success for each task over 50 episodes, with an equally weighted average across 10 tasks.The supplied passages describe the table's scope but do not provide its cell values.Source: §D.2 · p. 22, Table 12 · p. 22

D.3 EXPERT SELECTION STATISTICS

  • RoboCasa365 expert routing is nearly diagonal, whereas LIBERO-10 shows confusions among tasks sharing objects or intermediate goals.Figure 8 shows row-normalized Top-1 routing confusion matrices, with requested tasks as rows and selected experts as columns.Source: §D.3 · p. 23, Figure 8 · p. 23

D.4 INSTRUCTION AND OBSERVATION ROBUSTNESS

  • POLICYWEAVE leads on native and lexical/syntactic instructions, while Semantic-NN leads on goal-state rephrasings; closed-loop results follow the native-instruction ranking.The comparison uses 900 demonstration-first observations per condition, and selectors remain fixed during evaluation.Source: §D.4 · pp. 23–24, Table 15 · p. 24
  • On initial rollout observations, Top-1 accuracy is 98.67% for POLICYWEAVE, 92.33% for the MLP, and 81.44% for Semantic-NN.The observation-timing evaluation also reports accuracy above 99% across 900 contexts at each of timesteps 0, 16, 32, and 48.Source: §D.4 · p. 24, Table 16 · p. 24

D.5 POLICYWEAVE SCORER ABLATION

The scorer ablation compares key, value, and combined update responses across layers and score variants, then tests routing as the expert bank grows. Multi-layer key scoring gives high expert recall with selections close to one expert, and routing remains accurate with larger banks.

D.5 POLICYWEAVE SCORER ABLATION

  • The 66 scorer configurations vary key, value, or combined responses, score normalization, and layer subsets using 900 initial demonstration observations.The base pass supplies context at eight cross-attention layers with a zero action latent and timestep zero.Source: §D.5 · p. 24, Table 17 · p. 25
  • Aggregating all eight layers favors K over V and K + V for both raw and normalized scores, although V often performs better at individual layers.Cross 3 gives the strongest single-layer K result.Source: §D.5 · p. 24
  • 98.56–100% expert recall with mean k between 1.006 and 1.042 is achieved across 18 multi-layer variants.Using all eight layers, normalized K scores achieve 99.44% recall at mean k = 1.006, while raw K scores achieve 99.78% at 1.007.Source: §D.5 · p. 24

D.6 SCALING THE EXPERT BANK

  • Bank-size statistics cover routing accuracy, expert selection, and routing latency for banks evaluated on 250 native-instruction observations.The supplied caption defines the reported latency as p50 / p95 for a batch of five with the model and expert bank resident on the GPU.Source: Table 18 · p. 25
  • After the strongest distractor is added at K = 2, Top-1 accuracy and expert recall remain 98.00% through K = 18.The bank-size tests use five tasks; exact adapter storage grows linearly, and routing occurs once when the context-conditioned merged update is formed.Source: §D.6 · pp. 24–25

E.1 SETUP AND TRAINING

The Piper experiments train a shared base and task-specific experts, compare them with co-training baselines, and evaluate fixed episode-level routing on four physical manipulation tasks. The paper presents successful POLICYWEAVE rollouts for all four tasks.

E.1 SETUP AND TRAINING

  • The Piper setup uses one AgileX arm, two RGB views, instructions, robot state, and four tasks: bottle reorientation, block stacking, button activation, and cup placement.Actions specify six absolute joint targets and gripper opening; demonstrations are recorded at 30 Hz.Source: §E.1 · p. 25
  • Each task has 51 demonstrations: one per task adapts the shared base, then 50 train a rank-8, scale-16 LoRA expert while preserving normalization and the adapted interface.Expert updates cover hidden action-transformer linear layers, excluding the input-output projectors; the VLM remains frozen.Source: §E.1 · p. 25, Table 19 · p. 26
  • Co-training baselines use all 51 demonstrations per task and train for up to 40,000 steps, with GR00T and π0.5 using different adaptation configurations.GR00T retains the adapted base, frozen action interface, LoRA configuration, and optimizer settings of the single-task experts; π0.5 also updates action projections, timestep embedding, and adaptive normalization.Source: §E.1 · p. 26
  • Each method is evaluated over 20 rollouts per task, and POLICYWEAVE fixes its merged update for each episode after using the simulation experiments' selection rule.Both model families predict 16-step action chunks; GR00T uses four flow integration steps and π0.5 uses ten.Source: §E.1 · p. 26

E.2 REAL-ROBOT DEMONSTRATIONS

  • Figure 9 presents successful POLICYWEAVE rollouts for bottle reorientation, block stacking, button activation, and cup placement.Each row follows a video from left to right, with elapsed-time stamps and wrist-camera insets.Source: §E.2 · p. 26

F LIMITATIONS

The paper identifies limits in the evaluated architectures and deployment settings, the shared-interface scope, and the episode-fixed selection policy. Physical-robot evaluation covers POLICYWEAVE, not static merging.

F LIMITATIONS

  • Experiments use GR00T N1.5 and a Piper arm; other VLA architectures and broader deployment settings remain to be studied.Static merging is evaluated only in simulation, and physical deployment would require reliable coefficient selection and safety validation.Source: F LIMITATIONS · p. 26
  • The method merges experts from a common base with a shared embodiment and action interface; other embodiments or additional action channels require further interface alignment.Source: F LIMITATIONS · p. 26
  • Merged parameters remain fixed throughout each episode, so tasks identified only through interaction or changing goals would require online policy updates.Source: F LIMITATIONS · p. 26

Related papers

Loading 2609.33125v1…