PaperlayerView exact revision on arXiv

Source-linked AI summary

Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies

Ce Hao, Xuanran Zhai, Yaohua Liu, Harold Soh

arXiv:2601.21251v1cs.RO

TL;DR

Multi-task diffusion policies face a generalization–efficiency challenge because scaling model size is costly for real-time manipulation. SMP addresses this with a compact, state-adaptive orthogonal skill basis, sticky routing, and adaptive expert activation, and reports higher multi-task and transfer success at substantially lower inference cost than large diffusion baselines. The study supports reusable skills and sparse activation as a practical approach to scalable manipulation, while remaining scoped to relatively small backbones and bimanual tasks.

  • Problem

    Multi-task diffusion policies require generalization across diverse tasks, but scaling policy networks incurs inference costs that can be impractical for real-time manipulation.

  • Method

    SMP learns reusable skills in a state-adaptive orthogonal action basis, composes them with sticky routing, and activates a compact expert subset during inference.

  • Results

    SMP achieves higher multi-task and transfer success at substantially lower inference cost than large diffusion baselines in simulation and on a real dual-arm platform.

  • Takeaways & Limitations

    Compact reusable skills and selective activation provide a practical path toward scalable, transferable multi-task manipulation within the evaluated settings.

  • Takeaways & Limitations

    The study uses relatively small diffusion backbones and focuses on bimanual manipulation, leaving larger models, broader platforms, and robustness under noise or domain shift for future work.

Abstract

from arXiv · show

Diffusion-based policies have recently shown strong results in robot manipulation, but their extension to multi-task scenarios is hindered by the high cost of scaling model size and demonstrations. We introduce Skill Mixture-of-Experts Policy (SMP), a diffusion-based mixture-of-experts policy that learns a compact orthogonal skill basis and uses sticky routing to compose actions from a small, task-relevant subset of experts at each step. A variational training objective supports this design, and adaptive expert activation at inference yields fast sampling without oversized backbones. We validate SMP in simulation and on a real dual-arm platform with multi-task learning and transfer learning tasks, where SMP achieves higher success rates and markedly lower inference cost than large diffusion baselines. These results indicate a practical path toward scalable, transferable multi-task manipulation: learn reusable skills once, activate only what is needed, and adapt quickly when tasks change.

1 INTRODUCTION

SMP addresses the challenge of multi-task diffusion policies by abstracting reusable manipulation skills and activating only task-relevant experts. Evaluations report strong multi-task and transfer performance with lower inference cost than diffusion baselines.

  • 1 INTRODUCTION: Multi-task diffusion policies face a trade-off between generalization and the inference cost of oversized networks.Scaling policy networks can support interpolation across unseen tasks, but real-time manipulation requires moderate model size and sampling latency.
  • 1 INTRODUCTION: Skill-based policy learning offers an alternative by abstracting task-invariant skills from demonstrations and reusing them across tasks.Earlier approaches primarily target exploration or hierarchical control rather than efficient manipulation-skill abstraction.
  • 1 INTRODUCTION: SMP abstracts manipulation skills through a state-dependent orthonormal action basis and slowly varying sticky gates.The method also uses a variational objective and a state-only router to produce phase-consistent behaviors.
  • 1 INTRODUCTION: SMP’s adaptive expert activation selects a compact subset of experts during inference to reduce computation while maintaining action-sampling accuracy.The framework is designed for compact, reusable, transferable skills and real-time bimanual manipulation.
  • 1 INTRODUCTION: Simulation and real-world evaluations show consistently good multi-task performance, lower inference cost, stable routing, cross-task skill reuse, and effective few-shot transfer.The reported analyses associate orthonormal skills and sticky routing with fewer gate switches, while adaptive activation reduces active parameters and latency.

2 BACKGROUND AND RELATED WORK

Prior work pursues manipulation generalization through diffusion policies, skill abstraction, and mixture-of-experts architectures. However, existing approaches generally do not explicitly decorrelate action spaces or stabilize routing for transparent cross-task bimanual skill reuse.

  • 2 BACKGROUND AND RELATED WORK: Diffusion-based generative policies achieve strong single-task manipulation performance and stable training through iterative denoising or flow-matching dynamics.Extensions include longer-horizon trajectory or decision transformers and vision–language–action models for broader generalization.
  • 2 BACKGROUND AND RELATED WORK: Skill-abstraction research includes unsupervised diversity-based discovery, hierarchical options, and compact latent skill identifiers learned from demonstrations.These approaches aim to reuse task-agnostic motion primitives across tasks.
  • 2 BACKGROUND AND RELATED WORK: Mixture-of-experts architectures scale capacity and route computation across modules in diffusion-based policies.The related-work discussion positions MoE as a complementary route to generalization and computational allocation.
  • 2 BACKGROUND AND RELATED WORK: Most prior approaches do not explicitly decorrelate action spaces or enforce phase stability, leaving skills entangled and bimanual-role reuse opaque.Rapid routing switches are also identified as a limitation of existing methods.

3 PRELIMINARIES

The preliminaries formulate multi-task manipulation as conditional action imitation from demonstrations and describe diffusion policies as conditional denoising processes. Training uses a standard diffusion objective to support efficient test-time sampling.

  • 3 PRELIMINARIES: The problem formulation defines multiple subtasks sharing robot state and action spaces, with the task index included in the state as an identifier.Demonstration trajectories contain consecutive state–action pairs, and one policy is trained to imitate demonstrations across all tasks.
  • 3 PRELIMINARIES: Diffusion policies model a clean action as the endpoint of a latent Markov chain and generate it through conditional reverse denoising.The forward process progressively adds noise, while the reverse process is conditioned on the current state.
  • 3 PRELIMINARIES: Training maximizes the conditional data likelihood using a standard DDPM parameterization or an equivalent noise-prediction loss.The resulting objective supports efficient conditional sampling at test time.

4 METHOD: SKILL MIXTURE-OF-EXPERT POLICY

SMP represents actions in a state-adaptive orthogonal skill basis and combines skills with slowly varying gates. Variational training disentangles coefficient learning from basis learning, while adaptive activation selects only the most important experts at inference.

  • Skill disentanglement: Direct mixtures can be non-identifiable, so SMP assigns each skill a distinct orthogonal direction in the current action space.This provides non-overlapping contributions and well-conditioned coefficient supervision.
  • Skill representation: Actions are decoded as B(g ⊙ z), where orthonormal basis vectors provide additive skill contributions.Each skill contributes along its own one-dimensional subspace, yielding decoupled gradients.
  • State-adaptive basis: SMP generates B(s) with a thin-QR retraction, producing a state-adaptive orthogonal frame that avoids discontinuous column-sign flips.The moving frame adapts to task geometry while preserving orthogonality.
  • Sticky routing: Sticky Dirichlet-Markov gates preserve phase-like activation patterns while global-usage priors discourage skill collapse.The persistence parameter controls temporal stickiness, while other priors encourage broad but non-uniform skill use.
  • Variational objective: Variational training combines reconstruction, gate/global-usage regularization, and coefficient regularization aligned with diffusion priors.Only the reconstruction pathway updates the skill basis, while coefficient supervision remains stable in skill space.
  • Adaptive expert activation: Adaptive inference ranks experts by gate mass and activates either the top-k experts or the smallest set covering a target mass fraction.Because the selection score is additive, sorting yields sparse, state-dependent activation that reduces inference cost while preserving accuracy.

5 EXPERIMENTS

Experiments evaluate SMP on simulated and real bimanual manipulation, testing multi-task performance, structured routing, computation, and transfer. Across these settings, SMP combines reusable skill structure and adaptive expert activation with strong performance and lower inference cost.

  • Experimental setup: SMP is evaluated on RoboTwin-2 and RLBench-2 simulation benchmarks for bimanual multi-task manipulation, alongside real-robot and transfer experiments.The evaluation compares SMP with diffusion, transformer, skill-abstraction, and feed-forward MoE baselines.
  • Multi-task results: Scaling RDT by 10× over DP improves success by only 19%, while DP, DP3, and ACT underfit multimodal bimanual multi-task distributions.Sparse Diffusion Policy also exhibits frequent gate switching and action oscillations because it lacks explicit geometric skill disentanglement.
  • Multi-task behavior: SMP uses orthonormal skill bases and sticky routing to organize actions by arm roles and pick–move–place phases, with sparse, phase-consistent expert activation.Left- and right-arm behaviors use different experts, while translation, rotation, and gripper-focused experts recur across trajectory phases and tasks.
  • Computation costs: SMP activates about 30% of its own parameters—roughly 7% of RDT’s total—while maintaining high success and lower inference time than single-backbone diffusion baselines.Adaptive top-k/coverage routing denoises only selected experts; increasing the activation budget slightly improves reconstruction but raises latency.
  • Transfer learning: With 10-shot adaptation, SMP achieves the best transfer success by fine-tuning only relevant existing experts rather than adding new experts.Concentrating limited adaptation data on the sparse active subset outperforms the compared transfer strategies.

6 CONCLUSION

SMP composes manipulation actions through compact orthogonal skills, sticky routing, and adaptive activation, with reported benefits for multi-task and transfer manipulation. The study remains scoped to relatively small diffusion backbones and bimanual manipulation.

  • Results were averaged over 100 episodes for the listed manipulation tasks.
  • SMP composes actions through a compact state-adaptive orthogonal skill basis with sticky routing and adaptive expert activation.These design elements are presented as the basis for reusable, phase-consistent skills.
  • SMP delivers higher multi-task and transfer success at substantially lower inference cost than large diffusion baselines.The conclusion also reports real-time control and cross-task skill reuse.
  • Skill composition: The evaluation includes skill-composition tasks such as placing fries in a skillet and placing a bottle in a cabinet drawer.These tasks reuse left- and right-hand behaviors from the multi-task set.
  • Limitations and future work: The study uses relatively small diffusion backbones and focuses on bimanual manipulation.Future work proposes larger models and datasets, single-arm and mobile manipulation, and broader real-robot studies.

ETHICS STATEMENT

The work states that it follows responsible research principles and involved no human or animal subjects or personally identifiable information.

  • The study follows principles of responsible stewardship, avoidance of harm, fairness, and respect for privacy.
  • No human or animal subjects were involved, and no personally identifiable information was collected.
  • The authors state that no additional ethical approvals were required.

REPRODUCIBILITY STATEMENT

The paper describes its algorithms, architectures, training procedures, experimental details, and robot hardware to support replication.

  • Algorithms, model architectures, and training procedures are described in the paper and Appendix A.
  • Experimental details are provided in Appendix B, including hyperparameters, dataset sources, and preprocessing steps.
  • Robot hardware details are provided to support replicability of the experiments.

DISCLOSURE OF LLM USE

The authors report using an LLM for language editing, idea validation, and error checking while retaining responsibility for the manuscript’s scientific content.

  • An LLM was used mainly to check and correct grammar and language issues in the manuscript.
  • The authors also used an LLM to validate ideas and check for errors.
  • The authors take full responsibility for the content, scientific ideas, analysis, and conclusions.

A SKILL MIXTURE-OF-EXPERTS POLICY METHOD DETAILS

SMP represents actions in a low-dimensional, state-dependent orthonormal skill subspace and combines skill coefficients with temporally sticky gates. Its variational formulation trains the basis, gates, and diffusion coefficients jointly while modeling out-of-subspace action energy as reconstruction noise.

  • Skill representation: SMP uses simplex-valued gates to weight K skill experts and state-dependent orthonormal bases to synthesize actions.The gate is a distribution over experts, while the basis provides the action-space frame for combining their contributions.
  • Sticky routing: Sticky Dirichlet–Markov dynamics encourage gates to remain near their previous values while retaining global usage preferences.κ controls temporal persistence, α0 anchors gates toward the global usage vector, and α controls prior diffuseness.
  • State-adaptive basis: Thin–QR retraction maps an unconstrained W(s) to an orthonormal basis B(s), while sign stabilization makes the basis evolve smoothly with state.The QR-based map is applied during each forward pass and differentiated automatically.
  • Action decomposition: The model decomposes actions into B(s)(g_t ⊙ z_t) plus a small residual, assuming demonstrations approximately occupy a K-dimensional state-dependent subspace.Residual energy outside the skill span is absorbed into reconstruction noise rather than modeled explicitly.
  • Variational training: Variational training combines reconstruction, gate, and diffusion coefficient objectives, with stop-gradient coefficient targets preventing diffusion loss from directly updating the basis.The reconstruction path still updates B toward capturing action energy.
  • Evaluation setting: Experiments use 50 teleoperated demonstrations per task in multitask training on a dual-arm PiPER platform, with SDP as a representative MoE diffusion baseline.The supplied setup states that methods receive an explicit task identifier and are trained on the union of task demonstrations.

B.3 SMP IMPLEMENTATION DETAILS

SMP combines a shared observation encoder, lightweight routing and basis networks, and reduced-width per-expert diffusion generators. Inference activates only experts whose router weights exceed a coverage threshold, producing a measured approximately 107 ms control-step runtime.

  • Configuration: SMP uses K = 8 experts and activation threshold τm = 0.95, with reduced channel widths controlling expert model size.The reported optimization setup uses an observation horizon of 3 and an action planning horizon of 8.
  • Architecture: SMP comprises a shared encoder, posterior and prior gates, a state-dependent basis generator, and K per-expert diffusion generators.At inference, selected expert outputs are combined through the learned basis, with a top-1 fallback if no expert exceeds the threshold.
  • Architecture: The observation encoder processes RGB inputs and robot state into a shared feature consumed by the gates, basis network, and diffusion experts.The implementation uses a ResNet-18 encoder and lightweight MLPs for gating and basis generation.
  • Training: All SMP components are trained end-to-end on one NVIDIA A6000 GPU without separate pre-training or optimization stages.The multi-task RoboTwin-2.0 and RLBench-2 runs use 3000 epochs and approximately 20–25 hours of wall-clock time.
  • Inference cost: Approximately 107 ms per control step is the measured end-to-end SMP inference time on an NVIDIA A6000 GPU.The encoder runs once, while the basis network and selected diffusion experts run in parallel; gating adds relatively small overhead.

C.1 ABLATION OF STICKY GATE FUNCTION

Ablations show that sticky routing, state-dependent bases, and adaptive expert activation each contribute to SMP’s multitask behavior. Removing stickiness or replacing the adaptive basis causes large performance losses, while activation controls trade expert count against success.

  • Sticky gate: 0.44 versus 0.54 success rate on RoboTwin-2 results when the sticky gate is removed instead of using SMP.The no-sticky variant also exhibits high-frequency expert switching and less interpretable gate trajectories.
  • Sticky gate: Moderate prior settings perform best: κ ≈20–50 is strongest, while overly weak or strong α, α0, or κ degrades success.κ = 0 is the worst case and corresponds to the no-sticky-gate variant.
  • Adaptive activation: 0.53 versus 0.54 RoboTwin-2 success shows a small but consistent benefit from adaptive rather than fixed top-k expert activation.The fixed-top-k ablation activates four experts at every timestep while leaving other components unchanged.
  • Adaptive activation: τm = 0.95 activates about 2.3 experts per step and reaches roughly 0.54 success, while τm = 0.90 activates about 1.4 experts and reaches roughly 0.46.Increasing the threshold improves success until saturation, after which additional experts provide little gain.
  • Skill basis: 0.40 versus 0.54 RoboTwin-2 success results when a fixed global skill basis replaces SMP’s state-dependent basis.The state-free basis cannot align skill directions with local manipulation geometry, with larger declines in transfer settings.
  • Skill basis: 0.32 RoboTwin-2 success is reported for the global PCA skill basis, which can approach collapse across multitask and transfer evaluations.Although PCA is orthogonal and low-rank, it is global and task-agnostic rather than state-adaptive.
  • Accuracy–efficiency trade-off: SMP reaches 0.54 success with 258M total parameters but approximately 80M activated at inference and approximately 107 ms latency.The reported comparison places SMP above scaled baselines while scaling increases baseline inference time.
Loading 2601.21251v1…