RoboCode: Learning Reusable Action Primitives for Robot Manipulation

1 Introduction

• Demonstrations share short motion patterns such as reaching, lifting, placing, and pushing, suggesting reusable primitives can represent action structure across tasks.Tokenized action dimensions can split coordinated actions, while task-level commands may require large task-specific vocabularies.Source: §1 · p. 1
• Its two stages learn a shared primitive interface from pooled multi-task action chunks, then freeze it while task-specific observation-to-gate policies generate continuous chunks.Primitive learning uses demonstrations without manually designed primitives, task rewards, or task labels.Source: §1 · p. 1
• ROBOCODE represents short-horizon actions as soft compositions of multiple learned primitives instead of direct continuous predictions or single-code assignments.A single primitive may not express all motion factors in a short action chunk.Source: §1 · p. 1
• 75.6% average success rate, with ROBOCODE highest or tied across five ManiSkill tasks, compared with 68.8% for ACT and 41.6% for DP-Large.Controlled ablations and transfer experiments examine soft composition and cross-task primitive sharing.Source: §1 · p. 2

2.1 Imitation Learning for Robot Manipulation

• Imitation policies range from current-action prediction to temporally extended chunks and sequences, including ACT, Diffusion Policy, and generalist policies such as Octo and RDT-1B.These developments broaden imitation learning toward joint modeling of temporally extended robot behavior.Source: §2.1 · p. 2

2.2 Motion Primitives and Skill Learning

• Motion-primitives research spans hierarchical skills, interaction-grounded primitives, demonstration-segmented skills, and cross-embodiment prototypes.These methods differ in what counts as a reusable unit, but share the goal of capturing recurring structure across manipulation behaviors.Source: §2.2 · p. 2

2.3 Structured Action Representations and Codebooks

• Structured action methods use dimension-wise or sequence tokens, coarse-to-fine modes, or learned codebooks; ROBOCODE instead softly combines multiple primitive codes into a continuous action-generating latent.Similarity-based gates activate multiple entries whose weighted combination is decoded into an action chunk.Source: §2.3 · pp. 2–3

3 Problem Setup

• The prediction objective is a future action chunk from the current state input, with H denoting the action-chunk horizon.The setup considers multi-task offline imitation learning from demonstrations.Source: §3 · p. 3
• Rather than directly regressing a chunk, ROBOCODE uses a learnable shared primitive codebook to represent it as a soft composition of primitive codes.K is the number of primitive codes and C is the latent code dimension.Source: §3 · p. 3
• Stage 1 pools action chunks across tasks to train an action encoder and shared interface; the encoder maps chunks to latents, gates produce primitive activations, and a decoder reconstructs chunks.The primitive codebook, gating operator, and action decoder are shared across tasks.Source: §3 · p. 3
• Stage 2 freezes the learned interface and trains a task-specific state encoder to map current inputs into the shared latent space for primitive selection and chunk decoding.This separates shared action-representation learning from task-specific primitive-selection policies.Source: §3 · p. 3

4.1 Stage 1: Shared Primitive Autoencoder

• Stage 1 jointly trains an action encoder and shared primitive interface on action chunks, which the interface reconstructs through its decoder.The action encoder maps chunks into the codebook latent space, and the interface converts that representation into gates.Source: §4.1 · pp. 3–4
• The shared Stage 1 interface uses soft gates so each action chunk can compose multiple primitive codes rather than selecting one code.The gates are based on cosine similarity followed by a shifted sigmoid; reconstruction and projection preserve chunk reconstruction and latent consistency, while sparsity and diversity terms regularize gate usage and codebook diversity.Source: §4.1 · pp. 4–5

4.2 Stage 2: Observation-to-Gate Policy

• Stage 2 freezes the codebook, gating operator, and decoder, then trains a task-specific state encoder to select primitives from the current state input.Training matches predicted gates to target gates inferred from expert future chunks, with an additional action consistency term.Source: §4.2 · p. 5

4.3 Closed-Loop Execution

• At inference, ROBOCODE executes only the first part of each predicted action chunk, then observes the environment again and replans.This receding-horizon procedure uses closed-loop feedback while executing temporally extended chunks.Source: §4.3 · p. 5

5.1 Experimental Setup

• Experiments use successful demonstrations from five ManiSkill tasks, with sliding-window samples and a shared primitive interface trained before task-specific encoders.The shared interface is frozen for Stage 2; closed-loop policies replan every eight steps.Source: §5.1 · pp. 5–6

5.2 Training Diagnostics and Primitive Analysis

• Stage 1 training and validation losses remain closely aligned, while Stage 2 validation losses fall sharply in the first 10–20 epochs and then stabilize across tasks.The authors report no pronounced validation-loss increase in Stage 1.Source: §5.2 · p. 6, Figure 3 · p. 7
• Stage 1 reconstruction MSE averages 0.0187, while gate-prediction MSE averages 1.56 × 10^-4 and action MSE averages 0.0733.PushCube has the largest reconstruction MSE (0.0378) and action MSE (0.2236), although its broad set of successful trajectories means offline action error does not imply poor task performance.Source: §5.2 · pp. 6–7
• Selected codes exhibit distinct temporal coordination across action dimensions, including two-phase patterns, early transitions, recurring changes, and sustained fluctuations.Code 3 becomes nearly constant after approximately Steps 8–9, whereas Code 1 shows relatively high-frequency fluctuations, particularly in the second half.Source: §5.2 · p. 7, Figure 4 · p. 9
• Code usage combines cross-task sharing with specialization: Code 15 is frequent across tasks, while Code 29 is activated primarily in StackCube trajectories.PickCube and StackCube also show similar high usage of Codes 2, 3, 4, and 6; the paper suggests this overlap may reflect shared grasping and lifting motions.Source: §5.2 · pp. 7–8

5.3 Closed-Loop Evaluation

• 75.6% average success rate lets RoboCode lead or tie on all five tasks, versus 68.8% for ACT, 41.6% for DP-Large, and 18.8% for Diffusion Policy.It improves LiftPegUpright from 56% to 80%, PickCube from 86% to 92%, and ties ACT at 98% on PushCube.Source: §5.3 · p. 8, Table 4 · p. 8
• DP-Large improves average success over compact Diffusion Policy from 18.8% to 41.6% but remains below ACT and RoboCode on every task.RoboCode requires the fewest steps on PickCube, PushCube, and StackCube; steps-to-success averages include successful episodes only, while average return is affected by task reward and termination behavior.Source: §5.3 · pp. 8–9

• ROBOCODE jointly learns an action encoder and shared primitive interface from multi-task chunks, then freezes it while training task-specific state encoders.Source: §6 · p. 9
• The interface reconstructs action chunks accurately and exhibits structured code reuse across tasks.Source: §6 · p. 9
• Figure 5 reports per-task fractions of validation trajectories activating each code at least once, using gate threshold 0.1.Source: Figure 5 · p. 10
• 75.6% average success rate, with ROBOCODE highest or tied across five tasks, versus 68.8% for ACT and 41.6% for DP-Large.Source: §6 · p. 10
• Evaluation remains simulation-only, leaving real-world transfer under uncertainty open; offline imitation policies may also face expert-distribution shift and compounding prediction errors.Source: §6 · p. 10