Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent

1 Introduction

• Robot policies must interpret instructions in unfamiliar scenes and translate that understanding into precise physical actions.Source: §1 · p. 2
• ZAP adds motion control to a pretrained vision-language model through supervised fine-tuning and reinforcement learning entirely in simulation.It starts from Qwen 3.6 27B for world understanding and reasoning, then trains on 40 simulated manipulation tasks without real-world demonstrations.Source: §1 · p. 2
• ZAP teaches motion from its own successful traces and simulated RL, using a teacher-free harness that commands end-effector poses and gripper openings.A dense reward ladder credits partial progress during RL.Source: §1 · p. 2
• 96.7% success on real hardware tasks, matching GPT-6 Astra in the same harness at 3× the speed and 1/17 the cost, with no real-robot training data.The same checkpoint transfers to a single-arm Franka and handles new objects, goals, strategies, and language.Source: §1 · pp. 2–3

Composing reasoning and motion

• ZAP learns motion itself and drops the fixed skill library, leaving scene reasoning and action sequencing to the model through a minimal harness.Source: §2 · p. 3

End-to-end policies

• Unlike end-to-end policies trained on robot actions, ZAP inherits language ability from pretraining and learns only motion in simulation.The paper states this avoids tying instruction following to the wording present in action data.Source: §2 · p. 3

Foundation models as robot agents

• ZAP uses its own successful rollouts for fine-tuning and trains RL across 40 tasks without a frontier teacher, then composes motions it never practiced.The paper contrasts this with compact tool-calling agents distilled from frontier teachers and policies built around grasp macros.Source: §2 · p. 3

3.1 One interface across simulation and hardware

• The method uses an identical tool harness in simulation and on hardware, with camera observations, object readouts, pose commands, and controller feedback.The model issues perception and movement calls through a five-tool interface; it plans approaches, grasps, lifts, and releases rather than using grasp macros or motion primitives.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Learning to move in simulation

• Forty simulated tasks cover grasping, lifting, pick-and-place, stacking, and reorientation, with deliberately simple goals that do not require heavy reasoning.The simulated tasks are distinct from RoboLab-120 and test whether learned motion combines with pretrained reasoning on unseen tasks.Source: §3.2 · p. 4
• GRPO trains across all 40 tasks with a dense, task-specific reward ladder because pass/fail rewards can provide no learning signal when tasks have zero successes.For pick-and-place, partial credit rewards approach, grasp, lift, and transport; completing the task receives 5.0 minus a small length penalty. The partial-credit values were not ablated and were chosen so task success outweighs partial reward.Source: §3.2 · pp. 4–5

3.3 Initializing the model

• The model is initialized with successful traces mined from a short GRPO warm-up, using oversampling for harder tasks; prefix-tree search finds traces for tasks that sampling still fails to solve.Search replays partial episodes from deterministic starts, favors high dense reward and less-explored nodes, and can expand to about 1,000 leaves per task.Source: §3.3 · p. 5

4.1 Simulation on the Bimanual YAM

• 74.0% on RoboLab-120 after RL follows 58.3% after SFT and 8.3% for the base model, exceeding GPT-6 Astra and Opus 5.5.The motion training uses 40 simple simulated tasks and transfers to new objects and tasks.Source: Table 2 · p. 5, §4.1 · p. 5

4.2 Real-world transfer

• 96.7% success on real tasks matches GPT-6 Astra in the same harness; GPT-6 Astra in the Robocurve harness reaches 36.7%, MolmoAct2 33.3%, and base Qwen 16.7%.The real-world results cover 30 trials per model.Source: Table 3 · p. 6, §4.1 · p. 6, §4.2 · p. 6
• 146 s per task and $0.29 per task for ZAP compare with 397 s and $4.87 for GPT-6 Astra; ZAP's response time is 6.2 s per turn versus 19 s.The models take a similar number of turns; the passage attributes the time difference to faster model responses.Source: §4.2 · p. 6

4.3 Transfer to a new embodiment

• 55.9% on RoboLab-120 exceeds Lakeside (51%) and FLUX 3 Action (42.9%) on a single-arm Franka without Franka training.The same checkpoint transfers from bimanual YAM arms; the harness handles embodiment-specific joint control.Source: §4.3 · pp. 6–7, Table 4 · p. 7

4.4 Ablations

• 17.2% to 58.3% on RoboLab-120 follows replacing 455 early mined traces with 3.9k traces mined after longer GRPO runs, especially on harder tasks.The authors report that broader task coverage carries over to tasks the model never trained on.Source: Table 5 · p. 7, §4.4 · p. 7
• A 9B Qwen-3.5 policy trained on separate single-arm Isaac Sim tasks exceeds current state-of-the-art VLAs on RoboLab-120, where its evaluation tasks were unseen during training.Training used 4.6k mined examples followed by a final GRPO run.Source: Table 5 · p. 7, §4.4 · p. 7

• The evaluated tasks, objects, scenes, prompts, strategies, and reasoning requirements are absent from training.Source: §5 · pp. 7–8

A new object and a new task: the test tube

• The model completes unseen test-tube placement by rotating a side-lying tube upright and lowering it into a rack hole, despite the hole being hidden from every camera.Neither the objects nor the goal appear in training; it receives only a text instruction, without demonstrations or parameter updates.Source: §5 · p. 8

Steered by the instruction

• On the same starting scene, the model follows a changed instruction by placing the tube in the rack’s top-left hole rather than choosing a hole itself.The default prompt lets the model choose a hole; the revised prompt specifies the top-left location.Source: §5 · p. 8

A new strategy on request: bimanual handoff

• With no training interaction close to a handoff, the model transfers the tube between arms mid-air and then places it in the rack.The handoff requires coordinating the grippers: the receiving arm grasps before the giving arm releases, while avoiding a collision.Source: §5 · p. 8

Complex reasoning: tic-tac-toe and chess

• For tic-tac-toe, the model identifies the center square and draws an X while maintaining marker contact without pressing hard enough to dislodge it.The instruction specifies both the location and the motion needed to draw there.Source: §5 · pp. 8–9
• For chess, the model reasons from the board and “black to move, mate in one” to select the mating piece and target square, then executes the move.Source: §5 · p. 9, Figure 6 · p. 9

• 96.7% success on real hardware tasks; ZAP matches GPT-6 Astra in the harness while running 3× faster and 17× cheaper, and transfers to a single-arm Franka.The conclusion also reports compositional performance on new real tasks, objects, language, and goals, and argues that extending motion skills calls for more simulated tasks rather than robot demonstrations.Source: §6 · p. 9

Tools

• The model controls the robot through five tools; only move_to and set_gripper move it, leaving the model to plan each approach, grasp, lift, and release.move_to sets a complete pose for one or both arms, while set_gripper sets jaw opening; the other tools manage perception queries, a persistent short plan, and episode termination.Source: A The harness · p. 19

Poses

• Each move_to specifies an absolute tool-center-point position in meters and full orientation in a shared world frame, with no relative moves or grasp macros.The model therefore plans the motion sequence itself; learned pick-and-place motions carry over to tube rotation, handoffs, and drawing.Source: A The harness · p. 19

Orientation

• Orientation uses slot yaw, tilt, and tilt direction to specify jaw alignment and tool pointing, avoiding Euler-angle singularities for horizontal tools.Tilt is 0° top-down and 90° horizontal; tilt direction is physically ambiguous only when the tool points straight up or down.Source: A The harness · p. 19, Figure 7 · p. 20

Observations and perception

• Perception continuously segments queried objects in each camera stream and combines masks with depth to estimate 3D centroids, heights, and principal axes.The model can change queries, detections are not fused across cameras, and simulation perception uses rendered images and depth rather than privileged object poses.Source: A The harness · p. 19

Validation; Controller and outcome reports; One harness for simulation and hardware

• Before execution, the harness rejects move_to calls outside declared robot limits and returns the reason; the controller also refuses invalid joint paths.The same harness serves simulation and hardware; moving to a real rig or Franka requires swapping the embodiment description and low-level backend.Source: A The harness · pp. 19–20

B A real-robot episode, turn by turn

• The “pick up both blocks” episode succeeded in 9 turns and 74.6 s, using 4 robot actions, 118,985 input tokens, and 1,920 output tokens.The episode ran on the real bimanual rig.Source: B A real-robot episode, turn by turn · pp. 20, 22
• The agent queried for a plain wooden block and a white block marked “A” to obtain metric positions for both objects.The next snapshot ran the queries on its exact frame.Source: B A real-robot episode, turn by turn · p. 20
• Depth measurements informed an updated two-item plan assigning one block to each arm and specifying top-down approaches, grasp heights, and lifts to z=0.15 m.The plan included verifying that both blocks remained held.Source: B A real-robot episode, turn by turn · p. 21
• Both arms moved to approach positions, descended to the blocks, and closed their grippers; the grippers stalled on the objects as intended.The arms then lifted both blocks to z=0.15 m.Source: B A real-robot episode, turn by turn · pp. 21–22
• The agent checked that both blocks were visibly retained and lifted, then declared success and ended the episode.Their measured top heights increased from about 0.01/0.025 m to 0.15/0.16 m.Source: B A real-robot episode, turn by turn · p. 22

Training details

• GRPO training used 2 groups per task and 8 rollouts per group, totaling 640 episodes per batch across 40 tasks.Training used LoRA with rank 64, up to 50 turns per episode, and a maximum sequence length of 127k tokens.Source: C Training details · p. 23
• The GRPO setup omitted standard-deviation and per-sequence length normalization and enabled STAPO, a DPPO mask with threshold 0.2, and maximum staleness 5.Training was asynchronous and used LoRA.Source: C Training details · p. 23
• Prefix-tree search branches from promising partial episodes instead of restarting each attempt, pruning branches that keep failing and allowing about 1,000 leaves per task.A node is created after each physical action settles.Source: Figure 8 · p. 23

D Per-task real-world results

• Table 6 presents per-task successes and defines RC as the Robocurve harness.The supplied passage gives no per-task success values.Source: Table 6 · p. 23

E Extended related work

• ZAP keeps reasoning separate from motion but drops fixed skill libraries, letting the model reuse learned motion for tasks such as tube rotation or marker strokes.The interface uses tools in physical units.Source: E Extended related work · p. 23
• Prior embodied multimodal policies and VLAs combine reasoning and motion, while WAMs also predict future observations; reported compositional failures include new tasks, perturbations, and skill combinations.The passage notes that compositional action-data coverage is expensive to collect.Source: E Extended related work · p. 24
• ZAP learns full absolute end-effector poses through SFT and RL in simulation, using native text generation to command motion.The comparison passage describes EASG as generating continuous translations and rotations as text from a grounded action frame.Source: E Extended related work · p. 24
• Robot-agent approaches include code-writing policies and frontier agents acting through tools, while compact-agent work includes distillation and simulated training.The closest compact-agent examples listed include Guava, RACaP, Robo-Harness K1, and CaP-RL.Source: E Extended related work · p. 24
• The same checkpoint shows new real-world strategies and transfers from bimanual YAM to simulated Franka without retraining; the paper also reports 3× faster and 17× cheaper execution than the matched frontier model.The passage identifies mid-air handoff and marker drawing as new strategies.Source: E Extended related work · pp. 24–25
• ZAP’s tool interface keeps observations and actions the same in simulation and on hardware, so dynamics and perception—not the interface—differ across settings.The passage places this approach alongside domain and dynamics randomization and procedurally generated simulation transfer.Source: E Extended related work · p. 25