Source-linked AI summary
Atomic-WAM: Execution-Grounded Atomic-Motion Reasoning for World Action Models
Shijun Shi, Zhe Liu, Zhi Hou, Yinan Chen, Tao Zhang, Kai Hu, Yong Liu, Jiangning Zhang
TL;DR
Task-level instructions can leave the next physical interaction underspecified, and supervised atomic-motion learning alone does not ensure that selected motions suit the executor. Atomic-WAM uses a vision-language reasoner to describe state-dependent atomic motions for a WAM, and adapts the reasoner with execution-grounded rewards. On RoboLab, it reports higher task success under both default and vague instructions than Cosmos3-Nano-Policy.
Problem
Task-level instructions may leave the object interaction and local motion unspecified, and supervised atomic-motion learning alone does not guarantee compatibility with the learned executor.
Source: §I · p. 1Method
A vision-language reasoner uses the task instruction and visual context to describe the next local interaction for a WAM, while EG-GRPO adapts the reasoner using physical outcomes from a fixed executor.
Source: §I · pp. 1–2Results
39.6% task success under default instructions and 31.6% under vague instructions on RoboLab, improving over Cosmos3-Nano-Policy by 2.8 and 11.0 percentage points, respectively.
Source: §V · p. 7Takeaways & Limitations
The larger gain under vague instructions indicates particular effectiveness when task instructions are underspecified.
Source: §V · p. 7
Abstract
from arXiv · showhide
World Action Models (WAMs) typically adapt video foundation models to robotic control through joint video and action prediction. However, task-level instructions may leave the next physical interaction underspecified, requiring semantic grounding and state-dependent motion selection. To this end, we introduce Atomic-WAM, an understand-then-execute framework that connects task reasoning to action generation through atomic-motion descriptions. Specifically, a vision-language reasoner describes the next local motion from the task instruction and visual context, guiding a WAM to predict future video and an action chunk. Furthermore, an automatic annotation pipeline curates temporally aligned, contact-enriched supervision from multi-view DROID demonstrations. To make atomic-motion predictions executable by the WAM, we post-train the reasoner with Execution-Grounded GRPO (EG-GRPO). With the WAM fixed, EG-GRPO adapts the reasoner through rewards for motion appropriateness and execution success. Finally, Atomic-WAM achieves 39.6% task success under default instructions and 31.6% under vague instructions in a real-to-sim evaluation on RoboLab. Project page: https://ssj9596.github.io/Atomic-WAM-project
I. INTRODUCTION
Task-level instructions can leave the next object interaction and local motion unspecified, while supervised motion selection alone does not ensure compatibility with an executor. Atomic-WAM links task reasoning to video–action generation through atomic-motion descriptions, supported by curated supervision and execution-grounded reasoner adaptation; it reports 39.6% success under default instructions and 31.6% under vague instructions on RoboLab.
I. INTRODUCTION
- Task-level instructions can leave object choice, interaction phase, and local motion unspecified, creating semantic-grounding and temporal-granularity gaps for control.The instruction “Put the red things in the bin” requires identifying relevant objects and deciding how to manipulate them.Source: §I · p. 1
- Atomic-WAM uses a vision-language reasoner to describe the next local interaction from task and visual context, guiding a WAM to predict future video and an action chunk.The reasoner revises its description as execution changes the scene while the task-level objective remains fixed.Source: §I · p. 1
- The annotation pipeline partitions DROID demonstrations into atomic motions and enriches grasp descriptions with visible contact details to supervise motion selection and execution.The resulting labels provide supervision at the same atomic-motion granularity for both purposes.Source: §I · p. 1
- EG-GRPO adapts the reasoner using physical outcomes from a fixed executor as a learning signal for atomic-motion selection.The paper motivates this loop by noting that supervised atomic-motion learning alone does not guarantee compatibility with the learned executor.Source: §I · pp. 1–2
- 39.6% task success under default instructions and 31.6% under vague instructions on RoboLab.The paper reports these results for its out-of-distribution real-to-sim evaluation.Source: §I · p. 2
A. Embodied foundation models.
The related work contrasts vision–language models’ semantic priors with WAMs’ modeling of physical dynamics, then reviews intermediate representations for connecting reasoning and robot control. It also motivates RoboLab as an out-of-distribution real-to-sim evaluation setting.
A. Embodied foundation models.
- VLA models transfer vision–language representations to action prediction, whereas WAMs couple future visual prediction with robot action generation; Atomic-WAM combines a VLM reasoner with a WAM.The cited discussion characterizes VLA pretraining as emphasizing visual–semantic alignment and WAM video pretraining as modeling environment dynamics.Source: A. Embodied foundation models · p. 2
B. Embodied reasoning for robot control.
- Prior work uses intermediate plans, subtasks, motions, and events to ground reasoning in robot behavior, while Atomic-WAM updates its reasoner from rewards after executing candidate motions with a frozen WAM.The related-work passage says these execution outcomes are generally not used to directly optimize the high-level reasoner under a downstream policy.Source: B. Embodied reasoning for robot control · p. 2
C. Robot simulation evaluation.
- RoboLab evaluates real-world-trained policies on 120 novel tasks in simulation to target out-of-distribution task generalization.The paper adopts this real-to-sim setting because physical resets vary across real-world trials and training–evaluation overlap can saturate conventional simulation benchmarks.Source: C. Robot simulation evaluation · pp. 2–3
III. METHOD
Atomic-WAM connects task reasoning to video–action generation with state-dependent atomic-motion descriptions. Its data curation constructs temporally coherent motion labels from fine-grained physical clips.
III. METHOD
- The method comprises supervised demonstration curation, an understand-then-execute framework, and closed-loop optimization using execution feedback.The stages respectively construct atomic-motion data, connect reasoning to video–action generation, and improve motion selection.Source: §III · p. 3
A. Atomic-Motion Data Curation
- End-effector velocity changes and gripper-state transitions partition demonstrations into physical clips with locally consistent motion and gripper behavior.Gripper transitions capture interaction phases that velocity changes alone may miss.Source: A. Atomic-Motion Data Curation · p. 3
- Three-view clip annotations are merged with boundary observations and temporal context into atomic motions, then enriched with visible grasp details and checked for caption correctness and execution success.The pipeline yields 369k segments from 57k episodes; Qwen3.8-Max checker pass rate is 96.8%, and a manual review of 200 checker-passed samples found 90.5% agreement with human judgments.Source: A. Atomic-Motion Data Curation · p. 3
B. Supervised Understand-then-Execute Learning
Atomic-WAM separates task-conditioned motion description from video–action generation: a reasoner predicts the next atomic motion, and a WAM uses it to predict future video and actions. The components are trained with separate objectives and optimizers.
B. Supervised Understand-then-Execute Learning
- The reasoner predicts the next atomic-motion description from the task instruction and visual context, which the WAM uses to generate future video and an action chunk.After execution, the resulting observation is incorporated into the visual context for the next motion prediction.Source: B. Supervised Understand-then-Execute… · pp. 3–4
- Atomic-motion annotations serve as the reasoner’s prediction targets and the WAM’s language conditions.The two components are trained in parallel using separate forward passes and optimizers.Source: B. Supervised Understand-then-Execute… · p. 4
- Joint flow matching trains the WAM to predict future video latents and action chunks from the current observation, robot state, and atomic-motion description.For each modality, the target is interpolated with Gaussian noise before velocity prediction.Source: B. Supervised Understand-then-Execute… · p. 4
- The reasoner updates its LoRA parameters while the WAM updates its generator parameters using the description encoded by the frozen base reasoner.The reasoner’s LoRA is disabled for the WAM’s forward pass.Source: B. Supervised Understand-then-Execute… · p. 4
C. Closed-Loop Execution-Grounded Reasoner Adaptation
EG-GRPO adapts atomic-motion selection using outcomes from a frozen executor. Candidate motions are compared from the same state, with rewards reflecting both task appropriateness and successful execution.
C. Closed-Loop Execution-Grounded Reasoner Adaptation
- EG-GRPO uses physical outcomes from the frozen WAM to post-train the reasoner, without task-success labels, on a limited set of RoboLab tasks.The method addresses motions that may appear reasonable locally but be poorly realized by the executor or become inappropriate after execution errors.Source: C. Closed-Loop Execution-Grounded… · p. 4
- Each candidate motion is executed independently from the same initial state, and one highest-reward endpoint is randomly selected as the next decision state.The reasoner samples a new candidate group from that state.Source: C. Closed-Loop Execution-Grounded… · pp. 4–5
- Table I reports RoboLab-120 results under default and vague instructions using task success rate and subtask-progress score.The caption states that results are evaluated over ten runs per task.Source: Table I · p. 5
- The reward combines a binary score for whether a motion is an appropriate next step with a binary score for whether the executor completes it.Inappropriate motions receive zero reward regardless of execution outcome; appropriate motions receive the full reward only when successfully executed.Source: C. Closed-Loop Execution-Grounded… · p. 5
- Group-normalized advantages update the reasoner with a GRPO objective while the base reasoner and downstream executor remain frozen.Only the reasoner’s LoRA parameters are updated.Source: C. Closed-Loop Execution-Grounded… · p. 5
A. Experiment Setup
The experiments use atomic-motion segments derived from successful DROID demonstrations and evaluate Atomic-WAM on RoboLab tasks under default and vague instructions.
A. Experiment Setup
- 57k successful DROID demonstrations are converted into 369k atomic-motion segments for supervised training.Source: A. Experiment Setup · p. 5
- The evaluation covers 120 RoboLab tasks over 10 random seeds, with both default and vague instructions.Vague instructions leave more details implicit and place greater demands on semantic grounding.Source: A. Experiment Setup · p. 6
- Task success rate and subtask-progress score are reported across visual-grounding, procedural, and relational competencies.The benchmark includes color, semantics, and size; affordances, reorientation, and stacking; and conjunctions, counting, and spatial relationships.Source: A. Experiment Setup · p. 6
B. Main Results
Atomic-WAM leads the RoboLab comparisons under default and vague instructions, with smaller success-rate decline under vague instructions than Cosmos3-Nano-Policy. Its clearest competency gains occur on visual tasks, while procedural performance remains constrained by the underlying WAM.
B. Main Results
- Atomic-WAM achieves the best overall performance in both instruction settings: 39.6% SR and 53.7 subtask-progress under default instructions, and 31.6% SR and 46.7 under vague instructions.Its success rate drops 8.0 percentage points from default to vague instructions, compared with 16.2 points for Cosmos3-Nano-Policy.Source: B. Main Results · p. 6
- Visual-task success rises from 26.1% to 34.0% under default instructions and from 13.5% to 26.3% under vague instructions; vague relational success rises from 32.4% to 38.6%.Under default instructions, procedural success is 25.9%, below Cosmos3-Nano-Policy's 27.3%.Source: B. Main Results · p. 6
- Atomic-WAM's clearest advantage is on tasks requiring resolution of information implicit in instructions; procedural tasks remain constrained by the underlying WAM.The competency results show smaller procedural gains, including a default-instruction result below Cosmos3-Nano-Policy.Source: B. Main Results · p. 6
C. Analysis and Ablations
Analysis and ablations find that execution-success feedback improves reasoner adaptation, while stronger reasoning alone and motion-appropriateness rewards alone are insufficient. Contact-enriched supervision improves success, and the training setup trades reproduction fidelity for shorter training time.
C. Analysis and Ablations
- Atomic-motion descriptions raise success from 30.0% to 32.5%, and adding visible grasp points and gripper-position details raises it further to 36.7%.The passage links the added contact-level detail to reduced spatial ambiguity in grasping actions.Source: C. Analysis and Ablations · p. 6
- Reducing the global batch size to 768 cuts reproduction training time from more than 700 hours to approximately 80 hours, but the reproduced baseline reaches 30.0% versus 36.8% for the published checkpoint.The supervised Atomic-WAM reaches 36.7% at both 13,000 and 14,000 steps; at 18,000 steps, success falls to 30.0% for Atomic-WAM and 26.7% for the reproduced baseline.Source: C. Analysis and Ablations · p. 6
- The trained reasoner completes 23 tasks, while Codex completes 33 without backtracking and 50 with it; stronger reasoning capabilities alone provide only limited improvements.The larger benefits from backtracking demonstrate the necessity of execution feedback to adapt action selection to the WAM.Source: C. Analysis and Ablations · p. 7
- Adding execution success through EG-GRPO raises success to 39.2%, whereas GRPO without the execution reward reduces it from 36.7% to 34.2%.The authors note that motion appropriateness alone is insufficient when the WAM cannot reliably execute a selected motion.Source: C. Analysis and Ablations · p. 7
- The current EG-GRPO setup uses human judgments of execution success because evaluating every local rollout is costly in inference latency and token usage.Thirty updates correspond to 960 rollouts; the authors identify an efficient automatic execution evaluator as future work.Source: C. Analysis and Ablations · p. 7
V. CONCLUSION
Atomic-WAM translates task-level instructions into state-dependent atomic motions, supported by contact-enriched supervision and reasoner adaptation using physical outcomes. On RoboLab, it outperforms Cosmos3-Nano-Policy by larger margins under vague instructions, where task details are underspecified.
V. CONCLUSION
- The framework translates task-level instructions into state-dependent atomic motions before action generation.It combines temporally aligned, contact-enriched supervision with EG-GRPO adaptation using physical outcomes.Source: §V · p. 7
- Atomic-WAM achieves 39.6% and 31.6% success under default and vague instructions, improving over Cosmos3-Nano-Policy by 2.8 and 11.0 percentage points, respectively.The larger gain under vague instructions indicates particular effectiveness when task instructions are underspecified.Source: §V · p. 7