PaperlayerView exact revision on arXiv

Source-linked AI summary

ARC: A Reasoning Recipe for Robot Foundation Models

Gokul Puthumanaillam, Tao Sun, Elie Aljalbout, Moritz Reuss, Zhaoshuo Li, Fabio Ramos, Ankit Goyal, Jenai Xuning Yang

arXiv:2610.12386v1cs.ROcs.AI

TL;DR

Robot foundation models are commonly improved with larger models, more demonstrations, and costly training, motivating ways to improve existing models without those additions. ARC adds action-grounded causal traces generated from demonstrations and adapts pretrained models to use them for control. The adapted models improve benchmark performance, with gains up to 50.0 p.p. on RoboLab-Reasoning-50.

  • Problem

    Improving robot foundation models commonly relies on more robot data, larger models, and costly foundation-scale training, raising the question of how to improve existing models without these additions.

    Source: §1 · p. 2

  • Method

    ARC automatically generates action-grounded causal traces from demonstrations and adapts pretrained robot foundation models to use them for control.

    Source: §1 · p. 2

  • Results

    ARC establishes new state of the art on RoboLab-120 and MolmoSpaces and improves RoboLab-Reasoning-50 task success by 46.8 p.p. for π0.5 and 50.0 p.p. for Cosmos3-Nano-Policy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results show that pretrained robot foundation models can gain zero-shot task performance without additional robot demonstrations or foundation-scale training.

    Source: §6 · p. 10

    ARC depends on faithful reasoning from its VLM; weaker reasoning can misinterpret tasks or propose inappropriate actions that carry into execution.

    Source: F Limitations of ARC · p. 36

Abstract

from arXiv · show

The prevailing approach to improving robot foundation models (RFMs) relies on larger models, more robot demonstrations, and costly training at scale. We show that there exists an effective and efficient complementary approach: the right reasoning recipe can substantially improve the zero-shot task performance of existing state-of-the-art RFMs. We refer to this recipe as ARC. It consists of three key ingredients: a reasoning trace, a scalable automatic labeling pipeline, and a strategy for adapting pretrained RFMs to use these traces for control. First, we find that effective reasoning traces should be grounded in the robot's next action and explain its causal structure: why the action is appropriate and what effect it should produce. Second, we show that these traces can be generated automatically from existing demonstrations, enabling us to construct ARC-Trace-DROID from DROID without collecting new robot data. Third, we show how state-of-the-art VLAs such as $\pi_{0.5}$ and WAMs such as Cosmos3-Nano-Policy can learn to use these traces for control, with fine-tuning and inference tailored to each model's architecture and capabilities. Using ARC, we obtain gains in zero-shot RFM performance that, to our knowledge, are unprecedented without additional robot demonstrations or foundation-scale training. The adapted models establish a new state of the art on RoboLab-120 and MolmoSpaces, with gains of up to 50 percentage points on RoboLab-Reasoning-50. On real robots, ARC improves $\pi_{0.5}$'s task success by 82.2 percentage points. Project website: https://arc-robot-reasoning.github.io/

1 Introduction

ARC proposes improving pretrained robot foundation models through action-grounded causal reasoning, automatically labeled from existing demonstrations and used to train models for reasoning-guided control. Its evaluation reports substantial gains across simulation and real-robot benchmarks, alongside training and inference efficiency improvements.

1 Introduction

  • Effective control traces should explain why an action is appropriate and its expected effect while specifying the concrete next action.The traces connect the current state to the next action and resulting state.Source: §1 · p. 2
  • ARC automatically labels existing demonstrations with causal traces aligned to action chunks, creating ARC-Trace-DROID by relabeling DROID without new robot data.The pipeline addresses the fact that demonstrations record observations and actions but typically omit reasoning behind decisions.Source: §1 · p. 2
  • Pretrained RFMs can be fine-tuned to condition behavior on reasoning traces while retaining existing control capabilities.The same traces and training data are used across models, with fine-tuning and inference tailored to each architecture and capability.Source: §1 · p. 2
  • On RoboLab-Reasoning-50, ARC improves task success by 46.8 p.p. for π0.5 and 50.0 p.p. for Cosmos3-Nano-Policy.On real robots, gains reach 82.2 p.p. on reasoning and long-horizon tasks.Source: §1 · p. 2
  • ARC establishes new state of the art on RoboLab-120 and MolmoSpaces, with MolmoSpaces gains of 27.2 p.p. for π0.5 and 18.6 p.p. for Cosmos3-Nano-Policy.Across all three RoboLab-120 settings, the adapted models rank first and second.Source: §1 · p. 2
  • Intermediate language alone provides limited improvement; larger gains arise from combining causal reasoning with action-grounded training, while efficiency also improves.Cosmos3-Nano-Policy reaches baseline success with 4.3× fewer training updates; π0.5 and Cosmos3-Nano-Policy match their base policies with 10× and 2× fewer denoising steps, respectively.Source: §1 · p. 2

2 Related Work

Prior robotics research uses reasoning to select steps, generate programs, revise plans, or guide action prediction, while other approaches provide spatial, motion, or predictive representations. ARC focuses on language-based reasoning supervision that can be shared across robot foundation models without architecture changes.

2 Related Work

  • Robotics reasoning has been used both to determine robot steps and to teach policies which information matters for action prediction.Examples include skill selection, program generation, plan revision, and reasoning annotations for policy action prediction.Source: §2 · p. 3
  • Prior control guidance includes spatial representations, keypoints and geometric constraints, motion-based trajectories, and predictions of future scene evolution.These approaches provide explicit spatial or temporal targets, with control connections depending on the representation.Source: §2 · p. 3
  • ARC focuses on reasoning supervision shareable across RFMs without additional architecture changes, unlike guidance approaches whose control connection depends on representation.Prior spatial and temporal methods connect to control through mechanisms such as optimization, intermediate tokens, or image inputs.Source: §2 · p. 3
  • Natural language can express task conditions, spatial relationships, and intended outcomes without committing to robot coordinates, and many RFMs already support it.Prior work uses language for fine-grained motions, semantic subtasks, and reasoning about plans, objects, and robot state.Source: §2 · p. 3

3 Preliminaries and Problem Statement

The paper formalizes robot tasks as partially observed interactions driven by an instruction, observations, and action chunks. It frames reasoning as an intermediate representation before action prediction and seeks to improve task success without new architecture, foundation-scale pretraining, or a large additional robot dataset.

3 Preliminaries and Problem Statement

  • A task is an instruction-conditioned interaction in which the robot observes its environment and executes actions, potentially in multi-action chunks.A demonstration records the instruction and observation-action trajectory.Source: §3 · p. 3
  • A standard policy predicts actions from task and interaction history, whereas a reasoned policy first forms an intermediate representation relevant to the current decision.The formulation represents reasoning generation and action prediction as conditional distributions.Source: §3 · p. 4
  • The objective is to learn a reasoned policy that improves task success while retaining existing control abilities, using demonstrations with observations and actions but no reasoning supervision.The stated goal excludes a new model architecture, foundation-scale pretraining stage, or another large robot dataset.Source: §3 · p. 4

ARC Framework

ARC combines action-grounded causal traces, automatic relabeling of existing demonstrations, and fine-tuning that conditions pretrained models on those traces for control. Its traces connect current task-relevant facts to the intended effect and corresponding action.

4.1 Action Grounded Causal Reasoning

  • Arc-Trace represents each action chunk with seven elements linking task-relevant state and cause to consequence, intended effect, action, avoidance, and completion.The traces explain an action through the effect it is intended to cause, rather than describing the action in isolation.Source: §4.1 · pp. 4–5
  • With Arc traces, the action expert attends more to natural-language reasoning, especially spans connecting the current condition, intended effect, and action.Attention is a diagnostic, not proof of behavioral dependence; the authors interpret the change as evidence that traces carry information used by the action expert.Source: §4.1 · pp. 4–5

4.2 ARC-Trace: Reasoning Supervision at Scale

  • ARC-Trace automatically relabels DROID demonstrations, producing approximately 75K episodes and 1.2M action-aligned frames without collecting new robot trajectories or manually annotating actions.The pipeline uses the instruction, video, and action sequence to recover supervision; it generates an episode description, state checks, and keyframe-aligned reasoning traces.Source: §4.2 · p. 5

4.3 Fine-Tuning for Reasoning-Guided Control

  • Fine-tuning conditions action prediction on aligned traces while retaining each model’s flow-matching objective, with implementation tailored to VLA and WAM architectures.The VLA uses a two-layer encoder for trace embeddings, while the WAM’s generator attends to trace tokens without architectural changes.Source: §4.3 · pp. 5–6
  • For π0.5, the counterfactual loss uses counterfactual samples and action noise to provide negative supervision without alternative-action demonstrations.The expectation in Lcf covers counterfactual samples and action noise, whereas Lpred uses factual samples only.Source: §4.3 · p. 6
  • At inference, frozen ARC-fine-tuned models use externally generated traces from task instructions and current camera observations, refreshed during execution.The external VLM is used because the models’ built-in language components are less reliable at generating traces than at using them for control.Source: §4.3 · p. 6

5 Experiments

Across RoboLab-120 and MolmoSpaces, ARC improves both base models without benchmark-specific training; the experiments also report stronger reasoning-task performance, hardware transfer, robustness, and efficiency.

5.1 Quantitative Results: Simulation and Hardware Benchmarks

  • ARC substantially improves both models across RoboLab and MolmoSpaces, with π0.5 outperforming stronger pretrained policies and Cosmos3-Nano-Policy improving without benchmark-specific training.The reported comparisons cover both benchmarks and different model architectures and starting performance levels.Source: Table 1 · p. 7, Table 2 · p. 7, §5.1 · p. 7
  • On paired hardware tasks, π0.5+Arc substantially improves π0.5’s task success without hardware-specific fine-tuning.The evaluation recreates 28 RoboLab-Reasoning-50 tasks on DROID hardware using the same checkpoints as in simulation.Source: §5.1 · p. 7, Table 3 · p. 7

5.2 Key Findings and Ablations

  • ARC improves execution on long-horizon, dynamic, and reasoning-intensive tasks, including step sequencing, adapting to scene changes, discovering prerequisites, and correcting failed actions.The reported hardware rollouts and reasoning-focused evaluations show these behaviors during execution.Source: §5.2 · p. 8, Figure 5 · p. 8
  • ARC-finetuned policies maintain comparable success across vague, default, and specific instructions, while base policies deteriorate as instructions become less explicit.The trace interprets the instruction in the scene context and identifies the intended effect of the next action.Source: §5.2 · p. 9, Figure 6 · p. 9
  • π0.5+Arc matches its ten-step base policy with one Euler step, while Cosmos3-Nano+Arc matches its four-step base policy with two UniPC steps.Cosmos3-Nano still loses performance at one step.Source: §5.2 · p. 9, Figure 7 · p. 9
  • On Cosmos3-Nano-Policy, ARC reaches the instruction-only baseline’s 10K-iteration performance with approximately 4.3× fewer training updates.The study uses the model’s open training implementation, and ARC continues to improve with further training.Source: §5.2 · p. 9, Figure 8 · p. 9
  • Performance remains comparable across the evaluated external VLM trace generators, while ablations indicate that causal context and an action implication play complementary roles.The action implication connects the state change to a physical step; the causal account explains why that change matters.Source: §5.2 · p. 9, Figure 9 · p. 10
  • Full fine-tuning lets prediction errors reshape model features, whereas action-head-only training leaves pretrained language features fixed.The authors interpret this as helping the backbone distinguish conditions and intended effects that call for different actions.Source: §5.2 · pp. 9–10

6 Conclusion

ARC improves zero-shot task execution in pretrained RFMs without new robot demonstrations or repeated foundation-scale training. Applied to π0.5 and Cosmos3-Nano-Policy, it establishes new state of the art on RoboLab-120 and MolmoSpaces and improves robustness and efficiency.

6 Conclusion

  • ARC establishes new state of the art on RoboLab-120 and MolmoSpaces using pretrained RFMs without new robot demonstrations or repeated foundation-scale training.The conclusion also reports improved robustness to ambiguous and underspecified instructions and increased training and inference efficiency.Source: §6 · p. 10
  • The paper identifies policies that reliably generate their own causal traces and extending the recipe across embodiments as promising future directions.Source: §6 · p. 10
  • The conclusion characterizes grounded causal language as a cheap, high-leverage way to improve RFMs without rebuilding them.Source: §6 · p. 10

AI Use Statement

Vision-language models generated action-grounded causal annotations for ARC-Trace and reasoning traces for evaluation. The authors state that they take responsibility for the final manuscript, reported results, and AI-assisted content.

AI Use Statement

  • Vision-language models automatically generated action-grounded causal annotations for ARC-Trace.Source: AI Use Statement · p. 11
  • During evaluation, vision-language models generated reasoning traces from task instructions and camera observations.The statement says the models and prompting procedures are described in the methodology and appendix.Source: AI Use Statement · p. 11
  • The authors take responsibility for the final manuscript, reported results, and all AI-assisted content.Source: AI Use Statement · p. 11

Reproducibility statement

The reproducibility materials direct readers to the trace-construction pipeline, model-specific training and inference details, and evaluation protocols. The appendix also provides recipes and supporting materials, while models, code, and the ARC-Trace dataset are slated for release after review.

Reproducibility statement

  • The reproducibility statement points to Appendix A.1 for ARC-Trace construction and labeling prompts, and Appendices B.1–B.3.1 for model training configurations, hyperparameters, inference prompts, and deployment procedures.Source: Reproducibility statement · p. 11
  • Evaluation protocols, benchmark construction, baseline implementations, and hardware setup are described in Appendices C.1.3, C.3, and C.2.Source: Reproducibility statement · p. 11

Appendix

  • The appendix provides detailed labeling, training, and inference recipes, hyperparameters, prompts, qualitative results, and a discussion of limitations.Source: Appendix · p. 16
  • The models, code, and ARC-Trace dataset are to be released after the review period, and experiment videos are available on the project website.Source: Reproducibility statement · p. 11, Appendix · p. 16

A.1 ARC-Trace: Data Labeling Details

This section describes how ARC-Trace labels DROID demonstrations and how π0.5 and Cosmos3-Nano-Policy use the traces in training and inference. It also outlines the reasoning-task examples, benchmark categories and protocols, policy baselines, and real-robot evaluation setup.

A.1 ARC-Trace: Data Labeling Details

  • From 95,658 DROID training episodes, ARC labels approximately 74,740 (78.1%) and produces approximately 1.2 million frames through narration/keyframe selection, predicate generation, and trace composition.Episode context helps explain progress beyond what an isolated image shows; annotations preserve frame and timestamp alignment, and full videos are used offline rather than as deployed policy inputs.Source: §A.1 · p. 17

B.1 VLA Fine-Tuning Details

  • For π0.5, trace tokens are separately encoded and appended to image-and-instruction context, allowing the action expert to condition on them alongside noisy action chunks.The trace encoder uses two transformer blocks; joint training applies flow matching to factual examples and a counterfactual penalty to counterfactual examples.Source: §B.1 · pp. 20–21

B.2 WAM Fine-Tuning Details

  • For Cosmos3-Nano-Policy, the trace is appended to the instruction for the reasoner, whose representations condition the generator’s action and future-observation predictions.Training uses the ground-truth trace both as generator conditioning and as next-token supervision for the reasoner; prediction losses also update reasoner representations, while the counterfactual objective is omitted.Source: §B.2 · pp. 21–22

B.3 Inference

  • At evaluation, an external VLM generates a current-observation-grounded trace, while the frozen ARC-fine-tuned policy uses it with the instruction and observations to produce actions.Responses are checked for format and length, and rejected responses leave the previous accepted trace unchanged.Source: §B.3 · p. 23

B.3.1 Real-Time Deployment

  • Real-time deployment runs trace generation asynchronously while the robot executes 15-action chunks at 15 Hz, using the latest validated trace without blocking action generation.Qwen3.6-35B-A3B-FP8 is served locally; FP8 inference and prefix caching reduce computation and memory costs.Source: §B.3.1 · p. 23

C.1.1 RoboLab-120

  • RoboLab-120 evaluation covers all 120 tasks with 10 episodes per task for vague, default, and specific instructions, with frozen models and no benchmark training.That is 1,200 episodes per model for each instruction setting, following the released benchmark tasks, scenes, and success criteria.Source: §C.1.1 · p. 24

C.1.2 MolmoSpaces

  • MolmoSpaces evaluation uses the same checkpoints on all nine reported manipulation tasks under the official protocol, preserving its released environments and success criteria.Other baseline-policy results are taken from the corresponding benchmark reports and official leaderboards.Source: §C.1.2 · p. 24

C.1.3 RoboLab-Reasoning-50

  • RoboLab-Reasoning-50 introduces 50 tasks spanning target inference, multi-step execution, discovery, and exclusion reasoning; 34 leave the target unnamed and 16 name it.The benchmark combines 27 tasks adapted from real-world evaluation with 23 simulation-only tasks; evaluation uses 10 episodes per task and binary success based on final-scene predicates.Source: §C.1.3 · p. 24
  • The task examples cover long-horizon instructions, discovery of hidden objects, and negation-based exclusions, including sorting objects while omitting specified items.Examples include revealing and moving an object hidden under a plate, and sorting items into bins while leaving out a named object.Source: §C.1.3 · pp. 25–26

C.2 RoboLab-Reasoning-Hardware

  • The DROID embodiment pairs a seven-degree-of-freedom Franka Panda arm and Robotiq gripper with paired hardware and simulation tasks, evaluated for three episodes per task without hardware-specific fine-tuning.The task pairs use the same instructions, and the simulation-evaluated checkpoints are reused on hardware.Source: §C.2 · p. 26

Category: Long horizon 6 tasks

C.3 Evaluated Policies

  • π0.5’s baseline uses the DROID joint-position checkpoint with no added reasoning and ten Euler steps, while the explicit-subtask baseline feeds GPT-5.5-generated plans to the unchanged policy.The plan contains at most 12 coarse subtasks; a client-side wrapper replaces the instruction with the active subtask, and GPT-5.5 checks completion every 30 simulation steps. The plan is generated once per episode, with no replanning, and the external VLM supplies prompts and completion judgments rather than robot actions.Source: §C.3 · p. 32
  • Cosmos3-Nano-Policy is evaluated from its DROID checkpoint as a world-action model with a vision-language reasoner and diffusion generator that jointly predicts future observations and actions.The two components have separate parameters and are connected through attention at every layer.Source: §C.3 · p. 32

C.4 Detailed RoboLab-120 results.

The detailed RoboLab-120 results compare policies under vague, default, and specific instructions, reporting success rates and additional movement metrics. Figure 20 ranks leading policies and marks ARC policies and their gains over base policies.

C.4 Detailed RoboLab-120 results.

  • Success rates, end-effector speed, and EE SPARC are reported alongside available uncertainty estimates.Source: §C.4 · p. 32
  • Figure 20 ranks the top 11 policies, including both ARC variants, under vague, default, and specific instructions.The three instruction settings are shown from top to bottom; green rows mark ARC policies, and +values denote success-rate gains over their respective base policies.Source: §C.4 · p. 32, Figure 20 · p. 33

D Training Hyperparameters and Configurations.

The appendix presents training and model-configuration tables for the π0.5-DROID and Cosmos3-Nano-Policy ARC implementations.

D Training Hyperparameters and Configurations.

E Additional Qualitative Results

Additional qualitative hardware results illustrate ARC traces accompanying multi-step object transfers. The examples include container-count goals, exclusions, and rearrangement constraints.

E Additional Qualitative Results

  • In a transfer example, the banana is temporarily moved from the bowl to make room for plate contents, then the marker is placed in the bowl.Source: E Additional Qualitative Results · p. 35
  • A two-items-per-container task proceeds through transfers that leave the orange bowl with two items before moving on to the black bowl.The trace describes returning the banana to the cleared plate, adding a blue block to the orange bowl, and then transferring a green block toward the bowl.Source: E Additional Qualitative Results · p. 35
  • For a crate task excluding green objects, the trace directs eligible items into the crate while keeping the green pear and block outside.The sequence identifies the orange, banana, and blue block as eligible objects.Source: E Additional Qualitative Results · p. 35
  • Other examples show transferring fruit into a bowl while leaving the can and wooden block behind.Source: E Additional Qualitative Results · p. 36

F Limitations of ARC

This section documents ARC’s limitations around reasoning quality, incomplete observations, and the cost and staleness of external reasoning, including its video-annotation prompts, frame-by-frame reasoning, and inference-time use. It describes how the reasoner is constrained to a short formatted paragraph and how the policy uses a fixed bootstrap description while awaiting one.

F Limitations of ARC

  • A weaker or mistaken reasoner can misread a task or propose an inappropriate action, and stronger language grounding may carry errors into execution.Comparable performance across evaluated VLMs does not establish robustness to arbitrary reasoning errors; stronger or task-adapted reasoners and checks against observations may improve reliability.Source: F Limitations of ARC · p. 36
  • Occlusions can hide relevant scene details, causing an uncertain grasp or placement to be mistaken for a completed step and corrupting subsequent reasoning.Short observation histories, clearer viewpoints, explicit uncertainty, and requesting another observation are proposed improvements.Source: F Limitations of ARC · p. 36
  • External reasoning adds computation, memory use, and latency; asynchronous deployment can also leave the policy using an outdated trace while new reasoning is generated.The trade-off is especially important when scenes change rapidly; distillation, selective trace refresh, and freshness checks are proposed mitigations.Source: F Limitations of ARC · p. 36

G Prompts

  • The appendix reproduces prompts for video annotation, frame-by-frame reasoning with predicates, and inference-time reasoning, with episode-specific values supplied in braced placeholders.Accompanying explanations are not part of the prompts.Source: G Prompts · p. 36

G.1 Video annotation

  • Video annotation prompts require a dense, high-fidelity timeline that records contact, object motion, lift-off, corrections, and uncertainty, alongside keyframes of meaningful states and transitions.The prompt cautions against inferring a clean lift or placement when the wrist-camera view is partial or ambiguous.Source: §G.1 · pp. 37–38

G.2 Frame-by-frame reasoning with predicates

  • Frame-by-frame reasoning uses predicates as symbolic anchors and generates causal reasoning for each current keyframe while maintaining episode continuity without inferring from future frames.The required structure covers perceived state, cause, possible consequence, chosen effect, action implication, forbidden effect, completion, predicates, and visual evidence.Source: §G.2 · p. 39

G.3 Inference-time reasoner prompts

  • At inference time, the reasoner receives current exterior and wrist images and the task instruction, returns a grounded actionable paragraph, and passes it to the policy only after validation.The prompt directs fresh scene assessment, visual grasp and completion checks, and forward-looking recovery; rejected replies leave the policy using its last accepted paragraph.Source: §G.3 · pp. 40–42
  • The system prompt requires the exact opening “The robot sees,” four to seven sentences, and fewer than 900 characters, with additional content and formatting rules.These acceptance checks are distinct from the generation instructions.Source: §G.3 · p. 42
  • Until a valid reasoner paragraph arrives—or when the server starts with --no-reasoner—the action policy receives a fixed bootstrap paragraph.The example describes an idle, open gripper and directs the robot to center its fingers above the task object before descending, while avoiding collisions or dragging; the task remains incomplete until the object reaches its goal.Source: §G.3 · pp. 42–43

Related papers

Loading 2610.12386v1…