PaperlayerView exact revision on arXiv

Source-linked AI summary

Where Do Embodied Decisions Come From? Rethinking Latent and Explicit Reasoning

Yuan Lin, Ziyue Zhou, JinLong Zhao, Pei Liu, Haipeng Liu, Pan Zhou, Kun Zhan

arXiv:2609.34794v1cs.CE

TL;DR

The paper asks where explicit reasoning should sit in embodied policies, since CoT can harm stronger agents and its role in decision computation is not well resolved. It probes this with action-before-explanation generation and intervention-based analysis; across the reported tasks, DTE improves performance and increases reliance on perception-grounded computation. The authors conclude that explicit CoT may serve better as a training-time signal than as an inference-time mediator.

  • Problem

    Whether explicit CoT should mediate embodied action generation remains unresolved, despite its increasing use and capability-dependent effects in embodied agents.

    Source: §1 · p. 1

  • Method

    The paper introduces DTE, which predicts actions before explanations, and uses interventions and a TTA-trained, DTE-inference evaluation to probe decision pathways and ordering.

    Source: §1 · p. 2

  • Results

    DTE improves policy performance and increases reliance on perception-grounded computation across autonomous driving and robotic manipulation.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    For capable embodied policies, explicit CoT may be better used as a training-time signal than as an inference-time condition for action generation.

    Source: §1 · p. 3, §5 · p. 10

    The evaluation focuses on short-horizon tasks, omits alternative non-semantic or structured representations, and relies on proprietary driving data that may limit metric reproducibility.

    Source: §5 · p. 10

Abstract

from arXiv · show

Chain-of-Thought (CoT) reasoning is increasingly incorporated into Vision-Language-Action (VLA) models, yet it can degrade the performance of stronger embodied agents. We investigate this capability-dependent effect by distinguishing explicit reasoning from latent decision computation, i.e., perception-grounded computation that directly supports action prediction. Under the standard \textit{think-then-act} (TTA) paradigm, intervening on the generated CoT while fixing the visual input and model parameters causes a substantial performance collapse, with Navigation F1 dropping from 72.14% to 11.84%, demonstrating the strong influence of explicit reasoning on action generation. We then propose \textit{decide-then-explain} (DTE), which predicts actions before generating explanations, and introduce Visual Conditional Contribution (VCC) and Reasoning Conditional Contribution (RCC) to characterize the resulting decision process. Across autonomous driving and robotic manipulation, DTE consistently outperforms TTA and conventional \textit{no-CoT} baselines, while exhibiting greater reliance on perception-grounded computation. Further TTA-trained, DTE-inference experiments show that this benefit is not solely attributable to retraining under the new factorization. Our results suggest that for capable embodied agents, explicit CoT may be better used to shape decision computation during training rather than mediate action generation at inference time. Code: https://github.com/ocean-luna/openvla-decide-then-explain.

1 INTRODUCTION

The introduction asks where explicit reasoning belongs in embodied policies, given that CoT can help weaker models but harm stronger ones. It presents DTE, which generates actions before explanations, as a structural probe of decision pathways and reports improved performance alongside greater reliance on perception-grounded computation.

1 INTRODUCTION

  • Navigation F1 fell from 72.14% to 11.84% when the generated CoT was corrupted while visual observations and model parameters stayed fixed.The intervention supports a narrower conclusion: CoT content can materially alter behavior when inserted between observation and action, but it does not show CoT is the sole decision source.Source: §1 · p. 2
  • Because TTA and DTE differ in both training and inference factorization, their direct comparison does not isolate inference-time ordering; a TTA-trained, DTE-inference test probes ordering without DTE retraining.The authors frame DTE as a structural probe rather than a clean causal intervention on a fixed policy.Source: §1 · pp. 2–3
  • VCC and RCC quantify action-distribution changes under visual and reasoning interventions without equating either contribution with reasoning itself.When a pathway is structurally absent, its contribution is marked N/A rather than zero.Source: §1 · pp. 2–3
  • Random or irrelevant CoT controls and visual × CoT interventions test whether policy sensitivity reflects semantic reasoning content and how it interacts with visual evidence.The introduction treats CoT quality as distinct from simply having reasoning tokens present.Source: §1 · p. 3
  • CoT benefits weaker foundation models but can become neutral or harmful for stronger ones across autonomous driving and robotic manipulation.The paper also identifies CoT quality as an independent axis, so the effect is not reducible to a CoT-versus-no-CoT comparison.Source: §1 · pp. 1, 3
  • DTE places explanations after action generation and consistently improves policy performance while increasing reliance on perception-grounded computation.The authors treat DTE as a structural probe because its training and inference factorization both differ from TTA.Source: §1 · p. 3
  • The authors conclude that explicit CoT may be more useful as a training-time signal than as an obligatory inference-time intermediate between perception and action.Their framing distinguishes explicit CoT from perception-grounded computation without claiming to identify a unique latent reasoning mechanism.Source: §1 · p. 3

2 RELATED WORKS

Related work describes diverse VLA architectures that commonly generate reasoning tokens before actions, while leaving the functional role of those tokens underexplored. Work on explicit and latent reasoning raises a related mechanistic question, but prior VLA studies do not directly disentangle whether generated CoT drives decisions or decisions form primarily through latent visual representations.

2.1 VISION-LANGUAGE-ACTION MODELS

  • VLA systems span diverse architectures, yet most share an autoregressive formulation that generates reasoning tokens before actions, whose role in decision computation remains underexplored.The paper shifts the focus from architectural design to the structural role of reasoning in autoregressive decision generation.Source: §2.1 · p. 4

2.2 EXPLICIT AND LATENT REASONING IN LLMS AND MLLMS

  • Prior work distinguishes verbalized CoT from latent computation, but VLA studies have not directly disentangled whether generated CoT drives decisions or decisions form through latent visual representations.The paper identifies end-task performance as the predominant prior evaluation lens, leaving the relative roles of explicit and perception-grounded pathways insufficiently resolved.Source: §2.2 · p. 5

3 METHODOLOGY

The methodology uses generation order to probe whether decisions depend on explicit CoT or can be formed from observations and prompts, then quantifies modality contributions and attention allocation.

3.1 EMBODIED DECISION-MAKING FORMULATION

  • The formulation contrasts decisions conditioned on visual observations and prompts alone with decisions modeled jointly with an explicit reasoning sequence.The stated goal is to identify which pathway dominates decision-making, not to introduce a new decision architecture.Source: §3.1 · p. 5

3.2 GENERATION PARADIGMS AS STRUCTURAL PROBES

  • TTA places reasoning before the decision and conditions the decision on it, whereas DTE generates the decision first and removes CoT from its causal path.The DTE explanation is generated after the decision is committed.Source: §3.2 · pp. 5–6

3.3 METRICS FOR CONTRIBUTION

  • Contribution metrics measure decision-probability shifts under targeted modality ablations, complemented by attention ratios; all metrics are computed against ground-truth decisions.Source: §3.3 · p. 6

3.3.1 VISUAL AND REASONING CONDITIONAL CONTRIBUTION

  • VCC and RCC are dataset-mean log-probability shifts under visual and reasoning ablations, respectively; visual ablation uses a black frame and reasoning ablation equal-length padding.Positive visual contribution indicates increased decision confidence; reasoning contribution can support, be unused by, or conflict with the decision.Source: §3.3.1 · p. 6

3.3.2 ATTENTION RATIO

  • Attention ratios average attention across layers and heads for visual, prompt, and previously generated tokens; the three ratios sum to 1.They descriptively indicate attention allocation patterns across paradigms and complement the contribution metrics.Source: §3.3.2 · pp. 6–7

4 EXPERIMENTS

The experiments examine how CoT quality and generation order affect embodied decisions across autonomous driving and robotic manipulation. They find that DTE improves performance while reducing reliance on explicit reasoning at action time.

4.3.1 COT EFFECTIVENESS INVERSELY CORRELATES WITH FOUNDATION MODEL CAPABILITY

  • Across matched driving-model stages, CoT benefits weaker pre-aligned models but degrades stronger AD-general and object-specific models.The experiments include Navigation Decision and Target Path Generation; performance is evaluated on held-out splits.Source: §4.3.1 · p. 7, §4.1 · p. 7, §4.2 · p. 7

4.3.2 CAUSAL ROLE OF COT CONTENT IN DECISION-MAKING

  • Under TTA, corrupted CoT collapses Navigation F1 from 72.14% to 11.84%, while plausible-but-wrong CoT reduces it to 16.8%.The second intervention randomly inverts directional decision tokens in the model’s scene-grounded trace; both interventions also degrade Target Path Generation.Source: §4.3.2 · pp. 7–8, Table 2 · p. 8

4.3.5 GENERALIZATION ACROSS DOMAINS, MODEL CLASSES, AND TRAINING REGIMES

  • On robotic manipulation, standard think-then-act CoT raises baseline success from 62.0% to 67.1%, while decide-then-explain reaches 69.2% (std dev < 0.5%) on OpenVLA LIBERO-90.Source: §4.3.5 · p. 9, Table 6 · p. 9

4.3.3 LATENT REASONING DOMINATES WHEN IS REMOVED FROM THE CAUSAL PATH

  • With DTE, visual contribution rises, reasoning contribution falls to zero, and attention shifts toward visual tokens as performance improves over TTA.DTE predicts the action before its explanation, so CoT no longer conditions the action.Source: §4.3.3 · p. 8

4.3.4 COT BENEFIT IS QUALITY-DEPENDENT: STRONG MODELS REQUIRE HIGH-QUALITY REASONING

  • A concise, causally organized refined CoT recovers performance on the object-specific model, but the authors say this does not remove TTA’s structural vulnerability.The authors also characterize flawless-CoT curation as expensive, difficult to scale, and insufficient to resolve that vulnerability.Source: §4.3.4 · pp. 8–9, Table 3 · p. 8

4.3.5 GENERALIZATION ACROSS DOMAINS, MODEL CLASSES, AND TRAINING REGIMES

  • On three video-understanding benchmarks, decide-then-explain prompting consistently improves zero-shot performance in two general-purpose VLMs.The models are Qwen2.5-VL-7B and Video-R1; the paper presents this as supplementary evidence beyond its primary embodied focus.Source: §4.3.5 · p. 9

5 CONCLUSION AND LIMITATIONS

The conclusion argues that capable embodied agents benefit from deriving actions from latent visual representations rather than placing explicit CoT on the inference-time causal path. The authors limit this conclusion to short-horizon tasks and identify constraints on their operationalization and reproducibility.

5 CONCLUSION AND LIMITATIONS

  • The authors conclude that corrupted CoT can collapse TTA performance, whereas DTE removes reasoning reliance, increases visual contribution, and improves performance.They attribute residual gains over no-CoT baselines to training-time CoT supervision and describe CoT as an auxiliary training signal rather than an inference-time mediator.Source: §5 · p. 10
  • The evidence focuses on short-horizon tasks, leaving long-horizon planning dynamics unexplored.The authors also define latent reasoning strictly as the absence of CoT tokens and rely on proprietary driving data, which may limit quantitative reproducibility.Source: §5 · p. 10

A.1.1 VCC IMPLEMENTATION DETAILS

  • VCC’s visual ablation uses a black-frame placeholder; the authors interpret cv as association with decision confidence, not strictly causal contribution.They report consistent relative rankings across conditions while noting possible systematic bias from the out-of-distribution placeholder.Source: §A.1.1 · pp. 13–14

A.1.2 RCC IMPLEMENTATION DETAILS

  • RCC replaces reasoning content with equal-length padding, preserving positional and structural context to isolate the semantic contribution of generated reasoning.The authors use this post-generation recomputation because attention masking is infeasible in their autoregressive setting.Source: §A.1.2 · p. 14

A.1.3 ATTENTION RATIO: LIMITATIONS

The appendix qualifies attention ratios and explains how autoregressive factorization couples decision learning to reasoning, alongside dataset construction and evaluation details.

A.1.3 ATTENTION RATIO: LIMITATIONS

  • Attention ratios are limited because layer averaging treats abstraction levels equally, and attention indicates where the model looks rather than necessarily what information it uses.The authors present the ratios as complementary evidence for coarse shifts in computational focus, not as direct measures of causal information flow.Source: §A.1.3 · p. 14

A.1.4 OPTIMIZATION PERSPECTIVE: REASONING-CONDITIONED VS. DECOUPLED DECISION LEARNING

  • TTA trains decisions conditioned on reasoning tokens, whereas DTE optimizes decisions directly from visual observations and prompts; this derivation interprets optimization structure, not proof of internal interference.The authors frame this perspective as complementary to causal analyses and caution that it does not establish the internal source of performance degradation.Source: §A.1.4 · pp. 14–16

B.1 DATASET SETUP

  • The supervised dataset covers Navigation Decision and Target Path Generation, with visual observations, instructions, action outputs, and CoT annotations.Navigation Decision includes 175k samples across four sub-tasks; Target Path Generation includes 160k samples and five future waypoints, intent, a target box, and goal coordinates.Source: §B.1 · p. 16, §B.1.1 · p. 16, §B.1.2 · p. 17

B.1.3 WITHOUT COT DATASET

  • The without-CoT variant removes explicit reasoning annotations while keeping input instructions and action supervision unchanged.Improved CoT annotations retain decision-relevant observations, strengthen cue-to-action mapping, and remove redundant descriptions and advice.Source: §B.1.3 · p. 17, §B.1.4 · pp. 17–18

B.1.5 DECIDE-THEN-EXPLAIN DATASET

  • DTE places the decision before the reasoning process while keeping other content identical, so reasoning does not autoregressively condition decision generation.Source: §B.1.5 · p. 18

B.1.6 EVALUATION SET STATISTICS

  • Evaluation uses disjoint held-out test sets of approximately 5,000 Navigation Decision samples and 16,000 Target Path Generation samples.The authors describe small absolute improvements as practically meaningful given near-ceiling performance on structured tasks.Source: §B.1.6 · pp. 18–19

B.2 PROPRIETARY DATA AND REPRODUCIBILITY SCOPE

The capability-dependent reversal is clearest in proprietary driving experiments, while the DTE probe and recipe are presented as portable to public embodied settings.

B.2 PROPRIETARY DATA AND REPRODUCIBILITY SCOPE

  • The exact capability-dependent reversal is not claimed to be reproducible end-to-end on current open data, which lack matched staged checkpoints, task-specific action supervision, and reasoning annotations in one domain.Source: §B.2 · p. 19
  • The TTA-versus-DTE structural probe can be implemented by reordering outputs in a public VLA or VLM without changing model weights or the underlying task.With supervised reasoning data, the authors say the same factorization can also be trained directly.Source: §B.2 · p. 19
  • Public OpenVLA experiments on LIBERO-90 and zero-shot public VLM studies support portability, but do not replace the proprietary capability-scaling analysis.Source: §B.2 · p. 19

C.1 TRAINING DETAILS

The appendix reports the VLA training setup and a staged curriculum progressing from visual-language alignment to task-specific embodied decisions.

C.1 TRAINING DETAILS

  • Training uses 256 NVIDIA H20 GPUs, AdamW, an initial learning rate of 5 × 10−5, linear decay, and cross-entropy loss.Source: §C.1 · p. 20
  • The driving VLA combines a SigLIP-based visual encoder, Qwen2.5-7B decoder, and MLP projector, with all modules jointly optimized unless otherwise specified.Source: §C.1 · p. 20

C.2 TRAINING STAGES

  • The curriculum trains projector alignment first, then general driving capabilities, object-specific understanding, and task-oriented post-training for Navigation Decision and Target Path Generation.The general and object-specific stages use supervised fine-tuning; the task-oriented stage follows the vision-language-action datasets described in Appendix B.Source: §C.2 · p. 20

D.1 EVIDENCE OF LATENT REASONING

Without explicit CoT, the model still attends to task-critical visual entities; the appendix also details robotic OpenVLA and zero-shot VLM setups.

D.1 EVIDENCE OF LATENT REASONING

  • Without-CoT attention remains focused on the same task-critical visual entities as with-CoT attention, supporting the authors’ claim of visual reasoning without explicit traces.The authors interpret the preserved semantic focus as evidence that reasoning is implicitly encoded in shared representations.Source: §D.1 · pp. 20–21, Figure 2 · p. 21

D.2.1 MODEL ARCHITECTURE

  • The LIBERO-90 OpenVLA experiments use a Llama2-7B backbone, DinoV2 and SigLIP vision encoders, and an autoregressive action decoder adapted to generate reasoning and actions.Source: §D.2.1 · p. 21

D.2.3 DATASET AND PREPROCESSING

  • The OpenVLA study uses 3,917 successful demonstration trajectories across 90 tasks, with third-person RGB observations, instructions, 7-DoF actions, and structured CoT annotations.Source: §D.2.3 · p. 22

D.2.4 EVALUATION PROTOCOL

  • LIBERO-90 success is evaluated over 10 trials per task, with success requiring goal completion within 400 steps and stationary no-op episodes filtered out.Source: §D.2.4 · p. 22

D.3.3 PROMPTING STRATEGY

  • The paradigm shifts only inference-time token ordering: task description, answer format, and evaluation criteria stay constant, with no parameter updates; Qwen2.5-VL-7B uses its instruction-tuned variant without further fine-tuning.Source: §D.3.3 · p. 23, §D.3.2 · p. 23

E.1 CAUSAL INTERFERENCE EXPERIMENT DETAILS

The experiments test whether changing explicit CoT while holding other inputs fixed alters decisions, and whether that effect persists when the trace remains plausible but its directional conclusion is inverted. Both interventions substantially degrade performance, supporting a causal role for CoT under think-then-act inference.

E.1.1 SETUP

  • The controlled setup replaces the Object-Specific model’s generated <think> block with a fixed trace while keeping visual observations, task instructions, and parameters identical.The injected Navigation Decision trace was sampled from the model’s own CoT outputs.Source: §E.1.1 · p. 23, §E.1.2 · p. 23

E.1.2 CORRUPTED COT CONTENT

  • The fixed traces prescribe a right turn or right-forward steering regardless of the actual scene or target, making their conclusions inconsistent with some inputs.The Target Path Generation trace describes a static scene and directs the vehicle toward a specific target location.Source: §E.1.2 · pp. 23–24

E.1.3 WHY THIS IS A VALID CAUSAL TEST

  • Navigation F1 fell from 72.14% to 11.84% under fixed-trace intervention, while Target Path Generation worsened from ADE 1.83 m to 2.00 m and FDE 3.17 m to 3.48 m.The intervention changed only CoT content, leaving visual observations, instructions, and model parameters unchanged.Source: §E.1.3 · p. 24

E.1.4 PLAUSIBLE-BUT-WRONG COT INTERVENTION

  • The second intervention preserves scene descriptions and spatial analysis but inverts only directional decision tokens; coverage is near-complete for navigation and partial for path generation.This design addresses the alternative that the model merely copies directional words rather than follows the reasoning conclusion.Source: §E.1.4 · p. 24
  • Plausible directional inversion reduced Navigation F1 to 16.8% and worsened Target Path Generation to ADE 1.88 m and FDE 3.27 m.The smaller degradation is consistent with scene descriptions that remain visually consistent and partial intervention coverage for path generation.Source: §E.1.4 · p. 25

F.1 LATENCY ANALYSIS OF GENERATION PARADIGMS

Decide-then-explain allows actions to be returned before explanations, creating an early-truncation latency advantage while retaining an option for full explanations. Its reported navigation gain over the no-CoT baseline is consistent with CoT supervision benefiting decision learning indirectly.

F.1 LATENCY ANALYSIS OF GENERATION PARADIGMS

  • When both paradigms generate identical action and reasoning tokens, their full-output latencies are theoretically equivalent because total token counts match.Autoregressive generation latency scales linearly with total output tokens.Source: §F.1 · p. 25
  • Decide-then-explain permits stopping after the action, whereas think-then-act must finish reasoning first; action latency is O(Td) versus O(Tr + Td).Tr denotes reasoning tokens and Td decision tokens.Source: §F.1 · p. 25
  • Early truncation yields reported speedups of approximately 44× for autonomous driving and 26× for robotic manipulation.The latency analysis attributes these gains to producing decision tokens before the longer reasoning trace.Source: §F.1 · p. 25
  • The same decide-then-explain model can provide fast action-only output or a complete explanation without changing its weights.Think-then-act cannot suppress its reasoning phase in this way without switching to a without-CoT mode.Source: §F.1 · p. 26

G.1 INDIRECT PARAMETER LEARNING: CONSOLIDATED COMPARISON

  • Navigation F1 was 74.50% with decide-then-explain versus 73.49% without CoT, while its reasoning contribution coefficient was approximately zero.The passage attributes the remaining CoT benefit to training-time gradient updates that shape parameters and latent representations.Source: §G.1 · p. 26

G.2 TASK-WISE DIFFERENCES IN CONTRIBUTION METRICS

Contribution metrics differ by task: navigation relies more on explicit reasoning under think-then-act, while target path generation relies more on perception-grounded computation.

G.2 TASK-WISE DIFFERENCES IN CONTRIBUTION METRICS

  • Navigation Decision shows high RCC and low VCC under think-then-act, while Target Path Generation maintains comparatively high VCC across paradigms.The passage links these patterns to navigation’s scene-level semantic interpretation and waypoint generation’s precise spatial grounding in visual features.Source: §G.2 · pp. 26–27

H.1 IMPROVED COT: FORMAT ANALYSIS AND DESIGN PRINCIPLES

The appendix contrasts free-form, scene-centric CoT with an expert-guided format designed to retain task-relevant visual evidence and connect it directly to decisions. Corpus statistics report that the improved format reduces textual noise and length while retaining task-relevant scenes.

H.1.1 SCENE EXTRACTION AND GENERATION PIPELINE

  • A shared scene-recognition stage extracts task-relevant key scenes from each image to ground both CoT-generation pipelines in the same visual cues.Those extracted cues are supplied as structured perceptual inputs downstream.Source: §H.1.1 · p. 27
  • Original CoT uses the instruction, raw image, and extracted scenes for unconstrained scene analysis and decision reasoning, without expert guidance or structural constraints.Its resulting trace is a relatively comprehensive scene description followed by reasoning.Source: §H.1.1 · p. 27
  • Improved CoT derives concise expert annotations from a scene-level spatial map and rule-based reasoning, then guides a language model to express the resulting perception-to-decision logic.The annotations preserve links from scene perception through scene association to decision reasoning.Source: §H.1.1 · p. 27
  • The expert scaffold constrains generated CoT to remain causally aligned with task-relevant perceptual evidence instead of narrating and reasoning from scratch.Figure 3 contrasts verbose, scene-centric Original CoT with concise, causal Improved CoT; the former includes task-irrelevant features that can interfere with decision gradients.Source: §H.1.1 · p. 27, Figure 3 · p. 28

H.1.2 QUANTITATIVE COMPARISON

  • The corpus comparison measures average CoT character count, task-relevant scene retention, irrelevant scenes per sample, and generic safety-prompt occurrence.A language-model parser identifies scene and safety-prompt categories across the training set.Source: §H.1.2 · p. 27
  • Improved CoT preserves full task-relevant scene coverage while reducing irrelevant scene mentions from 4.2 to 0.3 per sample, eliminating generic safety prompts, and shortening CoT by approximately 42%.Table 10 reports corpus-level measures of scene retention, irrelevant-scene counts, safety-prompt frequency, and length.Source: §H.1.2 · pp. 27–28, Table 10 · p. 28
  • Original CoT catalogs environmental details such as arrows, zone labels, background objects, and generic descriptions alongside redundant safety prompts.Figure 3 characterizes this verbose, scene-centric style as introducing textual noise that can interfere with decision gradients.Source: §H.1.2 · p. 28
  • Improved CoT retains only observations directly influencing decisions, removes irrelevant descriptions and generic safety prompts, and explicitly maps observation to interpretation to decision.The format’s central contrast is task-oriented causal structure versus panoramic scene description.Source: §H.1.2 · pp. 28–29
Loading 2609.34794v1…