PaperlayerView exact revision on arXiv

Source-linked AI summary

DualCoT-VLA: Visual-Linguistic Chain of Thought via Parallel Reasoning for Vision-Language-Action Models

Zhide Zhong, Junfeng Li, Junjie He, Haodong Yan, Xin Gong, Guanyi Zhao, Yingjie Cai, Jiantao Gao, Xu Yan, Bingbing Liu, Yingcong Chen, Liuqing Yang, Haoang Li

arXiv:2603.22280v1cs.CVcs.RO

TL;DR

VLA models and existing CoT approaches struggle to combine fine-grained spatial perception with long-horizon planning while avoiding autoregressive latency and cumulative errors. DualCoT-VLA addresses these gaps with visual and linguistic CoT streams implemented through parallel learnable query tokens, achieving state-of-the-art results on LIBERO and RoboCasa GR1 and in real-world deployments.

  • Problem

    Existing CoT-based VLA models use isolated modalities and step-by-step autoregressive decoding, limiting combined spatial-planning reasoning and causing latency and cumulative errors.

    Source: §1 · p. 1, §2.2 · p. 4

  • Method

    DualCoT-VLA combines visual CoT for low-level spatial understanding and linguistic CoT for high-level planning through parallel learnable query tokens and single-step latent reasoning.

    Source: §1 · pp. 1–2, §5 · p. 12

  • Results

    DualCoT-VLA achieves state-of-the-art performance on LIBERO and RoboCasa GR1, with evaluations also conducted on real-world platforms.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The dual-stream design provides both low-level 3D spatial perception and high-level logical task planning while bypassing autoregressive reasoning.

    Source: §5 · p. 12

Abstract

from arXiv · show

Vision-Language-Action (VLA) models map visual observations and language instructions directly to robotic actions. While effective for simple tasks, standard VLA models often struggle with complex, multi-step tasks requiring logical planning, as well as precise manipulations demanding fine-grained spatial perception. Recent efforts have incorporated Chain-of-Thought (CoT) reasoning to endow VLA models with a ``thinking before acting'' capability. However, current CoT-based VLA models face two critical limitations: 1) an inability to simultaneously capture low-level visual details and high-level logical planning due to their reliance on isolated, single-modal CoT; 2) high inference latency with compounding errors caused by step-by-step autoregressive decoding. To address these limitations, we propose DualCoT-VLA, a visual-linguistic CoT method for VLA models with a parallel reasoning mechanism. To achieve comprehensive multi-modal reasoning, our method integrates a visual CoT for low-level spatial understanding and a linguistic CoT for high-level task planning. Furthermore, to overcome the latency bottleneck, we introduce a parallel CoT mechanism that incorporates two sets of learnable query tokens, shifting autoregressive reasoning to single-step forward reasoning. Extensive experiments demonstrate that our DualCoT-VLA achieves state-of-the-art performance on the LIBERO and RoboCasa GR1 benchmarks, as well as in real-world platforms.

1. Introduction

DualCoT-VLA addresses the limits of single-modal and autoregressive reasoning in VLA models by combining visual and linguistic CoT with parallel reasoning. It is evaluated on simulation benchmarks and real-world platforms, achieving state-of-the-art performance on LIBERO and RoboCasa GR1.

  • VLA models struggle with long-horizon logical planning and precise spatial perception, while existing CoT methods typically address only one modality.Source: §1 · p. 1
  • DualCoT-VLA combines visual CoT for low-level spatial cues with linguistic CoT for high-level logical plans.The two streams are designed to provide comprehensive multimodal reasoning.Source: §1 · pp. 1–2
  • Two sets of learnable CoT query tokens enable implicit parallel reasoning in a single forward pass instead of autoregressive decoding.The mechanism avoids the latency and compounding errors associated with step-by-step inference.Source: §1 · p. 2
  • The method is evaluated across simulation benchmarks and real-world platforms.The evaluation covers both LIBERO and RoboCasa GR1, alongside real-world robot experiments.Source: §1 · p. 2
  • DualCoT-VLA achieves state-of-the-art performance on LIBERO and RoboCasa GR1 and transfers task planning and 3D spatial perception to real-world environments.Source: §1 · p. 2

2. Related Work

Prior VLA reasoning methods separate linguistic and visual CoT, limiting their ability to combine semantic planning with spatial perception, and often decode reasoning autoregressively. DualCoT-VLA uses parallelized CoT query tokens to internalize both forms of reasoning in one forward pass.

Single-Modal Chain-of-Thought in VLA models

  • Linguistic CoT supports high-level logic but often lacks physical grounding, whereas Visual CoT captures visual details but struggles with abstract planning.Source: §2.1 · pp. 2–3
  • Existing single-modal approaches fail to synergize high-level semantic logic with low-level spatial perception.Source: §2.1 · p. 3

Autoregressive Reasoning in VLA models

  • Autoregressive reasoning generates steps token-by-token, causing severe inference latency and preventing full parallelization.Source: §2.2 · p. 3
  • DualCoT-VLA simultaneously distills 3D spatial priors and internalizes high-level linguistic plans through parallelized CoT query tokens in a single forward pass.The design eliminates token cascading errors and reduces inference latency.Source: §2.2 · p. 4

3. Methodology

DualCoT-VLA combines visual and linguistic implicit CoT streams with a parallel query-token mechanism, then conditions a diffusion action head on their unified reasoning context. Visual geometric distillation captures spatial priors, while linguistic supervision internalizes task plans.

Architecture

  • DualCoT-VLA comprises a VLM backbone, dual-stream implicit CoT mechanism, and downstream Diffusion Transformer action head.The VLM processes multimodal inputs while the two reasoning paths provide visual and linguistic features for action prediction.Source: §3.1 · p. 4, §3 · p. 4

Parallel Implicit CoT

  • Two learnable query-token sets let the VLM produce visual and linguistic CoT hidden states simultaneously in one forward pass.Visual queries contain M = 16 tokens, linguistic queries contain N = 4 tokens, and self-attention integrates both with visual observations and language instructions.Source: §3.2 · p. 4

Visual CoT

  • Visual CoT uses cross-attention to reconstruct frozen Depth Anything 3 dense features, training compact hidden states to encode low-level spatial priors.A geometric teacher supplies dense spatial features, while a mean squared error objective supervises reconstruction from the visual CoT tokens.Source: §3.3 · p. 5

Linguistic CoT

  • Linguistic CoT projects its hidden states into a frozen auxiliary language model, whose task-planning text reconstruction provides step-level supervision.The projected states act as prefix tokens for the decoder, and cross-entropy training encourages logical planning in N = 4 continuous tokens.Source: §3.4 · pp. 5–6

Action Prediction

  • The action head uses a Diffusion Transformer with Flow Matching, conditioning vector-field prediction on noisy actions, time, robot state, and the unified VLM context.The VLM hidden states combine spatial grounding and logical planning before being injected into the action head.Source: §3.5 · p. 6

4. Experiments

DualCoT-VLA is evaluated on simulation benchmarks, real-world manipulation, qualitative reasoning behavior, inference latency, and component ablations. Across these evaluations, it combines complementary visual and linguistic reasoning with parallel latent-space inference.

Experimental Setup

  • Experiments cover LIBERO, RoboCasa GR1, and AgileX Cobot deployments, testing spatial reasoning, long-horizon execution, and high-dimensional action coordination.LIBERO uses a 7-DoF arm, RoboCasa GR1 uses a 29-DoF dexterous hand, and the real-world platform uses onboard RGB cameras.Source: §4.1 · pp. 7–8

Benchmark Results

  • 98.2% Long, 99.4% Spatial, and 99.8% Object success rates show complementary benefits from linguistic planning and visual geometric reasoning.Linguistic CoT supports long-horizon planning, while Visual CoT extracts geometric information for visually intensive suites.Source: §4.2 · p. 9
  • 55.1% average success rate across 24 RoboCasa GR1 tasks includes 80.0% on CuttingboardToPan and 74.0% on PlacematToPlate.The strongest improvements are reported for spatially constrained dexterous-manipulation tasks.Source: §4.2 · p. 9

Reasoning Analysis

  • Qualitative probes recover dense 3D geometry from visual query tokens, while linguistic tokens decode scene state, object locations, and next-step plans.The visual probe maps compressed hidden states into depth maps, and an auxiliary decoder translates linguistic hidden states into structured text.Source: §4.3 · p. 10

Ablation Study

  • Ablations show Visual-only CoT reaches 99.4% on Spatial, Linguistic-only CoT raises Long from 92.0% to 96.0%, and the full model averages 98.8%.These results support complementary roles: linguistic CoT handles long-horizon planning, while visual CoT supplies geometric perception.Source: §4.5 · p. 12

5. Conclusion

DualCoT-VLA provides a dual-stream reasoning paradigm for robotic manipulation, combining low-level 3D spatial perception with high-level logical task planning in latent space. Its parallel implicit CoT mechanism uses learnable query tokens to support single-step reasoning and efficient control.

  • DualCoT-VLA combines low-level 3D spatial perception with high-level logical task planning for robotic manipulation.Source: §5 · p. 12
  • Parallel implicit CoT reasoning with learnable query tokens shifts reasoning from explicit autoregressive inference to single-step latent-space processing.Source: §5 · p. 12
  • The approach addresses single-modal reasoning isolation while bypassing the latency and compounding errors associated with autoregressive inference.Source: §5 · p. 12
  • Extensive experiments across simulation benchmarks and real-world platforms validate the effectiveness of the approach for generalist robotic control.Source: §5 · p. 12

Related papers

Loading 2603.22280v1…