PaperlayerView exact revision on arXiv

Source-linked AI summary

ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning

Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang, Fu-En Yang

arXiv:2507.16815v2cs.CVcs.AIcs.LGcs.RO

TL;DR

VLA systems need better explicit reasoning for long-horizon, adaptive embodied tasks, while supervised reasoning traces can overfit visual scenes and patterns. ThinkAct uses action-aligned reinforcement learning to train an MLLM, compresses its reasoning into a visual plan latent, and conditions a downstream action model. Across embodied reasoning and manipulation benchmarks, it reports few-shot adaptation, long-horizon planning, self-correction, and failure detection, while remaining susceptible to pretrained-MLLM hallucinations.

  • Problem

    Existing VLA approaches lack explicit reasoning for multi-step planning and adaptation, while supervised reasoning traces are costly and can overfit specific scenes or patterns.

  • Method

    ThinkAct trains an MLLM with action-aligned rewards for goal completion and trajectory consistency, then uses a compressed visual plan latent to condition a downstream action model.

  • Results

    ThinkAct demonstrates few-shot adaptation, long-horizon planning, self-correction, and failure detection across embodied reasoning and robot manipulation benchmarks.

  • Takeaways & Limitations

    The framework connects structured visual reasoning with executable actions for more deliberative and adaptable embodied AI systems.

  • Takeaways & Limitations

    Because ThinkAct builds on pretrained multimodal LLMs, visual or spatial hallucinations can produce incorrect plans and affect downstream execution.

Abstract

from arXiv · show

Vision-language-action (VLA) reasoning tasks require agents to interpret multimodal instructions, perform long-horizon planning, and act adaptively in dynamic environments. Existing approaches typically train VLA models in an end-to-end fashion, directly mapping inputs to actions without explicit reasoning, which hinders their ability to plan over multiple steps or adapt to complex task variations. In this paper, we propose ThinkAct, a dual-system framework that bridges high-level reasoning with low-level action execution via reinforced visual latent planning. ThinkAct trains a multimodal LLM to generate embodied reasoning plans guided by reinforcing action-aligned visual rewards based on goal completion and trajectory consistency. These reasoning plans are compressed into a visual plan latent that conditions a downstream action model for robust action execution on target environments. Extensive experiments on embodied reasoning and robot manipulation benchmarks demonstrate that ThinkAct enables few-shot adaptation, long-horizon planning, and self-correction behaviors in complex embodied AI tasks.

1. Introduction

ThinkAct addresses limitations in VLA reasoning by using reinforced visual latent planning to connect embodied reasoning with executable action. The framework targets few-shot adaptation, long-horizon planning, and self-correction in embodied tasks.

  • Existing VLA approaches often map multimodal inputs directly to actions, limiting explicit multi-step planning and adaptation to complex task variations.
  • Supervised CoT approaches train VLAs on costly annotated reasoning traces, making them prone to overfitting specific scenes or reasoning patterns.
  • ThinkAct uses a dual-system framework that connects structured reasoning with executable actions through visual latent planning.
  • Action-aligned rewards combine visual goal completion and trajectory alignment to ground long-horizon reasoning in embodied scenes.
  • ThinkAct demonstrates few-shot adaptation, long-horizon planning, and self-correction across diverse embodied manipulation tasks.

2. Related Works

Related work adapts vision-language models to action-centric tasks and adds explicit reasoning, but these approaches commonly rely on direct action prediction or supervised intermediate traces.

  • Action-centric VLMs incorporate points, visual trajectories, or curated instruction-following data to improve spatial-action understanding and failure detection.
  • Large-scale robot demonstrations support direct action execution in generalist VLA models such as OpenVLA, while visual traces enhance spatial-action awareness.
  • CoT-based VLA methods synthesize intermediate subgoals and use supervised fine-tuning to teach models to reason before acting.

3. Method

ThinkAct separates high-level visual planning from low-level control: an MLLM generates a visual plan latent, while an action model converts it into environment-specific actions. Reinforced visual feedback shapes planning toward task goals and plausible trajectories, and asynchronous execution supports slow reasoning with fast control.

  • 3.1. Problem Formulation: ThinkAct maps an observation and instruction to a visual plan latent that guides an action model in predicting executable actions for the target environment.
  • 3.2. Reinforced Visual Latent Planning for Embodied Reasoning: The framework uses action-aligned visual feedback because task-success rewards alone are restricted to specific simulators without visual-scene guidance.
  • 3.2. Reinforced Visual Latent Planning for Embodied Reasoning: The MLLM generates reasoning embeddings and a visual plan latent that is decoded into a 2D gripper trajectory with start and end positions.
  • 3.2. Reinforced Visual Latent Planning for Embodied Reasoning: The trajectory reward regularizes predicted trajectories toward demonstrated trajectory distributions, using distance measured by dynamic time warping.
  • 3.2. Reinforced Visual Latent Planning for Embodied Reasoning: The overall reward combines action-aligned visual feedback with format correctness, weighting goal and trajectory rewards equally.
  • 3.2. Reinforced Visual Latent Planning for Embodied Reasoning: GRPO samples response groups, evaluates their rewards, and regularizes updates against the original MLLM using KL divergence.
  • 3.3. Reasoning-Enhanced Action Adaptation: A latent projector injects the visual plan into the action model, whose state encoder, projector, and action model are trained by imitation learning on annotated demonstrations.
  • 3.3. Reasoning-Enhanced Action Adaptation: Reasoning and action execution can run asynchronously, allowing the MLLM to perform slow thinking while the action model performs fast control.

4. Experiment

ThinkAct is evaluated across robot manipulation, embodied reasoning, qualitative reasoning, reward ablations, and few-shot adaptation. Results show improved benchmark performance, long-horizon task decomposition, RL-enhanced reasoning, and adaptation from ten demonstrations per task.

  • Evaluation Setup: ThinkAct is evaluated on SimplerEnv and LIBERO for manipulation, and EgoPlan-Bench2, RoboVQA, and OpenEQA for embodied reasoning.Manipulation uses task success rate; reasoning uses accuracy, BLEU, and LLM-based scoring under the benchmarks’ original protocols.
  • Quantitative Evaluation: 84.4% is ThinkAct’s best overall LIBERO success rate, outperforming DiT-Policy and CoT-VLA across spatial, object, goal, and long-horizon settings.These subtasks test generalization across spatial layouts, object variations, goal diversity, and long-horizon planning.
  • Quantitative Evaluation: ThinkAct outperforms the second-best method by 2.5% on EgoPlan-Bench2 and 4.1 BLEU on RoboVQA.EgoPlan-Bench2 measures multi-step planning, while RoboVQA focuses on long-horizon robotic reasoning.
  • Qualitative Results: In LIBERO-Long, ThinkAct decomposes placing a book into picking it up, moving left to right, and placing it in the compartment, while its gripper follows the planned trajectory.The qualitative examples cover Simpler-Bridge and LIBERO-Long manipulation tasks.
  • Qualitative Results: RL improves embodied reasoning: the RL-tuned model answers correctly where the cold-start model misinterprets the question or focuses only on the current state.Figure 4 compares ThinkAct before and after RL on RoboVQA and OpenEQA examples, with incorrect and correct reasoning marked red and green.
  • Ablation Study: Removing either trajectory or goal reward reduces performance, while removing both leaves only marginal gains over SFT, showing both rewards support structured multi-step planning.The full ThinkAct version performs best across the ablation benchmarks, and the SFT cold-start model without RL performs worst.
  • Few-Shot Adaptation: With 10 demonstrations per task, ThinkAct achieves the highest LIBERO few-shot adaptation success rates and surpasses Magma by 7.3% on LIBERO-Goal and 9.5% on LIBERO-Spatial.The experiment evaluates unseen environments and new skills over 100 trials.

5. Conclusion

ThinkAct combines action-aligned reinforcement learning with reasoning-enhanced action adaptation to support long-horizon planning, few-shot adaptation, and self-correction. Its reliance on pretrained multimodal LLMs leaves it vulnerable to visual and spatial hallucinations that can affect execution.

  • ThinkAct reinforces visual latent planning for vision-language-action reasoning and combines it with reasoning-enhanced action adaptation.The framework is designed to connect deliberative reasoning with robust action execution in dynamic environments.
  • Extensive experiments across embodied reasoning and robot manipulation benchmarks demonstrate long-horizon planning, few-shot adaptation, failure detection, and self-correction.
  • Pretrained multimodal LLMs can hallucinate visual or spatial information, causing plans to reference incorrect object attributes or spatial relationships and affecting downstream execution.Latent planning and action grounding mitigate this limitation to some extent.
  • Greater autonomy in embodied systems may create risks from ambiguous commands, hallucinated visual reasoning, or overconfident chain-of-thought outputs, especially in safety-critical settings.The paper identifies safeguards and alignment with human intent as areas for future work.

A.1. Implementation Details

ThinkAct is trained in staged supervised and reinforced fine-tuning pipelines using visual trajectories, robotic and egocentric QA, failure cases, and video reasoning data. Cached visual plan latents then support efficient downstream action-model training.

  • Action Model Training: The MLLM generates visual plan latents offline for each observation-instruction pair, after which a diffusion action model is trained by imitation learning with the VLM frozen.
  • 2D Trajectory of Manipulation: Visual trajectories come from Open X-Embodiment and Something-Something V2, with gripper positions extracted from frames and camera motion reduced through stabilization.
  • Datasets: RoboVQA supplies approximately 5K long-horizon and 92K medium-horizon videos, while RoboFail contains 130 simulated and real-world robot manipulation failure cases.
  • Supervised Fine-Tuning for Cold Start: The supervised fine-tuning cold-start dataset contains 30K 2D visual trajectories, 50K RoboVQA samples, 50K EgoPlan-IT samples, and 165K Video-R1-CoT samples.
  • Reinforced Fine-Tuning for Eliciting Visual Latent Planning: The reinforced fine-tuning mixture contains 12.5K visual trajectories, 10K RoboVQA samples, 10K EgoPlan-IT/Val samples, 0.5K RoboFail samples, and 10K LLaVA-Video-178K samples.
  • Reinforced Fine-Tuning for Eliciting Visual Latent Planning: The reinforced mixture combines action-grounded and reasoning-intensive data to support physically executable and semantically coherent planning across diverse real-world tasks.

A.3. Evaluation Benchmarks

The evaluation covers simulated manipulation generalization, egocentric planning, robotic visual question answering, embodied question answering, and qualitative execution comparisons. These benchmarks vary visual conditions, task structure, embodiment, and reasoning demands.

  • Simulation Benchmarks: SimplerEnv evaluates VLA robustness under visual matching and variant aggregation settings spanning lighting, textures, backgrounds, distractors, and camera poses.
  • Simulation Benchmarks: LIBERO evaluates robotic manipulation generalization through spatial, object, goal, and long-horizon task suites, each tested over 500 trials.
  • Embodied Reasoning Benchmarks: EgoPlan-Bench2 measures egocentric long-horizon planning across 24 daily-life scenarios and includes 1,321 human-verified multiple-choice question–answer pairs.
  • Embodied Reasoning Benchmarks: RoboVQA evaluates long-horizon reasoning, contextual understanding, affordance-based decisions, and outcome classification using 1,893 validation video–text pairs scored by BLEU.
  • Embodied Reasoning Benchmarks: OpenEQA tests spatial, functional, and commonsense reasoning across more than 1,600 human-authored questions from over 180 real-world environments.
  • Qualitative Evaluation: Qualitative robot-execution comparisons include DiT-Policy, OpenVLA, and ThinkAct, while Figure A7 contrasts ThinkAct with and without reinforcement learning using red and green correctness markers.

B.2. Additional Qualitative Results

Additional qualitative examples show that reinforcement learning improves embodied reasoning and that ThinkAct can detect execution failures, replan, and recover successfully.

  • Reinforcement learning improves embodied reasoning in an EgoPlan-Bench2 example, enabling ThinkAct to predict the correct next action.
  • When a mug grasp fails, the reasoning MLLM identifies gripper difficulty and suggests repositioning before reattempting the grasp.
  • When an object is not picked up before basket placement, the MLLM detects the failure, replans pickup, and achieves successful completion.

B.4. Results of Smaller Model Size

ThinkAct remains effective with a smaller Qwen2.5-VL-3B backbone, outperforming similarly sized models across three embodied reasoning benchmarks. Additional results also show consistent gains in 5-shot LIBERO adaptation and demonstrate self-reflection and correction behaviors.

  • Smaller Model Generalization: ThinkAct consistently outperforms similarly sized models on EgoPlan-Bench2, RoboVQA, and OpenEQA.The comparison uses the Qwen2.5-VL-3B backbone.
  • Self-Reflection and Correction: Additional demonstrations illustrate ThinkAct’s self-reflection and correction capability.
  • 5-Shot Adaptation: With only 5 demonstrations per LIBERO task, ThinkAct outperforms comparative methods across all three evaluated tasks.Performance is measured over 100 trials per task.

B.6. Ablation Study

Ablations show that action-aligned visual rewards are important for planning, while reasoning-update frequency presents a performance–inference-cost trade-off. ThinkAct uses 75 actions per reasoning step, and its reasoning process incurs a modest speed penalty relative to OpenVLA.

  • Reward Ablation: Removing either r_goal or r_traj reduces LIBERO and OpenEQA performance, while removing both approaches the SFT baseline.The ablation supports the importance of combining goal-completion and trajectory-alignment rewards.
  • Reasoning Frequency: Average LIBERO success rates are 84.0%, 84.6%, 84.4%, and 83.7% for N=25, 50, 75, and 100, respectively.The results indicate relatively similar performance across update frequencies, with the lowest average at N=100.
  • Reasoning Frequency: N=100 may delay failure detection and self-correction, whereas N=25 adds inference cost without substantial performance gains.These considerations motivate selecting an intermediate reasoning-update frequency.
  • Inference Efficiency: ThinkAct takes 17% longer on average than OpenVLA because of its autoregressive reasoning process.The comparison is conducted on LIBERO tasks using an A100 GPU.
Loading 2507.16815v2…