PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning

Chi-Pin Huang, Yunze Man, Zhiding Yu, Min-Hung Chen, Jan Kautz, Yu-Chiang Frank Wang, Fu-En Yang

arXiv:2601.09708v2cs.CVcs.AIcs.LGcs.RO

TL;DR

VLA systems need strong reasoning for long-horizon, adaptive manipulation, but explicit chain-of-thought creates excessive inference latency. Fast-ThinkAct distills linguistic and visual planning into verbalizable latent representations and aligns them with action execution, achieving strong benchmark performance with up to 89.3% lower latency.

  • Problem

    VLA models struggle with long-horizon planning, failure correction, and novel scenarios, while lengthy explicit reasoning traces create an inference-latency bottleneck.

  • Method

    Fast-ThinkAct uses preference-guided teacher distillation and manipulation-trajectory alignment to learn compact verbalizable linguistic and visual planning latents for reasoning-enhanced policy learning.

  • Results

    Fast-ThinkAct outperforms comparison methods across diverse embodied manipulation and reasoning benchmarks while reducing inference latency by up to 89.3%.

  • Takeaways & Limitations

    Compact latent reasoning supports effective long-horizon planning, few-shot adaptation, and failure recovery while connecting multimodal planning to embodied action execution.

  • Takeaways & Limitations

    The pretrained-LLM verbalizer can hallucinate plausible but inaccurate descriptions, although grounded latent representations are used for action prediction.

Abstract

from arXiv · show

Vision-Language-Action (VLA) tasks require reasoning over complex visual scenes and executing adaptive actions in dynamic environments. While recent studies on reasoning VLAs show that explicit chain-of-thought (CoT) can improve generalization, they suffer from high inference latency due to lengthy reasoning traces. We propose Fast-ThinkAct, an efficient reasoning framework that achieves compact yet performant planning through verbalizable latent reasoning. Fast-ThinkAct learns to reason efficiently with latent CoTs by distilling from a teacher, driven by a preference-guided objective to align manipulation trajectories that transfers both linguistic and visual planning capabilities for embodied control. This enables reasoning-enhanced policy learning that effectively connects compact reasoning to action execution. Extensive experiments across diverse embodied manipulation and reasoning benchmarks demonstrate that Fast-ThinkAct achieves strong performance with up to 89.3% reduced inference latency over state-of-the-art reasoning VLAs, while maintaining effective long-horizon planning, few-shot adaptation, and failure recovery.

1. Introduction

Fast-ThinkAct addresses the difficulty of preserving reasoning and adaptive generalization in VLA tasks without incurring the latency of lengthy explicit reasoning traces. It compresses linguistic and visual planning into compact latent representations and connects them to action execution.

  • VLA agents must reason over complex visual and temporal contexts while adapting actions in dynamic environments.
  • Supervised action-data training supports basic skills but generalizes poorly to long-horizon planning, failure correction, and novel scenarios.Exhaustive robot demonstrations are impractical to collect.
  • Explicit reasoning improves VLA generalization but lengthy traces can take seconds per decision, limiting real-time performance and creating safety risks.Embodied agents may need decisions at 1-15 Hz, whereas lengthy reasoning can operate near 0.1 Hz.
  • Fast-ThinkAct compresses linguistic and visual planning into compact continuous latents through preference-guided distillation and visual trajectory alignment.A verbalizer decodes the latent reasoning, while trajectory alignment transfers visual planning capabilities from teacher to student.
  • The framework uses reasoning-enhanced policy learning to bridge high-level visual planning with low-level action execution.
  • 89.3% inference latency reduction is achieved over state-of-the-art reasoning VLAs while maintaining strong performance across embodied benchmarks.

2. Related Works

Related work extends foundation VLAs with explicit textual or visual reasoning and explores efficiency techniques for shortening or removing reasoning traces. Fast-ThinkAct instead targets compact latent representations that encode multimodal information for embodied control.

  • Foundation VLAs: Foundation VLAs train vision-language backbones on large-scale robot demonstrations to support embodied AI.OpenVLA and π0 are cited as examples.
  • Reasoning VLAs: Reasoning VLAs add intermediate reasoning through chain-of-thought annotations, visual goals, spatial representations, or interleaved vision-language-action pre-training.
  • Reasoning Efficiency: Length-penalty methods encourage shorter reasoning chains but can suffer from training instability, while latent-reasoning methods operate in continuous spaces.
  • Reasoning Efficiency: Existing LLM latent-reasoning techniques cannot directly transfer to VLA tasks because embodied control requires spatial-temporal understanding and semantic-action bridging.
  • Reasoning Efficiency: Reasoning dropout accelerates embodied reasoning by skipping test-time traces but can produce inconsistent planning because it relies on supervised embodied chain-of-thought.
  • Fast-ThinkAct: Fast-ThinkAct distills reasoning into compact latent representations that encode multimodal information for embodied reasoning.

3. Method

Fast-ThinkAct compresses textual and visual reasoning into verbalizable latent plans, then connects those plans to executable robot actions. Its teacher-student distillation and spatial-token design target efficient, grounded long-horizon control.

  • Framework overview: Fast-ThinkAct processes observation-instruction pairs with latent chain-of-thought reasoning to produce a compact visual plan that guides executable action prediction.The VLM produces visual plan latent c_t, which conditions the action model π_φ.
  • Verbalizable latent reasoning: The framework uses a textual teacher and latent student, with preference-guided learning transferring high-quality reasoning patterns into continuous latent representations.A verbalizer decodes student latents into text, while teacher-derived reward preferences distinguish higher- and lower-quality reasoning traces.
  • Visual plan distillation: Action-aligned visual plan distillation transfers trajectory-level visual planning by minimizing the L2 distance between teacher and student hidden states.The aligned hidden states correspond to the teacher’s preferred reasoning trace and the student representation encoding the visual plan.
  • Visual plan distillation: The student predicts K waypoints in parallel through learnable spatial tokens, replacing the teacher’s verbose autoregressive waypoint sequence.Each spatial-token hidden state is projected to a waypoint, supporting compact visual trajectory prediction.
  • Reasoning-enhanced policy learning: Reasoning-enhanced policy learning connects visual latent planning to low-level action generation by conditioning a diffusion Transformer action model on spatial-token representations.The action model’s cross-attention uses visual planning context together with state observations, and post-training updates the action components with imitation learning.
  • Inference: At inference, only the latent student VLM and action model are required; the verbalizer is used during training and optionally for interpretability.The student extracts visual latent planning from spatial-token KV caches to condition action prediction.

4. Experiment

Experiments evaluate Fast-ThinkAct across embodied reasoning and manipulation benchmarks, including long-horizon planning, few-shot adaptation, failure recovery, and inference efficiency. The method consistently achieves strong task performance while reducing reasoning latency.

  • Experimental Setup: Fast-ThinkAct uses Qwen2.5-VL 3B, teacher-student training with GRPO and visual rewards, and action models initialized from DiT-Policy or RDT.Training combines visual trajectories, QA data, and robotic action datasets.
  • Robot Manipulation: Fast-ThinkAct achieves the highest success rates across all evaluated LIBERO subtasks and SimplerEnv-Google.It outperforms OpenVLA, CoT-VLA, ThinkAct, and MolmoAct.
  • Robot Manipulation: 89.3% and 88.0% latency reductions versus ThinkAct-7B and MolmoAct-7B, respectively, accompany the manipulation gains.Inference is also 7× faster than ThinkAct-3B.
  • Bimanual Manipulation: Fast-ThinkAct improves RoboTwin2.0 success rates over RDT by 9.3% on easy tasks and 3.6% on hard tasks.Against ThinkAct, improvements are 3.3% and 1.7%, respectively.
  • Embodied Reasoning: Fast-ThinkAct exceeds the runner-up by 2.4% on EgoPlan-Bench2, 5.5 BLEU on RoboVQA, and 1.1 points on OpenEQA.The benchmarks cover multi-step planning, robotic manipulation reasoning, and zero-shot embodied-scene understanding.
  • Long-Horizon Planning: On long-horizon RoboTwin2.0 tasks, Fast-ThinkAct scores 48.8/16.8 on easy/hard settings, surpassing RDT and ThinkAct.The predicted visual traces guide feasible execution paths in complex manipulation scenarios.
  • Few-Shot Adaptation: Using 10 demonstrations per task, Fast-ThinkAct improves RDT and outperforms π0 and ThinkAct on medium- and long-horizon RoboTwin2.0 tasks.The gains occur with lower reasoning latency than ThinkAct.
  • Ablation Study: Removing verbalization or distillation losses reduces performance, showing the importance of preference guidance and trajectory-level alignment.The ablation attributes these components to high-quality reasoning and visual-planning transfer.

5. Conclusion

Fast-ThinkAct presents compact verbalizable latent reasoning for efficient vision-language-action planning. Experiments support strong performance with reduced inference latency, long-horizon planning, few-shot adaptation, and failure recovery, while verbalizer faithfulness remains limited by language-model hallucinations.

  • Conclusion: Fast-ThinkAct distills lengthy textual reasoning into compact latent representations using preference-guided distillation and visual trajectory alignment.Reasoning-enhanced policy learning connects embodied reasoning with executable actions.
  • Conclusion: Experiments demonstrate strong performance with significantly reduced inference latency across robotic manipulation and embodied reasoning benchmarks.The reported capabilities include long-horizon planning, few-shot adaptation, and failure recovery.
  • Limitations and Future Works: The verbalizer can hallucinate plausible but inaccurate descriptions because it is built on a pre-trained language model.The authors state that action execution uses grounded latent representations rather than verbalizer outputs.

A. Additional Experimental Setup

Algorithm 1 jointly optimizes a teacher and student to distill reasoning into compact latent representations. The student combines verbalization, trajectory distillation, and answer losses before returning the trained model.

  • Initialization: Algorithm 1 initializes teacher, student, and verbalizer models from a CoT-SFT checkpoint and runs iterative training over demonstrations.The procedure also specifies rollout size, latent reasoning steps, waypoints, and total iterations.
  • Teacher Optimization: The teacher is optimized with GRPO, and the highest- and lowest-advantage trajectories are selected for preference-guided student training.These trajectories are denoted τ+ and τ− in the procedure.
  • Student Distillation: The student performs autoregressive latent reasoning, verbalizes the latent state, and predicts spatial tokens for trajectory distillation.The procedure computes verbalization and distillation losses from these representations.
  • Optimization Objective: The student update combines verbalization, distillation, and answer losses before incrementing the training iteration.Algorithm 1 returns the updated student model after the loop terminates.
  • Training Procedure: Algorithm 1 jointly optimizes the teacher with GRPO and distills its reasoning into the student’s compact latent representations.This summarizes the complete training procedure described for Section 3.2.

A.2. Implementation Details

Implementation details cover verbalizer initialization, policy-model choices, and action-model conditioning. Fast-ThinkAct improves manipulation performance across different action models, supporting agnosticism to the underlying action architecture.

  • Verbalizer: The verbalizer is initialized from Qwen3-0.6B with cross-attention layers that condition on latent CoTs.A warm-up language-modeling stage aligns verbalizer outputs with latent representations before verbalization training.
  • Action Models: DiT-Policy is used for SimplerEnv, while RDT is used for LIBERO and RoboTwin2.0 with OXE and static ALOHA action data.These choices follow the comparison setups for the respective benchmarks.
  • Action Models: Fast-ThinkAct further enhances RDT’s manipulation capabilities on LIBERO and RoboTwin2.0.The policy-learning setup uses OXE and static ALOHA datasets for these evaluations.
  • Action Models: The approach is agnostic to the underlying action model choice.The implementation uses different action models across benchmark evaluations while retaining the reasoning-enhanced policy framework.

A.3.1. Dataset Sources

The paper draws on visual trajectories, robot-video QA, egocentric planning, and general vision-language data to train embodied reasoning. Its training pipeline combines supervised fine-tuning, chain-of-thought augmentation, and teacher-student latent distillation.

  • Dataset Sources: Single-arm training uses approximately 1.3M MolmoAct trajectories, while bimanual training adds approximately 92K dual-arm AIST samples.The trajectories are extracted and tracked to represent manipulation behavior visually.
  • Dataset Sources: The reasoning corpus includes RoboFAC, RoboVQA, ShareRobot, EgoPlan-Bench, Video-R1-CoT, and PixMo, covering failure analysis, manipulation QA, planning, and general visual understanding.RoboFAC contributes 64K failure-analysis QA pairs, RoboVQA contributes 798K QA pairs, and ShareRobot contributes over 1M QA pairs.
  • Training Pipeline: Supervised fine-tuning combines approximately 4M trajectory and QA samples to acquire visual understanding, task comprehension, and manipulation knowledge.The data spans multiple embodiments and scenarios.
  • Training Pipeline: Chain-of-thought fine-tuning samples approximately 200K embodied examples and adds 165K Video-R1-CoT samples to teach structured reasoning and transfer it to embodied tasks.Prompts distinguish reasoning-bearing samples from direct-answer samples using <think> and <answer> tags.
  • Training Pipeline: Teacher-student training uses nearly 50K balanced instances to transfer high-quality reasoning patterns through teacher GRPO and student latent distillation.The training set samples approximately 5,000 instances from each dataset and data type.

A.4.2. Robotic Manipulation Benchmarks

Robotic manipulation evaluation spans simulation environments with visual, task, embodiment, and horizon variation. The benchmarks test both single-arm and challenging bimanual generalization.

  • Simulation Benchmarks: SimplerEnv evaluates manipulation under variations in lighting, textures, backgrounds, and camera poses, with Google Robot tasks trained on OXE and evaluated in simulation.
  • Simulation Benchmarks: LIBERO tests spatial-layout, object, goal, and long-horizon generalization across four task suites, using 500 trials and three random seeds per suite.
  • Simulation Benchmarks: RoboTwin2.0 evaluates 10 bimanual tasks under easy and hard settings, with hard conditions adding clutter, lighting, texture, and height variation.Evaluation uses 50 clean expert demonstrations per task and 100 rollouts under each setting.
  • Failure Analysis: Figure 9 examines failure identification and analysis across execution stages and failure types in robotic manipulation.

B.1. Additional Quantitative Results

Additional quantitative results show that Fast-ThinkAct scales to larger backbones and improves the efficiency-performance trade-off over ThinkAct-3B and concise textual-reasoning baselines.

  • Larger Model Size: Fast-ThinkAct consistently achieves strong performance across EgoPlan-Bench2, RoboVQA, and OpenEQA with a 7B or 8B backbone.This result is presented as evidence that latent reasoning distillation scales to larger model backbones.
  • ThinkAct-3B Comparison: 89.7 vs. 83.1 on LIBERO and 68.7 vs. 64.7 on SimplerEnv-Google, with 7× faster inference at 805ms vs. 5674ms than ThinkAct-3B.The comparison uses models at the same 3B size.
  • Efficient Reasoning Baselines: Fast-ThinkAct achieves 53.3 with 6 latent tokens, exceeding the teacher’s 49.8 and concise textual baselines ranging from 46.3 to 47.8.The baselines use zero tokens, 6 textual tokens, or approximately 50 tokens with length-penalized RL.
  • Reasoning Trace Comparison: The OpenEQA example pairs a lengthy teacher reasoning trace with a student verbalized latent trace, illustrating the comparison used for reasoning quality.

B.2. Additional Qualitative Results

Qualitative results illustrate successful execution, bimanual coordination, failure analysis, and compact verbalized reasoning. The examples connect latent planning with manipulation robustness and interpretability.

  • Qualitative Robot Execution: Fast-ThinkAct improves challenging robotic execution over the base RDT action model through better spatial understanding and coordination.
  • Bimanual Manipulation Results: On RoboTwin2.0, predicted waypoints guide both grippers through bimanual tasks under easy and hard visual conditions.Hard conditions include different backgrounds and distractor objects.
  • Failure Identification and Recovery: Fast-ThinkAct identifies failure types and execution stages, then analyzes root causes such as an improper initial grasp.
  • Verbalized Latent Reasoning: The student produces compact, correct verbalized reasoning, whereas the teacher’s lengthy output sometimes contains erroneous steps.

B.3. Additional Ablation Study and Analysis

Ablations show that Fast-ThinkAct’s performance depends on compact latent reasoning design choices, including distillation components, early-layer conditioning, and the number of latent reasoning steps.

  • Additional Ablation Results on Manipulation Benchmarks: Removing ℒverb or ℒdistill progressively degrades performance across LIBERO, SimplerEnv-Google, and RoboTwin2.0.The full model outperforms the textual teacher and models without teacher-student training.
  • Ablation Study on Action Model Conditioning: 89.7 on LIBERO exceeds late-layer KV conditioning at 88.3 and output hidden states at 87.1.Early-layer representations better capture visual planning information for action-model conditioning.
  • Ablation Study on Latent Reasoning Steps: M = 6 achieves optimal performance, whereas M = 1 limits reasoning capacity and M = 30 or 100 may add redundant or noisy information.The default balances insufficient reasoning with excessive latent reasoning steps.
Loading 2601.09708v2…