PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving

Yuechen Luo, Fang Li, Shaoqing Xu, Zhiyi Lai, Lei Yang, Qimao Chen, Ziang Luo, Zixun Xie, Shengyin Jiang, Jiaxin Liu, Long Chen, Bing Wang, Zhi-xin Yang

arXiv:2509.13769v1cs.CV

TL;DR

CoT can improve driving decisions in complex scenes, yet fixed reasoning may be unnecessary in simple ones. AdaThinkDrive learns to choose between direct prediction and reasoning, reaching 90.3 PDMS on Navsim while reducing inference time against always-Think.

  • Problem

    In simple scenarios, CoT can add computational overhead without improving decision quality, while its benefits appear in more complex scenes.

    Source: §I · p. 1

  • Method

    AdaThinkDrive combines driving-data pretraining, paired Think and Non-Think SFT outputs, and GRPO with Adaptive Think Reward to select reasoning modes by scene complexity.

    Source: §I · p. 2, B. Data Preparation · p. 3

  • Results

    90.3 PDMS on Navsim, 1.7 points above the leading vision-only baseline; inference time is 14% lower than the always-Think baseline.

    Source: §I · p. 2

  • Takeaways & Limitations

    The results support adaptive thinking as a way to balance accurate and efficient decision-making in autonomous systems.

    Source: §V · p. 8

Abstract

from arXiv · show

While reasoning technology like Chain of Thought (CoT) has been widely adopted in Vision Language Action (VLA) models, it demonstrates promising capabilities in end to end autonomous driving. However, recent efforts to integrate CoT reasoning often fall short in simple scenarios, introducing unnecessary computational overhead without improving decision quality. To address this, we propose AdaThinkDrive, a novel VLA framework with a dual mode reasoning mechanism inspired by fast and slow thinking. First, our framework is pretrained on large scale autonomous driving (AD) scenarios using both question answering (QA) and trajectory datasets to acquire world knowledge and driving commonsense. During supervised fine tuning (SFT), we introduce a two mode dataset, fast answering (w/o CoT) and slow thinking (with CoT), enabling the model to distinguish between scenarios that require reasoning. Furthermore, an Adaptive Think Reward strategy is proposed in conjunction with the Group Relative Policy Optimization (GRPO), which rewards the model for selectively applying CoT by comparing trajectory quality across different reasoning modes. Extensive experiments on the Navsim benchmark show that AdaThinkDrive achieves a PDMS of 90.3, surpassing the best vision only baseline by 1.7 points. Moreover, ablations show that AdaThinkDrive surpasses both the never Think and always Think baselines, improving PDMS by 2.0 and 1.4, respectively. It also reduces inference time by 14% compared to the always Think baseline, demonstrating its ability to balance accuracy and efficiency through adaptive reasoning.

I. INTRODUCTION

AdaThinkDrive targets the mismatch between fixed reasoning and scene-dependent driving needs by selecting between direct prediction and explicit reasoning. Its reported Navsim results pair strong trajectory performance with lower inference time than always-Think.

I. INTRODUCTION

  • In both InternVL3 8B and 2B, Non-Think performs better at Level 1, while Think outperforms it at Levels 2 and 3.This comparison motivates reasoning that adapts to scene complexity rather than applying CoT universally.Source: §I · p. 1
  • AdaThinkDrive uses a fast-answering/slow-thinking mechanism to select between direct prediction and explicit reasoning based on scene complexity.Its training combines large-scale driving-data pretraining, dual-mode Navsim SFT, and GRPO rewards for trajectory accuracy, action rationality, and reasoning simplicity.Source: §I · p. 2
  • The authors’ comparative NAVSIM study identifies over-reasoning in simple scenarios as a limitation of existing CoT approaches and motivates adaptive reasoning.The introduction frames selective reasoning as a way to improve decision accuracy and inference efficiency.Source: §I · p. 2
  • 90.3 PDMS on Navsim, 1.7 points above the leading vision-only baseline; inference time is 14% lower than the always-Think baseline.The model uses CoT in 96% of challenging scenarios and direct trajectory prediction in 84% of simple scenarios.Source: §I · p. 2

II. RELATED WORK

The paper situates adaptive reasoning among VLA approaches that predict trajectories and among methods for triggering CoT efficiently. It formulates mode selection as choosing the reasoning mode that maximizes expected task utility.

A. VLA for Autonomous Driving

  • Trajectory-prediction VLA methods use intermediate CoT for interpretability and accuracy, but the paper reports that CoT benefits mainly complex scenarios and can have minimal or negative effects in simpler ones.The related-work discussion contrasts trajectory-prediction approaches with scene-understanding methods that produce high-level guidance.Source: A. VLA for Autonomous Driving · p. 2

B. Efficient Reasoning Models

  • Efficient-reasoning approaches include concise reasoning, dynamic early stopping, and on-demand reasoning, with the latter selecting whether to reason based on task complexity.The paper frames driving as requiring quick responses in simple scenes and careful analysis in challenging ones.Source: B. Efficient Reasoning Models · p. 2

III. METHODS

  • AdaThinkDrive’s method comprises data preparation, two-stage supervised fine-tuning, and reinforcement learning for complexity-aware thinking.Its framework figure describes vision and text inputs, mode selection, and reinforcement rewards including PDMS, format, endpoint, and Adaptive Think Reward.Source: §III · p. 2, Figure 2 · p. 3

A. Problem Formulation

  • For each query, the model jointly selects Thinking or Non-Thinking and generates an answer; the selected mode maximizes expected task-specific utility.The policy learns a mode for each query to maximize expected utility across query distributions.Source: A. Problem Formulation · p. 3

B. Data Preparation

Data preparation combines driving-domain knowledge sources with paired reasoning and direct-answer supervision. NAVSIM scenes are also labeled by complexity using road-boundary proximity and critical-object presence.

B. Data Preparation

  • Pretraining data combines open-source driving QA datasets with a NAVSIM multi-turn reasoning dataset covering road boundaries, critical objects, ego actions, and scene understanding.The stated aim is to build foundational driving knowledge and understanding of when CoT may help.Source: B. Data Preparation · p. 3
  • Hybrid SFT data includes reasoning-intensive and direct-answer examples, with generated reasoning traces, fine-grained scene annotations, and agent-interaction categories.CIPO-1 denotes ego-lane agents, CIPO-2 likely merging agents, and Motion Interaction agents whose predicted paths intersect the ego trajectory.Source: B. Data Preparation · p. 3, Figure 3 · p. 4
  • Each query receives both a Think-style response retaining its reasoning process and a structurally consistent Non-Think response that omits explicit reasoning.The paired dataset is used as a warm-up for distinguishing the two response styles.Source: B. Data Preparation · p. 3
  • NAVSIM scenes are divided into three complexity levels by road-boundary proximity and critical-object presence: neither condition defines Level 1, one defines Level 2, and both define Level 3.Levels 2–3 form D+ and Level 1 forms D−; these labels provide auxiliary initialization for reinforcement learning.Source: B. Data Preparation · pp. 3–4

C. Two-Stage Supervised Fine-tuning Processing

The two-stage supervised fine-tuning process first builds driving-domain understanding, then teaches trajectory prediction in both reasoning modes. This provides a unified starting point for adaptive reasoning during GRPO.

C. Two-Stage Supervised Fine-tuning Processing

  • The first fine-tuning stage injects general driving knowledge through a large corpus of driving-related QA pairs, including drivable-area, object-localization, and traffic-semantics tasks.This stage addresses driving-domain cognition before trajectory prediction is introduced.Source: C. Two-Stage Supervised Fine-tuning… · p. 4
  • The second stage introduces trajectory prediction and generates reasoning-inclusive Thinking outputs alongside Non-Thinking outputs containing only the final trajectory.Training supervises both outputs by maximizing their conditional likelihood.Source: C. Two-Stage Supervised Fine-tuning… · p. 4
  • Joint supervision teaches both response modes through a unified interface without bias toward either style, enabling either output for a query in the GRPO phase.This warm-start supports later adaptive reasoning.Source: C. Two-Stage Supervised Fine-tuning… · p. 4

D. Adaptive Thinking via Reinforcement Learning

AdaThinkDrive adds reinforcement learning to teach the policy when to use either reasoning mode while improving planning. Four complementary rewards assess trajectory quality, output format, endpoint accuracy, and reasoning-mode selection.

D. Adaptive Thinking via Reinforcement Learning

  • Reinforcement learning teaches the policy to select the more appropriate reasoning mode for each query while improving planning ability.After supervised fine-tuning, the policy already supports both modes without collapsing, but does not yet adaptively select between them.Source: D. Adaptive Thinking via Reinforcement… · p. 4
  • Four complementary rewards assess predicted trajectories, output format, endpoint accuracy, and adaptive reasoning.The format reward checks both the required think/answer tags and standardized trajectory representation, while the trajectory reward uses PDMS on a discrete 0–1 scale.Source: D. Adaptive Thinking via Reinforcement… · p. 4, 2) Format Reward · p. 4

2) Format Reward:

The reinforcement-learning stage combines multiple reward components and uses rollout comparisons to adjust whether the policy thinks or predicts directly. Its adaptive reward can revise scene-complexity labels when the alternative reasoning mode performs better.

3) Endpoint Reward:

  • The endpoint reward assigns 1.0 below 2 m deviation, then steps down through 0.8 below 4 m, 0.6 below 6 m, 0.4 below 10 m, and 0.2 below 15 m.It uses L1 distance from the ground-truth final trajectory point and assigns 0.0 otherwise.Source: 3) Endpoint Reward · p. 4

4) Adaptive Think Reward:

  • The reward corrects static scene tags when actual reasoning needs differ, helping avoid collapse into always Thinking or always Non-thinking responses.Dynamic correction is intended to improve decision-making efficiency and prediction accuracy.Source: 4) Adaptive Think Reward · p. 5
  • The adaptive reward uses multiple rollouts to dynamically adjust reasoning behavior, with Figure 4 describing rewards for Thinking or Non-thinking responses when scene-specific conditions are met.The method can correct a scene label when the other reasoning mode meets its performance and rollout-count conditions.Source: Figure 4 · p. 4, 4) Adaptive Think Reward · p. 5
  • For simple scenes, the policy rewards Thinking only when its average PDMS exceeds both Non-thinking PDMS and threshold T, and Thinking rollouts are more numerous.For challenging scenes, the rule reverses: Non-thinking must exceed Thinking PDMS and T, and Non-thinking rollouts must be more numerous.Source: 4) Adaptive Think Reward · p. 5
  • The overall reinforcement-learning reward integrates four specifically designed reward components.The passage introduces the components without enumerating them there.Source: 4) Adaptive Think Reward · p. 5
  • GRPO samples candidate outputs per query and optimizes the current policy using reward signals, with truncated importance weights and KL regularization to constrain updates.The truncated weights suppress excessive policy updates, while the KL term limits deviation from the reference policy.Source: 4) Adaptive Think Reward · p. 5
  • Reinforcement learning can develop strategies that adjust reasoning dynamically to scenarios of different complexity.The reward is designed to prevent over-reliance on manually defined scene tags as training progresses.Source: 4) Adaptive Think Reward · p. 5

A. Dataset and Metric

Experiments use NAVSIM and additional question-answering datasets, with PDMS as the closed-loop planning score. The implementation trains InternVL3-8B in three stages, ending with reinforcement learning.

1) Dataset:

  • Training uses NAVSIM reasoning data and reformatted visual VQA pairs from DriveLM, ImpromptuVLA, and LingoQA.NAVSIM is described as a planning-oriented autonomous-driving dataset built on OpenScene.Source: 1) Dataset · p. 5

2) Metric:

  • The NAVSIM benchmark uses the Predictive Driver Model Score (PDMS), which integrates five submetrics into a closed-loop planning score.Its submetrics are No At-Fault Collision, Drivable Area Compliance, Time-to-Collision, Comfort, and Ego Progress.Source: 2) Metric · p. 5

B. Implementation Details

  • InternVL3-8B is trained through driving-knowledge SFT, Think/Non-Think planning SFT, and reinforcement learning, each for two epochs.The stages use learning rates of 1×10−5, 4 × 10−5, and 2×10−6, respectively; the adaptive-reward threshold T is 0.9.Source: B. Implementation Details · p. 5

C. Performance Comparison

AdaThinkDrive leads the NAVSIM vision-only comparison and outperforms fixed Think and Non-Think RL modes, while using less inference time than Think RL. Its mode choices shift toward reasoning as scene complexity increases.

1) AdaThinkDrive Performance:

  • 90.3 PDMS places AdaThinkDrive 1.7 points above Hydra-NeXt, the prior best vision-only method.The result is reported under the vision-only setting; the paper also reports 93.0 PDMS in best-of-N planning with four generated candidates.Source: 1) AdaThinkDrive Performance · pp. 5–6, Table I · p. 6

2) Quantitative Evaluation of Adaptive Think:

1) AdaThinkDrive Performance:

  • 93.0 PDMS is reached in best-of-N planning by selecting the best of four generated trajectories with Navsim's reference evaluator.This is a separate planning setting from the vision-only comparison.Source: 1) AdaThinkDrive Performance · p. 6

2) Quantitative Evaluation of Adaptive Think:

3) Qualitative Analysis of Adaptive Think:

Qualitative examples contrast AdaThinkDrive with fixed-reasoning baselines: it avoids unnecessary reasoning in a simple scene and identifies a critical lead vehicle in a challenging one.

3) Qualitative Analysis of Adaptive Think:

  • In the simple scene, Think misclassifies a distant object as critical, leading to unnecessary reasoning and a trajectory that strays from the drivable area.AdaThinkDrive skips the redundant reasoning and directly predicts a smooth, accurate trajectory.Source: 3) Qualitative Analysis of Adaptive… · p. 7
  • In the challenging scene, Non-Think RL fails to assess the lead vehicle's distance and produces a risky plan.AdaThinkDrive identifies the critical object and generates a safe trajectory.Source: 3) Qualitative Analysis of Adaptive… · p. 7
  • The examples illustrate contrasting failure modes: unnecessary reasoning in the simple case and missed hazard assessment in the challenging case.AdaThinkDrive's responses differ across the two scene complexities.Source: 3) Qualitative Analysis of Adaptive… · p. 7

D. Ablation Studies

Ablations report gains from both driving QA pretraining and adaptive reinforcement learning, while reward comparisons show a further gain from Adaptive Think Reward. The conclusion frames adaptive reasoning as a way to align thinking with scene complexity.

2) Comparison of Reward Effectiveness:

1) Alabtion study on AdaThinkDrive:

  • 90.3 PDMS follows adding adaptive-think reinforcement learning and Adaptive Think Reward to QA pretraining and trajectory-data SFT.Trajectory-only SFT scores 86.2; adding large-scale driving QA pretraining raises PDMS to 87.5, a 1.3 increase, and the later adaptive RL stage adds 2.8.Source: 1) Alabtion study on AdaThinkDrive · p. 7

V. CONCLUSION

  • The conclusion presents Adaptive Think Reward as aligning reasoning behavior with scene complexity while targeting accurate and efficient decisions.The paper reports state-of-the-art NAVSIM performance for AdaThinkDrive.Source: §V · pp. 7–8
Loading 2509.13769v1…