PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, Jiaqi Ma

arXiv:2506.13757v3cs.CV

TL;DR

Existing VLA systems for autonomous driving can produce physically infeasible actions, complex intermediate structures, or unnecessarily long reasoning. AutoVLA unifies reasoning and action generation with discrete physical action tokens, dual thinking modes, and GRPO-based reinforcement fine-tuning. It reports competitive performance across open-loop and closed-loop benchmarks, while remaining highly GPU-dependent for deployment.

  • Problem

    Existing VLA models for autonomous driving face physically infeasible or structurally complex action generation and inflexible, inefficient reasoning.

  • Method

    AutoVLA integrates discrete physical action tokens into a pretrained VLM and uses SFT plus GRPO-based RFT to support fast and slow reasoning modes.

  • Results

    AutoVLA achieves competitive performance on both open-loop and closed-loop planning benchmarks and improves NAVSIM PDMS by 10.6% while reducing runtime by 66.8%.

  • Takeaways & Limitations

    The unified model supports adaptive reasoning by combining direct trajectory generation with longer chain-of-thought reasoning when needed.

  • Takeaways & Limitations

    Although inference reaches near-real-time operation at 1 Hz, AutoVLA remains highly GPU-dependent and requires substantial memory and computation.

Abstract

from arXiv · show

Recent advancements in Vision-Language-Action (VLA) models have shown promise for end-to-end autonomous driving by leveraging world knowledge and reasoning capabilities. However, current VLA models often struggle with physically infeasible action outputs, complex model structures, or unnecessarily long reasoning. In this paper, we propose AutoVLA, a novel VLA model that unifies reasoning and action generation within a single autoregressive generation model for end-to-end autonomous driving. AutoVLA performs semantic reasoning and trajectory planning directly from raw visual inputs and language instructions. We tokenize continuous trajectories into discrete, feasible actions, enabling direct integration into the language model. For training, we employ supervised fine-tuning to equip the model with dual thinking modes: fast thinking (trajectory-only) and slow thinking (enhanced with chain-of-thought reasoning). To further enhance planning performance and efficiency, we introduce a reinforcement fine-tuning method based on Group Relative Policy Optimization (GRPO), reducing unnecessary reasoning in straightforward scenarios. Extensive experiments across real-world and simulated datasets and benchmarks, including nuPlan, nuScenes, Waymo, and CARLA, demonstrate the competitive performance of AutoVLA in both open-loop and closed-loop settings. Qualitative results showcase the adaptive reasoning and accurate planning capabilities of AutoVLA in diverse scenarios.

1 Introduction

AutoVLA addresses limitations in end-to-end autonomous driving by unifying semantic reasoning and physically feasible trajectory generation in one VLA framework. Its supervised and reinforcement fine-tuning support adaptive reasoning, with experiments reporting strong performance across open-loop and closed-loop benchmarks.

  • Motivation: End-to-end systems can accumulate errors across perception, prediction, and planning, while conventional methods may lack world knowledge for challenging scenarios.VLMs and VLAs introduce reasoning and world knowledge, but existing VLA designs still face action-generation and reasoning-efficiency limitations.
  • Method: Supervised fine-tuning equips AutoVLA with fast trajectory-only and slow chain-of-thought reasoning modes, while reinforcement fine-tuning enables adaptive switching between them.The design targets both reasoning capability and reduced unnecessary reasoning across scenarios.
  • Method: AutoVLA integrates physical action tokens into a pretrained VLM, enabling direct autoregressive policy learning from visual observations and language instructions.The framework combines scene reasoning and action generation within a unified model rather than relying on downstream planners or decoders.
  • Results: RFT improves planning performance and runtime efficiency by minimizing redundant reasoning.The introduction attributes these gains to adaptive reasoning and reduced unnecessary computation.
  • Results: AutoVLA achieves superior performance across multiple autonomous driving benchmarks under both open-loop and closed-loop testing.The reported evaluation spans real-world and simulated datasets, including nuPlan, Waymo, nuScenes, and CARLA.

2 Related Work

Related work spans unified end-to-end driving architectures, language- or action-centric VLM/VLA systems, and reinforcement fine-tuning for trajectory planning. AutoVLA is positioned at the intersection of these directions, with RFT still relatively new in end-to-end VLM/VLA driving.

  • End-to-end Autonomous Driving: End-to-end driving methods unify tasks from perception through planning, while generative approaches target continuous and multimodal trajectory generation.The cited methods include unified Transformer architectures and generative trajectory models.
  • VLA and VLM for Autonomous Driving: VLM and VLA driving research includes language-centric scene understanding, high-level driving decisions, and efforts to bridge semantic reasoning with physical actions.The field is described as pursuing multiple directions toward connecting visual-language understanding with driving behavior.
  • Reinforcement Fine-tuning: Reinforcement fine-tuning has been used to improve performance and adaptability in language models and to align driving trajectories with safety constraints or human preferences.Prior autonomous-driving work applies RFT to trajectory generation and closed-loop reinforcement learning.
  • Reinforcement Fine-tuning: The application of reinforcement fine-tuning to end-to-end VLM- and VLA-based autonomous driving remains nascent.The related-work discussion contrasts this emerging area with prior RFT applications in other driving and language-model settings.

3 AutoVLA

AutoVLA unifies visual-language reasoning and physical trajectory generation in one autoregressive model. It is trained with supervised and reinforcement fine-tuning to support adaptive reasoning and task-specific planning optimization.

  • Framework: AutoVLA combines a VLM backbone with physical action-token generation in a unified autoregressive Transformer.The model processes visual and textual inputs and outputs reasoning and action tokens corresponding to vehicle movements.
  • Framework: AutoVLA accepts multi-view, multi-frame camera data, navigation instructions, and ego-vehicle states for scene reasoning and trajectory planning.The camera inputs include three viewpoints and four sequential frames, while the state includes velocity, acceleration, and historical actions.
  • Framework: Continuous vehicle trajectories are discretized into action tokens represented by short-term position and heading movements.This converts trajectory planning into a next-token prediction problem while maintaining physically feasible trajectory decoding.
  • Supervised Fine-tuning: Supervised fine-tuning trains both trajectory-only fast thinking and chain-of-thought slow thinking using corresponding action and reasoning sequences.The training data contains either final action tokens alone or reasoning followed by action tokens.
  • Supervised Fine-tuning: The supervised objective combines causal language-modeling loss for reasoning with an auxiliary action-token loss for planning accuracy.The output sequence contains language tokens followed by action tokens, and per-sample weighting addresses the imbalance between reasoning and action-only data.
  • Reinforcement Fine-tuning: Reinforcement fine-tuning with GRPO uses group-relative advantages and task-specific rewards to improve planning while reducing unnecessary reasoning.The reward combines a driving reward with a chain-of-thought penalty; for nuPlan, the driving reward uses PDMS.

4 Experiments

AutoVLA is evaluated across real-world and simulated datasets using open-loop and closed-loop benchmarks, with experiments covering scaling, reinforcement fine-tuning, tokenization, and training settings. Results report improved planning, adaptive reasoning efficiency, competitive benchmark performance, and advantages for physical action tokenization.

  • Data Scaling Results: Increasing training data consistently improves planning on nuPlan and nuScenes, while CoT supervision becomes advantageous at larger dataset sizes.On nuPlan, CoT does not outperform action-only below 50k samples; on nuScenes, CoT increasingly outperforms action-only in L2 distance and collision rate.
  • RFT Performance: 10.6% higher PDMS and 66.8% lower runtime result from reinforcement fine-tuning on the NAVSIM testing set.The runtime reduction is averaged over 500 testing scenarios, while the reward curve shows progressive policy improvement during RFT.
  • RFT Performance: RFT reduces unnecessary slow reasoning in simple scenarios and produces better trajectories than SFT in qualitative comparisons.A CoT length penalty encourages fast thinking for straightforward cases, while the RFT policy is optimized using a PDMS-based reward.
  • nuPlan Benchmark Results: AutoVLA achieves competitive NAVSIM results after RFT, with best-of-N planning attaining the highest PDMS.Best-of-N uses an oracle scorer to select the optimal trajectory from six generated candidates.
  • Waymo E2E Performance: Pretraining on nuPlan and nuScenes, CoT supervision, and RFT successively improve Waymo performance, with ADE-rewarded RFT achieving the best overall RFS.A construction-zone example demonstrates reasoning about occlusions and generating detour plans.
  • CARLA Closed-loop Performance: AutoVLA outperforms existing end-to-end models in overall driving score and success rate on the CARLA Bench2Drive closed-loop benchmark.The model is trained with both trajectory-only and CoT data and tested at a planning frequency of 2 Hz.

5 Conclusions

AutoVLA unifies scene reasoning and action generation in one autoregressive model, supporting fast and slow thinking while achieving competitive open- and closed-loop planning performance. Its dual-process adaptation reaches near-real-time inference but remains highly GPU-dependent.

  • AutoVLA unifies scene reasoning and action generation within a single autoregressive model.
  • SFT supports fast trajectory-only thinking and slow thinking with long chain-of-thought reasoning.
  • RFT penalizes unnecessary reasoning and aligns action generation with reward functions to improve performance and efficiency.
  • AutoVLA achieves competitive performance on open-loop and closed-loop planning benchmarks and exhibits strong reasoning capabilities.
  • 1 Hz inference is near-real-time, but the model remains highly GPU-dependent and requires significant memory and computing.

AutoVLA Supplementary Material

AutoVLA represents short-term feasible vehicle maneuvers as discrete physical action tokens, allowing continuous trajectories to be generated and reconstructed within the language model.

  • The action codebook is built from clustered vehicle motion segments sampled from the Waymo Open Motion Dataset.
  • Each sampled segment represents 0.5 seconds of motion and is characterized by displacement and heading change.
  • K-Disk clustering selects representative segments separated by δ = 0.05 m using average contour distance.
  • The resulting codebook contains K = 2048 tokens, each encoding a distinct physically feasible short-term vehicle behavior.
  • During training, continuous trajectories map to nearest codebook tokens; during inference, generated tokens are converted back into motion segments.

B Reasoning Data Collection

The reasoning-data pipeline uses a capable vision-language model, structured prompts, and human quality checks to generate chain-of-thought annotations for autonomous driving scenarios.

  • Reasoning Data Collection: An automated pipeline uses Qwen2.5-VL-72B to reduce human annotation reliance and distill reasoning knowledge into a compact model.
  • Prompting: The system prompt specifies the model role, chain-of-thought format, and representative examples organized around structured reasoning steps.
  • Prompting: The user message combines driving instructions, ego states, multi-view cameras, and ground-truth meta-actions to guide causal explanations.
  • Annotation Pipeline: Generated reasoning is capped at 700 textual tokens and incorporates reformatted answers from the DriveLM VQA dataset.
  • Quality Check: Human annotators assessed 3,000 samples, obtaining 88.8% overall accuracy under a binary quality criterion.
  • Examples: Examples show the pipeline reasoning about stop-sign progression, construction-related road control, pedestrians, traffic lights, and turning actions.

D.3 Implementation Details

RFT uses LoRA fine-tuning with a frozen vision encoder and the pretrained SFT model as the reference policy, reducing training costs and memory consumption.

  • RFT uses navtrain from nuPlan and validation data from Waymo for the corresponding benchmarks.
  • The pretrained vision encoder is frozen, while LoRA fine-tunes the model with rank 8, alpha 8, and dropout 0.1.
  • The pretrained SFT model serves as the reference policy during optimization.

E.1 Data Preprocessing

AutoVLA standardizes heterogeneous driving data into trajectory, visual-history, reasoning, and vehicle-state inputs, then evaluates planning across multiple benchmark protocols. Results on nuScenes show competitive planning, while reasoning helps most in complex scenarios rather than straightforward ones.

  • Data Preprocessing: The preprocessing pipeline standardizes trajectories, headings, multi-view image histories, chain-of-thought annotations, and vehicle states across datasets.Images contain four consecutive frames captured at 2 Hz, providing two seconds of history.
  • Data Preprocessing: nuPlan reasoning data combines 45.6k generated reasoning samples with remaining trajectory-only training data, using navtest for evaluation.The nuPlan trainval split supplies the scenarios, and NAVSIM defines the test protocol.
  • Data Preprocessing: nuScenes uses structured reasoning annotations for DriveLM-covered samples and trajectory supervision for other training samples, with validation used for testing.Evaluation reports L2 Distance and Collision Rate at 1, 2, and 3 seconds rather than averaging across horizons.
  • Data Preprocessing: Waymo and CARLA use dataset-specific sliding-window preparation, with Waymo addressing stationary-heading noise and CARLA using four-frame single-view inputs.CARLA trajectories are downsampled from 4 Hz to 2 Hz for closed-loop training and testing.
  • Evaluation: AutoVLA uses PDMS for nuPlan, L2 Distance and Collision Rate for nuScenes, RFS for Waymo, and four closed-loop metrics for CARLA.CARLA metrics are Driving Score, Success Rate, Efficiency, and Comfortness.
  • Reasoning Modes: Deterministic sampling produces fast thinking, whereas higher-diversity sampling supports slower, more elaborate reasoning chains.The fast setting uses temperature 0.1, top-p 0.01, and top-k 1; the slow setting uses temperature 1.0, top-p 0.5, and top-k 20.
  • nuScenes Results: On nuScenes, AutoVLA achieves competitive planning performance and generates safe trajectories with coherent, context-aware reasoning.Many nuScenes scenarios are straightforward, which may explain why adding reasoning does not improve quantitative metrics.

F.2 nuPlan Results

On nuPlan, AutoVLA uses fast thinking for simple curved roads and intersections, while slow chain-of-thought reasoning improves planning in complex, regulation-heavy scenes. Reinforcement fine-tuning removes redundant reasoning without sacrificing complex-scenario reasoning.

  • F.2 nuPlan Results: AutoVLA generates high-quality trajectories through fast thinking in relatively simple nuPlan scenarios such as curved roads and intersections.These scenarios do not require extensive reasoning according to the qualitative results.
  • F.2 nuPlan Results: Complex nuPlan scenes with numerous traffic regulations receive better, regulation-compliant plans when AutoVLA uses slow thinking with chain-of-thought reasoning.The qualitative comparison contrasts direct trajectory generation with reasoning-enhanced planning.
  • F.2 nuPlan Results: After reinforcement fine-tuning, AutoVLA preserves reasoning capabilities in complex scenarios while removing redundant reasoning.The reported qualitative comparison also indicates enhanced planning performance in complex scenarios.

F.3 Waymo Results

AutoVLA is pretrained on combined nuPlan and nuScenes data, fine-tuned on Waymo, and further optimized with reinforcement fine-tuning using ADE because RFS labels are scarce. It achieves strong Waymo benchmark performance and handles complex scenes with safe, context-aware trajectories.

  • F.3 Waymo Results: AutoVLA pretrains on combined nuPlan and nuScenes data before Waymo fine-tuning, then applies reinforcement fine-tuning with ADE as the primary reward.Only 480 RFS-labeled validation samples are available, motivating the ADE reward choice.
  • F.3 Waymo Results: AutoVLA ranks highly in Waymo RFS Overall and ADE and achieves the top RFS Spotlight score for challenging scenarios.These leaderboard results are reported as of May 22, 2025.
  • F.3 Waymo Results: Qualitative Waymo results show safe, context-aware trajectories and reasoning for complex nighttime traffic involving vehicles, a cyclist, and a green traffic light.The reasoning identifies maintaining a safe following distance as important for avoiding collisions.
  • F.3 Waymo Results: Multi-camera input consistently improves driving performance, while reasoning improves performance over action-only training when using only Waymo data.Pretraining on nuX also provides a substantial performance boost in the ablation study.

F.4 CARLA Results

AutoVLA is evaluated in CARLA closed loop with 2 Hz replanning from front-camera histories, route instructions, and vehicle state. Representative scenarios show safe cut-in response and smooth left-turn execution, alongside a stated need for stronger safety safeguards.

  • F.4 CARLA Results: CARLA evaluation queries AutoVLA every 0.5 seconds and converts planned trajectories into control commands.Inputs include four front-camera RGB images, route-derived high-level instructions, and IMU and speedometer state.
  • F.4 CARLA Results: AutoVLA successfully responds to a cut-in vehicle and executes a smooth left turn in representative CARLA closed-loop scenarios.The reported evaluation uses the CARLA simulator and illustrates two qualitative scenarios.
  • Broader Impacts: Because autonomous driving is safety-critical and language guidance is integrated, the paper calls for safeguards against adversarial attacks and unsafe instructions.It recommends secure vehicle communication and gated model-update release rather than online continual reinforcement fine-tuning on individual vehicles.
Loading 2506.13757v3…