PaperlayerView exact revision on arXiv

Source-linked AI summary

Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

NVIDIA, :, Yan Wang, Wenjie Luo, Junjie Bai, Yulong Cao, Tong Che, Ke Chen, Yuxiao Chen, Jenna Diamond, Yifan Ding, Wenhao Ding, Liang Feng, Greg Heinrich, Jack Huang, Peter Karkus, Boyi Li, Pinyi Li, Tsung-Yi Lin, Dongran Liu, Ming-Yu Liu, Langechuan Liu, Zhijian Liu, Jason Lu, Yunxiang Mao, Pavlo Molchanov, Lindsey Pavao, Zhenghao Peng, Mike Ranzinger, Ed Schmerling, Shida Shen, Yunfei Shi, Sarah Tariq, Ran Tian, Tilman Wekel, Xinshuo Weng, Tianjun Xiao, Eric Yang, Xiaodong Yang, Yurong You, Xiaohui Zeng, Wenyuan Zhang, Boris Ivanovic, Marco Pavone

arXiv:2511.00088v2cs.ROcs.AIcs.LG

TL;DR

Autonomous-driving VLMs often lack explicit or structured reasoning, and generalization remains difficult in ambiguous long-tail scenarios. Alpamayo-R1 links causally structured reasoning to trajectory planning, reporting substantial gains over end-to-end baselines, especially in rare safety-critical scenarios, with 99ms end-to-end latency.

  • Problem

    Many driving VLMs lack explicit reasoning or use unstructured reasoning, with generalization in ambiguous or compositional long-tail scenarios remaining difficult.

    Source: §1 · p. 2

  • Method

    AR1 pairs Chain of Causation reasoning with a Cosmos-Reason VLM and diffusion-based trajectory decoder, using staged SFT and RL to align reasoning with driving behavior.

    Source: §1 · p. 2, §3 · p. 6, §7 · p. 32

  • Results

    AR1 achieves substantial improvements over end-to-end baselines, with the largest gains in rare, safety-critical scenarios, while maintaining 99ms end-to-end latency.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The conclusion presents AR1 as an approach to improving autonomous driving performance in long-tail, safety-critical scenarios.

    Source: §7 · p. 32

    The current architecture generates reasoning traces for every input; adaptive invocation is left as a future direction.

    Source: §7 · p. 32

Abstract

from arXiv · show

End-to-end architectures trained via imitation learning have advanced autonomous driving by scaling model size and data, yet performance remains brittle in safety-critical long-tail scenarios where supervision is sparse and causal understanding is limited. We introduce Alpamayo-R1 (AR1), a vision-language-action model (VLA) that integrates Chain of Causation reasoning with trajectory planning for complex driving scenarios. Our approach features three key innovations: (1) the Chain of Causation (CoC) dataset, built through a hybrid auto-labeling and human-in-the-loop pipeline producing decision-grounded, causally linked reasoning traces aligned with driving behaviors; (2) a modular VLA architecture combining Cosmos-Reason, a vision-language model pre-trained for Physical AI, with a diffusion-based trajectory decoder that generates dynamically feasible trajectories in real time; (3) a multi-stage training strategy using supervised fine-tuning to elicit reasoning and reinforcement learning (RL) to enforce reasoning-action consistency and optimize reasoning quality. AR1 achieves up to a 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline, with a 35% reduction in close encounter rate in closed-loop simulation. RL post-training improves reasoning quality by 45% and reasoning-action consistency by 37%. Model scaling from 0.5B to 7B parameters shows consistent improvements. On-vehicle road tests confirm real-time performance (99 ms latency) and successful urban deployment. By bridging interpretable reasoning with precise control, AR1 demonstrates a practical path towards Level 4 autonomous driving. Model weights are available at https://huggingface.co/nvidia/Alpamayo-R1-10B with inference code at https://github.com/NVlabs/alpamayo.

1. Introduction

Alpamayo-R1 makes causally grounded reasoning a functional part of autonomous-driving trajectory planning, combining structured reasoning data, a diffusion-based trajectory decoder, and multi-stage training. Evaluations report improved performance over end-to-end baselines, including in rare safety-critical scenarios, with real-time latency.

Motivation

  • End-to-end systems remain fragile in long-tail safety-critical situations, while many driving VLM/VLAs lack explicit reasoning or use unstructured reasoning that overlooks driving structure.The paper identifies sparse supervision, ambiguous or compositional scenarios, and structural knowledge such as lane geometry, traffic rules, map priors, agent interactions, and dynamic constraints as challenges.Source: §1 · pp. 1–2
  • AR1’s design principle is to link observed scene evidence causally to driving decisions, which then condition low-level trajectories, making reasoning functional rather than merely explanatory.The paper argues that this alignment can improve training efficiency and closed-loop performance, particularly in safety-critical long-tail events.Source: §1 · p. 2

Contributions

  • The Chain of Causation labeling framework creates decision-grounded, causally linked reasoning traces through a hybrid human-in-the-loop and auto-labeling pipeline.The traces are aligned with driving scenarios and intended for scalable, high-quality data generation.Source: §1 · p. 2
  • A flow-matching diffusion action-expert decoder generates continuous, multimodal trajectory plans aligned with language reasoning while meeting real-time inference requirements.The decoder is designed to connect language outputs with trajectory planning.Source: §1 · p. 2
  • Multi-stage training builds on Cosmos-Reason, adds trajectory prediction, uses supervised fine-tuning on CoC data to elicit reasoning, and applies reinforcement learning to improve reasoning and action alignment.The stated objectives also include improving trajectory quality.Source: §1 · p. 2

Evaluation overview

  • Extensive open-loop, closed-loop simulation, and onboard evaluations show improvements over end-to-end baselines, with the largest gains in rare safety-critical scenarios and 99 ms end-to-end latency.The introduction reports real-time inference performance alongside those evaluation results.Source: §1 · p. 2

2. Related Work

The related work reviews VLMs and VLAs, reasoning-augmented driving, post-training alignment, and driving datasets, identifying gaps that motivate causally grounded reasoning linked to driving actions.

2.1. VLMs and VLAs in Autonomous Driving

  • Driving VLM work spans text-based planning, multimodal scene and trajectory prediction, and action-oriented VLAs, but many approaches remain reactive and struggle with ambiguous, long-horizon cases (Mao et al., 2023; Hwang et al., 2024; Zhou et al., 2025; Renz et al., 2025).Prior VLA approaches include waypoint generation, adaptive think-versus-act control, inverse-RL safety-efficiency balancing, and long-tail scenario targeting.Source: §2.1 · p. 3

2.2. Reasoning VLAs in Autonomous Driving

  • Reasoning VLAs use adaptive CoT, self-reflection, retrieval, temporal reasoning, memory, and tools, yet often lack causal grounding and reasoning-action consistency, motivating CoC and post-training RL (Luo et al., 2025; Yuan et al., 2025; Rowe et al., 2025).Poutine topped the 2025 Waymo Vision-Based End-to-End Driving Challenge, while the paper identifies free-form reasoning’s causal and action-consistency gaps.Source: §2.2 · pp. 3–4

2.3. Post-training Alignment

  • Imitation objectives can diverge from intended rewards, motivating RLHF and verifiable-reward RL; prior embodied VLA alignment targets action outcomes, while this work also aligns causal rationale (Christiano et al., 2017; DeepSeek-AI, 2025).The related work cites outcome-based RL as improving reasoning quality and frames causal, context-grounded rationale as a complementary alignment target for safety-critical driving.Source: §2.3 · p. 4

2.4. Vision-Language Datasets for Autonomous Driving

  • Driving datasets progress from perception and interaction annotations to planning-oriented and explicit reasoning traces, but often lack actionable causal links, motivating CoC’s decision-grounded, causally linked traces.Existing traces may use vague advice, superficial causal factors, or future events unavailable at decision time.Source: §2.4 · pp. 4–5

3. Building a Reasoning VLA Architecture

Alpamayo-R1 combines a vision-language backbone with domain-specific vision encoding and trajectory decoding to connect driving reasoning with action prediction. Its design targets multi-camera, multi-timestep awareness, evidence-grounded causal reasoning, real-time trajectory generation, and alignment between reasoning and executed actions.

3. Building a Reasoning VLA Architecture

  • AR1 processes multi-camera observations, historical egomotion, and optional instructions through Cosmos-Reason to produce reasoning traces, meta-actions, and future trajectories.The task is framed as sequential prediction from past observations to a 6.4-second future trajectory sampled at 10 Hz.Source: §3 · p. 6

3.1. VLM Backbone: Cosmos-Reason

  • Cosmos-Reason supplies the VLM backbone, pretrained on 3.7M VQA samples and including 24.7K curated driving video-VQA samples with scene descriptions, difficulty annotations, and distilled reasoning traces.Supplementary driving data adds 100K samples annotated with critical objects and reasoning for the next action.Source: §3.1 · p. 6

3.2. Domain-Specific Adaptations

  • AR1’s domain-specific adaptations address two deployment gaps: efficient encoding of multi-camera, multi-timestep inputs and precise trajectory decoding for real-time control.The section identifies vision encoding and action decoding as the principal design dimensions for adapting VLMs into autonomous-driving policy VLAs.Source: §3.2 · p. 7, §3.2.2 · p. 10

3.2.1. Vision Encoding

  • 288 tokens represent one timestep in a 7-camera setup—about 41.1 tokens per image, 3.9× fewer than single-image tokenization—using fixed-size triplanes.The cited example uses triplane grid sizes 96, 96, and 48 with patchification values of 8; token count is independent of camera count and resolution.Source: §3.2.1 · pp. 7–8

3.2.2. Trajectory Decoding

  • AR1 predicts unicycle-dynamics control inputs instead of raw position waypoints, then decodes trajectories through discrete VLM tokens and a continuous flow-matching action expert.The control representation is intended to reduce sensitivity to sensor noise and improve closed-loop performance; the combined decoding strategy supports faster trajectory decoding.Source: §3.2.2 · pp. 8–9

4. Chain of Causation Dataset: Learning Causally Grounded Reasoning VLAs

The Chain of Causation dataset structures driving reasoning around explicit decisions and observable causes, linking concise explanations to ego actions. Its curation and hybrid labeling procedures aim to produce scalable, causally grounded supervision and evaluation.

4. Chain of Causation Dataset: Learning Causally Grounded Reasoning VLAs

  • CoC addresses vague action descriptions, superficial reasoning, and future information leakage by grounding traces in decisions and causes observable in the history window.Each trace is associated with an explicit driving decision and includes only factors that motivate it; keyframes separate historical from future video segments.Source: §4 · pp. 10–11

4.1. Structured Chain of Causation

  • CoC samples combine a driving decision, open-ended critical components, and a composed trace, enforcing decision grounding, historical evidence, and annotation economy.The decision inventory is closed-set, while labelers can specify only the open-ended factors that directly influence the decision.Source: §4.1 · pp. 11–12

4.2. Data Curation

  • Rule-based curation selects and balances reactive and proactive clips; reactive keyframes precede behavior changes, while proactive scenarios use a keyframe range.Clips contain 20 seconds of data, with a 2-second history used to predict a 6-second future; reactive keyframes are typically about 0.5 seconds before a behavior change.Source: §4.2 · pp. 12–13

4.3. Hybrid Labeling Procedure

  • A hybrid human and auto-labeling procedure scales CoC data while retaining human-verified examples for SFT and evaluation.Human-verified data is on the order of 10% of the total and also supports auto-label evaluation.Source: §4.3 · p. 14

4.3.1. Human Labeling

  • Human annotators first identify causal components in the 2-second history, then filter unsafe data, select post-keyframe decisions, and write traces from those causes.A separate annotator checks each instance, and 10%–20% are additionally audited based on annotator performance.Source: §4.3.1 · p. 14

4.3.2. Auto-Labeling

  • Rule-based detectors identify meta-action transitions as auto-labeling keyframes, and VLMs produce structured decisions, causal factors, and concise traces from video and auxiliary signals.The pipeline uses 2 Hz video and separates the 2-second history used to identify causes from the subsequent 6-second future used to resolve the decision.Source: §4.3.2 · pp. 14–16

4.3.3. Evaluation

  • Structured CoC traces improve the causal relationship score by 132.8% relative to free-form reasoning traces; GPT-5 auto-evaluation aligns with human evaluation at 92%.The authors also argue that auto-labels need not be perfect: their primary value is enabling large-scale SFT that improves generalization across diverse driving scenarios.Source: §4.3.3 · pp. 16–17

5. Training Strategy

The training strategy progressively equips the VLM with action generation, CoC-grounded reasoning, and RL post-training to refine reasoning quality, align explanations with actions, and improve trajectory quality.

5.1. Action Modality Injection

  • Discrete trajectory tokens train the VLM jointly with reasoning, while a continuous flow-matching action expert produces feasible, multimodal trajectories faster at inference.Training trajectories use 64 waypoints with acceleration and curvature values, or 128 discrete tokens; inference uses the continuous decoder instead.Source: §5.1 · pp. 17–18

5.2. Eliciting Reasoning

  • CoC SFT teaches causal explanations tied to expert trajectories, but can overfit annotation artifacts, memorize patterns, hallucinate scene factors, and produce reasoning-action contradictions.Training samples pair multi-camera observations and CoC traces with ground-truth control trajectories.Source: §5.2 · pp. 18–19

5.3. RL-based Post-Training

  • RL post-training evaluates the model’s own rollouts with reasoning-quality, reasoning-action consistency, and trajectory-quality rewards, targeting inference errors and improving robustness and generalization.Unlike teacher-forced SFT, its feedback assesses both causal correctness and alignment with predicted actions.Source: §5.3 · p. 19, Figure 6 · p. 19

5.3.1. Post-Training Algorithm

  • GRPO optimizes relative advantages across sampled rollouts and uses KL regularization to limit reward over-optimization while preserving learned language and behavior priors.The reference policy is typically the SFT model.Source: §5.3.1 · p. 20

5.3.2. Reward Model

  • Large reasoning models score predicted traces against ground truth and scene observations for behavioral consistency and causal reasoning quality, providing feedback to refine grounded explanations.The critic is used for evaluation because its verification capabilities can be stronger than its driving-specific generation capabilities.Source: §5.3.2 · p. 20

Prompt : LLM Reasoning Grading Rubric

  • The rubric combines a 0–5 reasoning score with binary reasoning–meta-action consistency and trajectory terms for imitation, collision avoidance, and jerk.Consistency is rewarded only when the parsed intended behavior matches trajectory-derived meta-actions on both longitudinal and lateral axes.Source: Prompt : LLM Reasoning Grading Rubric · p. 21

5.3.3. Post-Training Data Curation for Cost-Effective Training

  • To reduce RL compute and data costs, post-training prioritizes samples where model logits disagree with explicit rewards and mixes in random samples for diversity and stability.The disagreement identifies samples where the model’s implicit preference conflicts with the reward signal.Source: §5.3.3 · pp. 21–22

5.3.4. Post-Training Infrastructure

  • A customized Cosmos-RL framework supports distributed loading, mixed-parallelism training, vLLM rollouts, and reward computation across multiple GPU nodes.The infrastructure is designed for scalable multimodal RL in autonomous-driving reasoning tasks.Source: §5.3.4 · p. 22

6. Experiments · 6.1. Evaluation Protocol

Alpamayo-R1 is reported to outperform a trajectory-only baseline, especially in challenging scenarios. The experiments evaluate prediction, closed-loop driving, architectural choices, and on-vehicle performance.

6. Experiments

  • Alpamayo-R1 significantly outperforms the trajectory-only baseline, particularly in challenging scenarios that require complex reasoning for better driving decisions.Source: §6 · p. 22
  • Table 6 reports AR1 paired entries of 0.275/0.908 at 3B and 0.254/0.794 at 0.5B, compared with base-model entries of 0.291/0.977 and 0.264/0.848.Models are evaluated on held-out CoC test data; the supplied passages do not label the two table values.Source: Table 6 · p. 23

6.1. Evaluation Protocol

  • The evaluation combines open-loop prediction on nominal and long-tail scenarios, AlpaSim closed-loop simulation, ablations of model and decoding choices, and on-vehicle road tests.Source: §6.1 · p. 23
  • Training and evaluation use geographically diverse internal US and EU driving data, with evaluation regions held out; the dataset spans 80,000 hours, over 2,500 cities, and 25 countries.The data covers varied roads, weather, times, and traffic densities.Source: §6.1 · p. 23
  • The CoC dataset contains 700K structured-reasoning video segments for fine-tuning reasoning and RL-based post-training alignment.Source: §6.1 · p. 24
  • Open-loop prediction is evaluated over a 6.4-second horizon using minADE6 across six predictions and ADE averaged over future timesteps.Source: §6.1 · p. 24
  • AlpaSim uses reconstructed 3D Gaussian Splatting scenes, an MPC tracking controller, bicycle-model vehicle dynamics, and recorded traffic trajectories; evaluation covers 75 challenging 20-second scenarios.The simulator is described by NVIDIA et al. (2025), with neural reconstruction technology attributed to Wu et al. (2025); runs stop at a close encounter or off-road event, and events after deviations over 4 m are excluded.Source: §6.1 · pp. 24–25
  • Closed-loop metrics are close encounter rate (all and at-fault), offroad rate, and AlpaSim score (all and at-fault), with at-fault measures excluding rear-end encounters.AlpaSim score measures average distance driven between offroad or close-encounter events.Source: §6.1 · pp. 24–25

6.2. Policy Improvements from Reasoning

Fine-tuning with Chain-of-Causation (CoC) reasoning improves open-loop trajectory prediction and closed-loop driving compared with trajectory-only policies. The evaluation tests held-out CoC data with and without route information.

Evaluation Setup

  • The evaluation fine-tunes a base model on CoC using trajectory-only, meta-action-plus-trajectory, or chain-of-causation reasoning-plus-trajectory supervision.At inference, CoC reasoning models generate explicit reasoning alongside trajectory predictions; all models are tested on held-out CoC data with and without route information.Source: §6.2 · p. 25

Open-Loop Improvements

  • AR1 improves open-loop prediction by 12% over the trajectory-only baseline in challenging scenarios (0.868m vs 0.994m), and by 4.8% with route information (0.794m vs 0.834m).Without route information, AR1 achieves a minADE6 of 0.955m at 6.4s, a 4.1% improvement over the base model; AR1-3B achieves 0.908m. CoC-enabled models also use route guidance and anticipate future interactions; qualitative examples show them yielding to vehicles where baselines fail.Source: §6.2 · p. 25

Closed-Loop Improvements

  • AR1 reduces the closed-loop close encounter rate by 35% versus trajectory-only (11% vs 17%) and raises the AlpaSim score from 0.38 to 0.50.Its off-road rate is comparable to the trajectory-only baseline (4% vs 3%).Source: §6.2 · p. 25

6.3. Improvements of Reasoning, Consistency, and Safety via RL Post-Training · 6.4. Public Benchmark Evaluation

RL post-training improves reasoning quality, reasoning–action consistency, and motion safety, while public evaluations show stronger open- and closed-loop performance for the 10B model than the 0.5B model. The sections also describe the reward components and evaluation setup underlying these findings.

6.3. Improvements of Reasoning, Consistency, and Safety via RL Post-Training

  • RL post-training addresses SFT’s lack of guarantees for causal grounding, reasoning–action faithfulness, and alignment with human driving norms by optimizing reasoning quality, consistency, and trajectory quality.The reward-component study post-trains a 0.5B AR1 model fine-tuned on CoC data.Source: §6.3 · pp. 25–26
  • Adding the reasoning reward improves the average reasoning score of the most-likely rollout among six generations by approximately 45% (3.1→4.5).Post-training also corrects missed construction barriers and pedestrians clearing the path in the illustrated examples.Source: §6.3 · p. 26, Figure 10 · p. 27
  • Joint reasoning and consistency rewards reduce most-likely mode ADE by 9.4% (2.12m→1.92m), improve reasoning score by 45% (3.1→4.5), and increase reasoning–action consistency by 37% (0.62→0.85).The evaluation uses the most-likely rollout among six generated rollouts.Source: §6.3 · p. 27, Table 9 · p. 27
  • Optimizing reasoning alone can worsen ADE and reasoning–action consistency, while the consistency reward anchors explanations to physically realizable actions.Examples show the aligned model completing a stop-and-go sequence and maintaining lane position when following a lead vehicle.Source: §6.3 · pp. 27–28, Figure 11 · p. 28
  • Adding a safety reward further reduces close encounter rate and stabilizes trajectories; the full reward configuration has the lowest close encounter rate while retaining ADE and consistency improvements.The reward penalizes unsafe or physically implausible trajectories.Source: Table 9 · p. 27, §6.3 · p. 28

6.4. Public Benchmark Evaluation

  • The public evaluation tests Alpamayo-R1-10B, using Cosmos-Reason with a 2B diffusion trajectory decoder, and Alpamayo-R1-0.5B on PhysicalAI-AV and AlpaSim; both generate CoC traces and trajectories.Open-loop evaluation uses held-out PhysicalAI-AV examples, and closed-loop evaluation uses challenging AlpaSim scenarios.Source: §6.4 · p. 28, Table 10 · p. 29
  • On 920 AlpaSim scenarios, Alpamayo-R1-10B vs Alpamayo-R1-0.5B yields a 16% reduction in off-road rate (16% vs 19%), 55% reduction in close encounter rate (4% vs 9%), and more than 2× AlpaSim score (0.72 vs 0.35).Closed-loop metrics are at-fault; both models predict CoC reasoning traces and trajectories.Source: Table 10 · p. 29, §6.4 · p. 29
  • On 644 held-out PhysicalAI-AV examples, Alpamayo-R1-10B achieves minADE6 of 0.849m at 6.4s, a 7.0% improvement over Alpamayo-R1-0.5B (0.913m).The reported result demonstrates the benefit of scaling the VLM backbone.Source: §6.4 · p. 28, Table 10 · p. 29

6.5. Ablation: VLM Backbone Selection

The ablations test model scale, training-data scale, and Physical-AI-focused pre-training; together, they motivate Cosmos-Reason as Alpamayo-R1’s backbone.

6.5.1. Model Size Ablation

  • 11% lower minADE6 for the 7B model than the 0.5B baseline showed that increasing general-purpose VLM backbone size improved open-loop trajectory prediction.All variants used identical data and were evaluated on the held-out 𝒟overall set without route information using minADE6 over a 6.4s horizon.Source: §6.5.1 · p. 29

6.5.2. Data Scaling

  • 0.874m at 2M video segments was the best reported performance, a 14.0% improvement; 500k reached 0.880m, a 13.4% improvement over 100k.The 100k model scored 1.111m without early stopping and 1.016m with early stopping; experiments held architecture, hyperparameters, and total training steps constant.Source: Figure 13 · p. 30, §6.5.2 · p. 30

6.5.3. Cosmos-Reason Physical AI Capabilities

  • Alpamayo-R1 adopts Cosmos-Reason because Physical-AI-focused pre-training adds driving-relevant capabilities beyond model scale alone.The section describes Cosmos-Reason as a strong foundation with Physical AI capabilities that general-purpose VLMs may lack.Source: §6.5.3 · pp. 30–31, §6.5.1 · p. 30
  • 66.2% accuracy let Cosmos-Reason-7B outperform tested zero-shot LingoQA (Marcu et al., 2024) baselines: GPT-4V (59.6%), Qwen2-VL-7B (52.6%), Qwen2.5-VL-7B (62.2%), InternVL3.5-8B (58.6%), and DeepSeek-VL-7B (46.4%).The paper attributes the advantage to Physical-AI-focused supervised fine-tuning, which improves driving-scene understanding.Source: §6.5.3 · p. 30, Table 11 · p. 31

6.6. Ablation: Action Modality Injection · 6.7. Ablation: Efficient Vision Encoding · 6.8. On-Vehicle Road Tests

AR1’s continuous flow-matching action decoding improves open- and closed-loop metrics, efficient vision encoders can reduce sensor tokens while maintaining driving quality, and on-vehicle tests demonstrate urban operation with 99ms inference latency.

6.6. Ablation: Action Modality Injection

  • Flow-matching trajectory decoding improves open- and closed-loop metrics, comfort, and inference speed over an otherwise matched model predicting six discrete trajectory tokens.The flow-matching inference uses five steps (δt=0.2) to reduce latency with negligible performance degradation.Source: §6.6 · p. 31

6.7. Ablation: Efficient Vision Encoding

  • Triplane tokenization achieves nearly identical minADE6 to the baseline while adding 6.3M parameters and reducing sensor tokens 3.6×.The comparison varies vision encoders in a four-camera setup and evaluates end-to-end open-loop driving quality by minADE6.Source: §6.7 · p. 31, Table 13 · p. 32
  • Flex compresses sensor tokens by up to 20×, adds 61.6M parameters to the driving model, and matches baseline driving quality.Flex is attributed to Yang et al. (2025).Source: §6.7 · p. 31, Table 13 · p. 32
  • AR1 uses single-image tokenization by default, while the preferred encoder varies with camera count, history length, and camera resolution.Few cameras and short histories favor single-image tokenization; more cameras favor triplanes with short histories and Flex with long histories.Source: §6.7 · p. 31

6.8. On-Vehicle Road Tests

  • On-vehicle urban tests show AR1 navigating complex scenarios without human intervention and producing reasoning traces that lead to appropriate actions.The tests indicate that simulation improvements transfer to real-world autonomous driving scenarios.Source: §6.8 · pp. 31–32
  • 99ms end-to-end inference latency on an NVIDIA RTX 6000 Pro Blackwell meets the typically 100ms real-time requirement for autonomous driving.The visual-token and route-information prefilling stage generates the key-value cache used for reasoning and trajectory decoding.Source: Table 14 · p. 32, §6.8 · p. 32

7. Conclusion

Alpamayo-R1 combines causally grounded reasoning with trajectory prediction, and evaluations show consistent improvements over end-to-end baselines, especially in challenging scenarios with complex agent interactions. The conclusion also identifies future research directions and releases model weights and inference code for reproducibility.

Conclusion

  • AR1 shows consistent improvements over end-to-end baselines, with pronounced gains on challenging scenarios involving complex agent interactions.The authors report evaluations across open-loop metrics, closed-loop simulation, and ablation studies.Source: §7 · p. 32
  • AR1 integrates structured chain-of-thought reasoning with trajectory prediction, using the CoC dataset and RL to align reasoning traces with driving behavior.The CoC dataset combines large-scale auto-labeling with human-in-the-loop labeling.Source: §7 · p. 32

Future Work

  • Future work targets hierarchical motion primitives, selective reasoning for safety-critical inputs, auxiliary visual tasks, and learned world models for forward and counterfactual simulation.Proposed auxiliary tasks include depth estimation, scene-flow prediction, and 3D Gaussian Splatting representations.Source: §7 · pp. 32–33

Open Source Release

  • The authors release AR1-10B weights and inference code, and report evaluation on the PhysicalAI-AV dataset and AlpaSim public scenario set for reproducible benchmarking.Source: §7 · p. 33

A. Contributors and Acknowledgments · A.1. Core Contributors

The core contributors covered dataset curation, reasoning-trace labeling, model and infrastructure development, training, and project leadership.

A.1. Core Contributors

  • TX, XW, YC, WD, and YW curated the autonomous-driving datasets and benchmarks.Source: §A.1 · p. 34
  • XW, YC, WD, BL, XY, and YW developed the reasoning-trace labeling pipeline and its infrastructure.Source: §A.1 · p. 34
  • YY, WL, YW, WD, TC, and RT built the supervised fine-tuning and reinforcement-learning infrastructure.Source: §A.1 · p. 34
  • BI and YW developed the vision encoder, while YY and YC built the flow-matching trajectory decoder.Source: §A.1 · p. 34
  • Teams trained the VLA models and Cosmos-Reason VLM backbone, and designed the data-mixture and reinforcement-learning post-training strategies.The passage attributes VLA supervised fine-tuning to YY, WL, YW, WD, JL, ZL, and PLL; Cosmos-Reason training to YW, WL, YY, XY, TL, and XZ; and post-training strategy and RL post-training to RT, TC, YW, WL, YY, and WD.Source: §A.1 · p. 34
  • YW and WL led the project, while MP conceived, coordinated, and guided the overall effort with BI supporting coordination and guidance.Source: §A.1 · p. 34

A.2. Contributors

The contributors covered end-to-end production development, data and reasoning pipelines, model training, evaluation, and on-vehicle testing.

A.2. Contributors

  • ST led end-to-end production development and provided key input on the data pipeline and model architecture.Source: §A.2 · p. 34
  • LP and JD led human annotation, while YS, EY, and TW built CoC labeling tools and SS developed an LLM-based evaluator.Source: §A.2 · p. 34
  • PM, GH, and MR trained the vision encoder; ML supported Cosmos-Reason, YD prepared cosmos AV training data, and ZP improved large-scale SFT workflows.Source: §A.2 · p. 34
  • FL and JB supported large-scale RL infrastructure, ES curated and preprocessed driving data, and DL, PL, and LM conducted on-vehicle tests and model profiling.Source: §A.2 · p. 34
  • KC, WZ, and JH improved the CoC auto-labeling pipeline.Source: §A.2 · p. 34

Related papers

Loading 2511.00088v2…