PaperlayerView exact revision on arXiv

Source-linked AI summary

NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning

Ishaan Rawal, Shubh Gupta, Yihan Hu, Wei Zhan

arXiv:2602.21172v3cs.AIcs.CV

TL;DR

Current VLAs require massive driving datasets and dense reasoning annotations, creating substantial data, annotation, training, and inference costs. NORD addresses this with reasoning-free, data-efficient training and Dr. GRPO, achieving competitive performance on NAVSIM and WaymoE2E while using less data and improving efficiency.

  • Problem

    Current VLAs rely on massive driving datasets and dense reasoning annotations, creating data, annotation, training, and inference costs that motivate reasoning-free, data-efficient alternatives.

  • Method

    NORD uses a reasoning-free, data-efficient VLA with Dr. GRPO, which removes the standard-deviation term from group-relative advantage to mitigate difficulty bias in weak-policy training.

  • Results

    NORD achieves competitive performance on NAVSIM and WaymoE2E without reasoning annotations and with at least 60% less data than reasoning VLAs, while improving inference time.

  • Takeaways & Limitations

    Efficient, high-performing VLAs can be trained without reasoning and large-scale datasets, improving token efficiency and inference speed.

  • Takeaways & Limitations

    The study focuses on VLAs fine-tuned on small-scale driving data without reasoning supervision, and Dr. GRPO remains imperfect at mitigating difficulty bias.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models are advancing autonomous driving by replacing modular pipelines with unified end-to-end architectures. However, current VLAs face two expensive requirements: (1) massive dataset collection, and (2) dense reasoning annotations. In this work, we address both challenges with NORD (No Reasoning for Driving). Compared to existing VLAs, NORD achieves competitive performance while being fine-tuned on <60% of the data and no reasoning annotations, resulting in 3x fewer tokens. We identify that standard Group Relative Policy Optimization (GRPO) fails to yield significant improvements when applied to policies trained on such small, reasoning-free datasets. We show that this limitation stems from difficulty bias, which disproportionately penalizes reward signals from scenarios that produce high-variance rollouts within GRPO. NORD overcomes this by incorporating Dr. GRPO, a recent algorithm designed to mitigate difficulty bias in LLMs. As a result, NORD achieves competitive performance on Waymo and NAVSIM with a fraction of the training data and no reasoning overhead, enabling more efficient autonomous systems. Website: https://nord-vla-ai.github.io/

1. Introduction

Existing VLA training commonly relies on large datasets with reasoning annotations and subsequent RL, creating substantial data, annotation, token, and deployment costs. NORD investigates whether competitive driving performance can instead be achieved with limited, reasoning-free data by addressing GRPO’s difficulty bias with Dr. GRPO.

  • Motivation: Large-scale reasoning data, annotation, training, and inference requirements make prevailing VLA pipelines costly for autonomous driving.These costs motivate training VLAs with less data and without reasoning supervision.
  • Motivation: NORD-BASE uses only 80,000 NAVSIM samples for reasoning-free SFT, reducing data by more than 60% versus reasoning-based models.It is subsequently post-trained with GRPO on the PDM score.
  • Initial finding: >12-point lower performance and only a +0.67% GRPO improvement initially made reasoning data appear necessary for strong results.The result came from a weak, data-efficient, reasoning-free model compared with reasoning-based baselines.
  • Analysis: GRPO struggles because polarized reward distributions from weak SFT policies disproportionately penalize intermediate-mean, high-variance scenarios.This interaction between GRPO and complex driving rewards is identified as difficulty bias.
  • NORD: NORD replaces GRPO with Dr. GRPO and achieves competitive NAVSIM and WaymoE2E performance without reasoning annotations and with at least 60% less data.The authors also report improved inference time and position the method as a data-efficient, reasoning-free VLA.

2. Related Works

Related work spans reasoning-centric, reasoning-free, and data-efficient VLAs, alongside methods for mitigating GRPO difficulty bias. NORD targets competitive performance using small, specialized in-domain driving data rather than external data or extensive reasoning supervision.

  • Reasoning-based VLAs: Reasoning-based VLAs use architectures and strategies such as hybrid control loops, unified transformers, retrieval-augmented CoT, and multi-agent reasoning.These approaches have achieved state-of-the-art performance on complex driving benchmarks but rely on large-scale reasoning data.
  • Reasoning-free VLAs: Reasoning-free VLAs map sensor inputs directly to trajectories or use generative approaches, but their competitiveness on complex long-tail benchmarks remains unresolved.Examples include EMMA, SimLingo, S4-Driver, ADriver-I, DrivingGPT, and DiffVLA.
  • Mitigating difficulty bias: Table 1 compares GRPO and Dr. GRPO for RL fine-tuning NORD-BASE on the NAVSIM test set, reporting failure for GRPO and significant gains for Dr. GRPO.The comparison concerns reinforcement-learning fine-tuning rather than the broader taxonomy of VLA architectures.
  • Data-efficient VLAs: NORD focuses on training a competitive model with small specialized in-domain driving data, unlike approaches that mitigate scarcity with massive external datasets.This defines a distinct data-efficiency setting for autonomous driving.
  • Mitigating difficulty bias: Difficulty-bias mitigation methods include filtering, curriculum learning, and advanced sampling, but these approaches are generally designed for binary rewards and can be impractical for expensive driving simulations.NORD’s related-work framing motivates optimization-level mitigation for driving settings.

3. Limitations of GRPO for Data-Efficient Training

GRPO provides limited benefit when post-training a weak, reasoning-free SFT policy because it struggles with high-variance reward groups that dominate the data. Dr. GRPO addresses this difficulty bias and enables optimization of harder driving scenarios.

  • 80,000 reasoning-free SFT samples produced only a 0.67% improvement after GRPO post-training.This was substantially below the 9% boost reported for AutoVLA using more than 212,000 samples and reasoning data.
  • Dr. GRPO effectively optimizes high-variance samples and enables complex maneuvers without collisions.The qualitative comparison contrasts successful sharp turns and lane changes with GRPO rollouts that collide.
  • High variance arises in intermediate-reward scenarios where weak SFT predictions inconsistently handle complex maneuvers such as sharp turns.Simple behaviors and extremely difficult scenarios tend to produce low-variance rollouts, whereas complex maneuvers often fail intermittently.
  • GRPO optimizes mainly low-variance samples while failing to improve the majority of high-variance samples.The high-variance region [0.2, 0.65] remains largely unchanged during training, while low-variance density increases.
  • NORD directly predicts action tokens without reasoning traces, reducing training and inference overhead.This design targets the data, annotation, training, and inference costs associated with reasoning-centric VLA pipelines.

4. NORD: No Reasoning for Driving

NORD is a reasoning-free VLA that shifts learning toward RL post-training from a deliberately weak, data-limited SFT policy. Dr. GRPO removes the variance-related optimization barrier, producing substantially larger gains than standard GRPO.

  • NORD omits reasoning annotations during training and inference and emphasizes RL post-training for token and data efficiency.NORD-BASE is intentionally trained on limited SFT data so subsequent RL carries more of the learning burden.
  • NORD predicts future ego-trajectories from camera images, past trajectory, speed, and acceleration using trajectory tokens.The trajectory is predicted at 10 Hz, with k-disc tokenization using a vocabulary of 2048.
  • Dr. GRPO replaces standard GRPO for optimizing the weak SFT policy NORD-BASE.The method is introduced specifically to address difficulty bias in weak-policy RL post-training.
  • Removing the reward standard-deviation term prevents high-variance groups from receiving an attenuated learning signal.Standard GRPO disproportionately favors low-variance groups, whereas Dr. GRPO enables hard scenarios to contribute sufficient gradient signal.
  • 11.68% improvement from the base model was achieved with Dr. GRPO, compared with 0.67% with GRPO.Dr. GRPO learns from mid-variance samples and enables optimization of samples with high intra-group variance.

5. Experiments

NORD is evaluated on NAVSIM and WaymoE2E as a reasoning-free, data-efficient VLA. It achieves competitive driving performance while using limited data, RGB inputs, and improved token and runtime efficiency.

  • WaymoE2E performance: NORD achieves a Rated Feedback Score of 7.709 on WaymoE2E, ranking third among VLAs without reasoning traces or ensembling.It uses 12,000 supervised-training samples and 450 RLFT samples.
  • Efficiency and scalability: NORD occupies the high-performance, high-data-efficiency region on NAVSIM and achieves competitive RFS on WaymoE2E with a fraction of the training data.The Pareto analysis uses RGB inputs and reports no reasoning supervision or ensembling for WaymoE2E.
  • WaymoE2E performance: NORD surpasses competitive models on ADE despite using at least 6× less training data.This result supports generalization under WaymoE2E’s challenging out-of-distribution evaluation setting.
  • NAVSIM performance: NORD achieves competitive NAVSIM performance using fewer than 90,000 samples, three camera frames, and no LiDAR or HD map features.NORD-BoN reaches a PDM score of 92.4 and surpasses reasoning-based AutoVLA-BoN.
  • Efficiency and scalability: NORD establishes a competitive performance baseline while methods with marginally higher scores use at least 3× more data.The comparison places NORD on the efficiency frontier across both benchmarks.
  • Efficiency and scalability: NORD is the most token- and runtime-efficient VLA in the reported comparison.Its direct trajectory-token prediction avoids explicit reasoning during inference.

6. Conclusion

NORD is a reasoning-free, data-efficient VLA that achieves strong autonomous-driving performance with fewer training samples and improved token and inference efficiency. Dr. GRPO mitigates difficulty bias better than GRPO, but remains imperfect.

  • NORD achieves strong autonomous-driving performance while eliminating language reasoning and substantially reducing training-data requirements.The conclusion presents this as evidence that efficient VLAs can be trained without reasoning and large-scale datasets.
  • NORD improves token efficiency and inference speed through reward analysis and a modified training pipeline.The conclusion attributes these improvements to the overall training approach rather than to reasoning generation.
  • Dr. GRPO mitigates difficulty bias better than GRPO but remains imperfect, leaving room for future work.

7. Comparison between GRPO and Dr. GRPO

The comparison shows Dr. GRPO generally improves RL fine-tuning over GRPO for NORD-BASE. The analysis connects this advantage to reward variance across driving scenarios and evaluates the associated reward components.

  • Training comparison: Dr. GRPO significantly outperforms GRPO on nearly every reported component except Ego Progress.Both methods improve over time, but GRPO consistently lags behind Dr. GRPO on training and validation curves.
  • Vocabulary ablation: A smaller 512-token trajectory vocabulary degrades NAVSIM performance compared with NORD’s 2048-token vocabulary.The reported explanation is that the smaller vocabulary represents complex maneuvers such as sharp turns less faithfully.
  • Reward functions: The total reward combines format, length, and dataset-specific rewards.Dataset-specific rewards use PDM for NAVSIM and normalized RFS for WaymoE2E.
  • Reward functions: Format reward is 0.25 for valid trajectory-token formatting and 0 otherwise, while length reward is 0.25 only for the correct token count.NAVSIM requires 8 trajectory tokens and WaymoE2E requires 10.
  • Reward functions: NAVSIM’s PDM score measures driving quality and safety using collision, drivable-area, progress, comfort, and time-to-collision factors.The score range is [0, 1].
  • Variance analysis: Dr. GRPO outperforms GRPO most clearly on medium- and high-variance samples, while GRPO improves strongly on low-variance samples.The y = x line denotes no change in PDM score; denser points above it indicate greater improvement.

10. Dataset Details

The experiments use separate NAVSIM and WaymoE2E data pipelines for supervised and reinforcement-learning fine-tuning. Dataset construction includes fixed splits, filtering, and benchmark-specific evaluation resources.

  • WaymoE2E data: WaymoE2E supervised fine-tuning uses 20% of valid frame sequences, split 85/15 into training and validation sets.Frames are filtered to ensure four preceding time steps are available.
  • RL fine-tuning: Dr.GRPO consistently outperforms GRPO on the reported training and validation curves.The comparison is shown for RL fine-tuning on the constructed data pipeline.
  • Evaluation: WaymoE2E test results report detailed benchmark performance for NORD.
  • WaymoE2E data: WaymoE2E RL fine-tuning extracts one sample per scenario from the validation set and splits samples 85/15.The validation set provides preference annotations for a single frame per scenario.
  • NAVSIM data: NAVSIM supervised fine-tuning uses an 80/20 training-validation split of the official training set.Input images are resized to satisfy the Qwen vision encoder’s pixel constraints.
  • NAVSIM data: The NAVSIM RLFT dataset filters out trivial constant-velocity behaviors and retains non-trivial, dynamically diverse trajectories.Samples with final-point displacement error below 0.2 m are discarded.

11. Implementation Details

NORD is fine-tuned on driving inputs to predict discretized trajectories, then optimized with Dr. GRPO using task-specific rewards and grouped rollouts.

  • Supervised Fine-Tuning: NORD uses Qwen2.5-VL-3B-Instruct to predict discretized trajectory tokens from multi-view images, past trajectories, and current vehicle kinematics.NAVSIM additionally provides a high-level driving command and three past trajectory tokens covering 1.5 seconds.
  • Reinforcement Learning: Dr. GRPO fine-tunes NORD with task-specific rewards by generating 8 rollouts per input to estimate group-relative advantages.The implementation uses asymmetric clipping with a high clip of 0.1 and a low clip of -0.2 to stabilize policy updates.
  • Reinforcement Learning: RL fine-tuning uses batch sizes of 128 trajectories for NAVSIM and 256 for WaymoE2E.Training uses 30 A100 GPUs for NAVSIM and 32 A100 GPUs for WaymoE2E, with mixed precision and gradient checkpointing.

12. Dataset Scale Estimation

The paper standardizes training-scale comparisons by aggregating reported dataset percentages and official splits, while also documenting representative NORD failure cases and their connection to difficulty bias.

  • Dataset Scale Estimation: The performance-efficiency frontier estimates total training samples for evaluated models across NAVSIM and WaymoE2E.Baseline methods often use multi-dataset mixtures or varying fractions of available data, requiring standardization.
  • Dataset Scale Estimation: Reported dataset percentages and official training splits are explicitly aggregated to standardize sample counts across methods.For WaymoE2E, HMVLM and DiffusionLTF are estimated at approximately 500k and 730k samples, respectively.
  • Failure Cases: NORD remains susceptible to failures in certain scenarios, with representative examples presented in Figure 13.The paper attributes these failures in part to Dr. GRPO remaining susceptible to difficulty bias.
  • Failure Cases: Targeted interventions that better account for task difficulty are proposed as a direction for further improving the performance frontier.This follows the paper’s observation that difficulty bias still affects policy optimization dynamics.
Loading 2602.21172v3…