PaperlayerView exact revision on arXiv

Source-linked AI summary

DriveZero: End-to-End Driving Beyond Human Demonstrations

Hao He, Chengcheng Hu, Zirun Su, Heng Zhang, Haisong Liu, Jinke Li, Haochen Tian, Zhenwei Shen, Hongyang Li, Zhichao Li, Yunchen Yang, Bochao Huang, Siyu Zhang, Kuangye Chen, Xiongjie Zhang, Wentao Dai, Hengchen Dai, Siyuan Liu, Zehao Huang, Naiyan Wang

arXiv:2609.06055v1cs.CV

TL;DR

End-to-end driving systems commonly imitate human logs, whose limited behavioral coverage constrains learned behavior. DriveZero separates perception and action, using DriveVFM for label-free visual pretraining and closed-loop DriveRL teacher rollouts for camera-only planner distillation. DriveRL reaches a 93.57 mean nuPlan score beyond the Log-Replay expert, while DriveZero achieves state-of-the-art results across NAVSIMv1, NAVSIMv2, and HUGSIM without human trajectory supervision.

  • Problem

    Human-log imitation provides limited behavioral coverage because each scene records one realized future, rarely includes safety-critical recovery, and omits states induced by the learned policy.

  • Method

    DriveZero separates perception and action, distills frozen vision foundation models into DriveVFM, trains a privileged DriveRL teacher with closed-loop reinforcement learning, and distills its rollouts into a camera-only planner.

  • Results

    DriveRL reaches a 93.57 mean score across three nuPlan community splits beyond the Log-Replay expert, while DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2, and HUGSIM without human trajectory supervision.

  • Takeaways & Limitations

    Closed-loop reinforcement learning can learn driving behavior beyond the logs that seed its training worlds, and teacher distillation transfers that behavior into a camera-only planner.

Abstract

from arXiv · show

Most end-to-end autonomous-driving systems learn by imitating human driving logs, leaving their learned behavior constrained by the quality and behavioral coverage of the recorded trajectories. This report presents DriveZero, an end-to-end system that learns driving behavior beyond human demonstrations. It decomposes driving into a perception model and an action model, pretrains each in the regime best suited to it, and combines them into one end-to-end planner. The two models call for different learning recipes: perception must understand the world, and benefits from massive and diverse visual data; action must interact with it, and requires closed-loop feedback. On the action side, we introduce DriveRL, a mixed-agent closed-loop reinforcement-learning framework. It converts real driving logs into interactive worlds, where a privileged teacher policy is trained with PPO through closed-loop rollouts. For the perception model, DriveVFM consolidates multiple frozen vision foundation models, including DINOv3, SigLIP2, SAM and Depth Anything V2, into a single backbone from raw images alone, requiring no task-specific annotations. DriveZero then unifies the two: a camera-only planner that distills the frozen DriveRL teacher through its rolled-out trajectories. The goal-conditioned teacher can moreover be queried under augmented driving intents, yielding diverse, goal-consistent supervision that logged data cannot provide. On nuPlan, DriveRL with value-guided test-time action search achieves a mean score of 93.57 across the Val14, Test14-hard, and Test14-random community splits in both non-reactive and reactive modes, exceeding the Log-Replay expert on all three splits. DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2 and the closed-loop HUGSIM benchmark without any human trajectory supervision.

1 Introduction

DriveZero addresses the limits of human-log imitation by separating perception from action, pretraining each with a suitable recipe, and reunifying them in a camera-only planner. Closed-loop DriveRL learns beyond logged behavior, while DriveVFM supplies label-free visual representations for distillation.

  • Motivation: Human-log imitation is limited by recorded trajectory quality and coverage, including missing valid actions, rare recovery maneuvers, and absent policy-induced states.These gaps can cause errors to compound when the learned policy leaves the demonstration distribution.
  • Action model: DriveRL converts real nuPlan logs into mixed-agent interactive worlds and trains a privileged teacher with PPO through closed-loop rollouts.The framework lets log replay, rule-based behaviors, and learned policies coexist within a common physical state–action interface.
  • Perception model: DriveVFM distills DINOv3, SigLIP2, SAM, and Depth Anything V2 into one driving backbone from raw images without task-specific labels.Its training can mix web-scale imagery with driving scenes to acquire driving-relevant semantics, geometry, and spatial structure.
  • Unification and results: DriveZero unifies the models by distilling DriveRL rollouts into a camera-only planner initialized with DriveVFM representations instead of human trajectories.It reaches 94.8 PDMS on NAVSIMv1 navtest and, with simulation scaling, reports 95.3 PDMS on NAVSIMv1, 57.1 EPDMS on NAVSIMv2, and 46.6 HD-Score on HUGSIM.

2 Methodology

DriveZero separates driving behavior learning, visual representation pretraining, and camera-only policy transfer. DriveRL learns a privileged teacher through closed-loop PPO in mixed-agent worlds, while DriveZero distills its rolled-out behavior into an image-based planner with diverse goal-conditioned supervision.

  • DriveRL: DriveRL trains a privileged teacher from scratch with PPO in interactive worlds reconstructed from real nuPlan logs.The simulator supports heterogeneous background behaviors and large-scale parallel rollouts.
  • DriveRL: DriveRL represents navigation with two goal points whose deployment look-ahead distances scale with vehicle speed and are recomputed at every step.Training goals are constructed from future logged ego positions, while deployment goals are selected along the current route.
  • DriveRL: Value-guided test-time search reranks policy-supported alternative first actions using the teacher critic’s delayed-return estimates.The modal action remains a conservative candidate, and search changes decisions only when an alternative is predicted to be better.
  • DriveZero: DriveZero bridges structured-state teaching and camera-only deployment by rolling out the frozen teacher on logged frames and distilling its trajectories without interactive student training.The student uses paired images and structured states from driving logs, requiring no rendering or human demonstrations.
  • DriveZero: Winner-takes-all distillation assigns each teacher trajectory to the closest student proposal, allowing different trajectory queries to represent distinct driving modes.The closest proposal is selected using L1 distance averaged over trajectory points, while proposal selection is handled by a scoring branch.
  • DriveZero: Goal augmentation queries the goal-conditioned teacher with alternative route intents to generate diverse, goal-consistent trajectories beyond the single behavior recorded in each log.The augmented route intent is also used to evaluate student proposals, preserving consistency among command, teacher trajectory, and supervision.

3 Experiments

Experiments evaluate DriveRL in closed-loop nuPlan settings and DriveZero across NAVSIM and HUGSIM, including test-time scaling, simulation-data scaling, and ablations. The results show improvements over human-log supervision and prior baselines, with gains from goal augmentation and complementary visual teachers.

  • Evaluation setup: DriveRL is evaluated on nuPlan, while DriveZero is evaluated on NAVSIMv1, NAVSIMv2, and HUGSIM using their respective sensor protocols.DriveRL uses structured scene states in closed loop; DriveZero uses camera images.
  • DriveRL results: 93.01 is DriveRL’s mean score across six nuPlan evaluations, exceeding Log-Replay in every non-reactive and reactive setting.The six evaluations combine Val14, Test14-hard, and Test14-random in non-reactive and reactive modes.
  • DriveRL results: 93.57 is the mean score after value-guided test-time search, a 0.56-point increase over the fixed DriveRL checkpoint.The candidate budget reaches 64, with the largest gains on non-reactive Test14-hard and Test14-random.
  • DriveZero results: 95.3 PDMS on NAVSIMv1 navtest, 57.1 EPDMS on NAVSIMv2 navhard, and 46.6 HD-Score on HUGSIM are reported for DriveZero-Scale without human trajectory supervision.DriveZero-Scale uses additional SimScale simulation data and achieves state-of-the-art results on all three benchmarks.
  • DriveZero results: DriveZero reaches 94.8 PDMS on NAVSIMv1 navtest without human trajectory supervision, matching the human driver.The scaled variant improves this result to 95.3 PDMS.
  • Ablation studies: DriveVFM scores 94.41 PDMS versus 93.88 for DINOv3, while complementary teachers add 0.72 points over the DINOv3 + SigLIP2 baseline.The controlled comparison keeps the downstream planning setup fixed; the additional teachers are SAM and Depth Anything V2.
  • Ablation studies: Goal augmentation raises DriveZero to 94.41 PDMS, exceeding DriveRL-only supervision by 0.80 points and human-trajectory supervision by 0.49 points.DriveRL-only supervision scores 93.61, while human supervision scores 93.92.
  • Qualitative analysis: Qualitative comparisons show DriveZero-Scale maintaining safe, smooth, collision-free trajectories where human or prior-model trajectories hesitate, collide, or leave the drivable area.These examples include sudden braking and clear-road scenes in navtest and HUGSIM.

4 Related Work

Prior end-to-end driving work largely relies on human-log imitation, while recent research explores closed-loop reinforcement learning, world models, multi-agent training, and visual foundation-model pretraining to broaden behavioral and visual supervision.

  • Closed-Loop Reinforcement Learning for Driving: Closed-loop reinforcement learning trains policies from states induced by their own actions rather than only fixed expert trajectories.
  • Closed-Loop Reinforcement Learning for Driving: World-model approaches reduce interaction cost by learning and exploring within learned environments for end-to-end driving.
  • Closed-Loop Reinforcement Learning for Driving: Multi-agent training replaces fixed background traffic with learning agents, supporting self-play and driving without human data.
  • Visual Foundation Models for Driving: Driving perception backbones draw on general-purpose, driving-specific, self-supervised, and geometry-oriented pretraining objectives.
  • End-to-End Autonomous Driving: Most camera-based end-to-end policies remain behaviorally supervised by ego trajectories recorded in human driving logs.

5 Conclusion

DriveZero separates perception and action, pretrains them with foundation-model distillation and closed-loop reinforcement learning, then reunites them by distilling teacher rollouts into a camera-only planner. It surpasses human-log-derived behavior on nuPlan and achieves state-of-the-art results across three benchmarks without human trajectory supervision.

  • 5 Conclusion: DriveVFM learns visual representations from raw images by distilling frozen vision foundation models, while DriveRL learns behavior through closed-loop reinforcement learning in log-initialized interactive worlds.
  • 5 Conclusion: DriveZero reunites perception and action by distilling the frozen DriveRL teacher’s rollouts into a camera-only planner.
  • 5 Conclusion: DriveRL exceeds the Log-Replay expert on all three nuPlan community splits in both non-reactive and reactive modes.
  • 5 Conclusion: DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2, and closed-loop HUGSIM without human trajectory supervision.

6 Contributors

The supplied material combines contributor listings with detailed DriveRL implementation and reward-design specifications, including simulator interfaces, structured inputs, goal conditioning, action distributions, and PPO value decomposition.

  • Implementation: DriveRL uses a mixed-agent simulator in which background actors receive behavior providers through a shared physical state–action interface.
  • Implementation: The teacher filters structured observations to bounded numbers of actors and map tokens while retaining short actor histories and a defined local map window.
  • Goal Conditioning: Two unordered goal anchors condition the policy through a permutation-invariant interface shared between training and deployment.
  • Action Model: The action head parameterizes independent Beta distributions for normalized jerk and steering rate, with evaluation using each distribution’s analytic mode.
  • Reward and PPO: The reward combines hard events, goal arrival, and a multiplicative soft driving-quality term, while an eight-channel critic structures value supervision without changing the scalar policy objective.
  • Experimental Scope: Earlier 32-GPU ablations are intended only for within-table comparisons, so their absolute scores should not be compared directly with the reported teacher.

A.5.1 Detailed nuPlan Metric Breakdown

The detailed nuPlan analysis reports component metrics across six evaluations and compares Single-Ego training with Self-Play variants under the same mixed-agent simulator and PPO objective. Self-Play is feasible but does not consistently outperform the Single-Ego baseline.

  • Detailed Metrics: Table A5 reports DriveRL component metrics across all six nuPlan evaluation settings, with higher values better for every column.
  • Self-Play Configuration: Self-Play assigns the same policy to the ego vehicle and up to ten vehicle NPCs, alongside IDM-controlled vehicles in the candidate set.
  • Self-Play Comparison: The best reported Self-Play variant disables history dropout, dynamics noise, and front-vehicle braking.
  • Self-Play Comparison: Self-Play mean performance is comparable to the Single-Ego baseline, demonstrating feasibility but not a consistent performance advantage.

A.5.3 Traffic-Light Input and Reward

Traffic-light conditioning improves evaluation scores, while adding a red-light reward reduces violations at the cost of lower aggregate scores. This trade-off motivates reporting compliance separately from benchmark performance.

  • Traffic-light state improves scores in all six evaluation settings.
  • The red-light reward lowers policy-only violations from 3.88% to 2.03%.
  • Adding the traffic-light reward decreases all six aggregate scores.
  • Aggregate nuPlan scores omit traffic-light violations, so violation rates are reported alongside aggregate metrics.

A.5.4 Goal-Anchor Layout

Goal-anchor layout affects DriveRL performance: local near-goal guidance is important, while far-goal duplication is harmful. Training with mixed goal distributions achieves the strongest mean performance.

  • Duplicating the far anchor lowers all six scores, whereas the near-only layout matches default average performance.
  • The near anchor provides important local guidance, while the far anchor helps most when paired with it.
  • The mixed training distribution achieves the highest mean score.
  • Earlier-goal duplication remains close to default performance, while later-goal duplication performs substantially worse under matched evaluation.

B.1 Models and Training

The DriveZero-Scale configuration is summarized through its model and training hyperparameters.

  • Table A12 summarizes the model and training hyperparameters of the camera-only DriveZero-Scale.

B.2 Visual Backbone Scaling

Larger DriveVFM backbones improve DriveZero performance under otherwise fixed training conditions. The largest variant also slightly exceeds a stronger general-purpose DINOv3 ViT-L backbone.

  • All backbone variants use the same pretraining sources, mixture ratios, downstream split, frozen weights, and rank-32 Q/V LoRA adapters.
  • PDMS increases from 94.41 with ViT-S to 94.54 with ViT-B and 94.83 with ViT-L.
  • The ViT-S-to-ViT-L scaling produces a 0.42-point PDMS gain.
  • DriveZero ViT-L reaches 94.83 PDMS, edging out DINOv3 ViT-L at 94.55 PDMS.

B.3 QK-Clip for Stable Training and FP16 Deployment

QK-Clip stabilizes DriveVFM training by constraining query–key logits, and it improves robustness when deploying the backbone in FP16.

  • Mechanism: QK-Clip monitors query–key logits after optimizer updates and rescales query and key projection weights to constrain their scale.This design targets large pre-softmax logits that can cause loss spikes and numerical sensitivity.
  • Training stability: QK-Clip produces smoother descent and lower loss later in training than the unconstrained configuration.Figure A1 compares optimization with and without QK-Clip.
  • FP16 deployment: QK-Clip reduces FP32-to-FP16 mean absolute feature error to 0.004 and preserves cosine similarity of 0.999.These results indicate near-lossless FP16 deployment, while QK-Norm leaves larger residual error.
  • FP16 deployment: QK-Clip yields the lowest FP32-to-FP16 feature deviation among the compared configurations.Bounding logits improves robustness when converting the backbone from FP32 to FP16.
Loading 2609.06055v1…