DriveZero: End-to-End Driving Beyond Human Demonstrations

• Imitation from human logs is limited to their quality and behavioral coverage: logs show one realized future per scene, rarely include safety-critical deviations or recovery, and omit policy-induced states.Errors can compound when the learned policy leaves the demonstration distribution.Source: §1 · p. 1
• With simulation-scaled training, DriveZero sets state of the art at 95.3 PDMS on NAVSIMv1 navtest, 57.1 EPDMS on NAVSIMv2 navhard, and 46.6 HD-Score on HUGSIM.Without human trajectory supervision, DriveZero reaches 94.8 PDMS on NAVSIMv1 navtest, on par with the human driver at 94.8.Source: Figure 1 · p. 2, §1 · p. 3
• On nuPlan, DriveRL’s mean score across Val14, Test14-hard, and Test14-random rises from 93.01 to 93.57 with test-time search, exceeding the Log-Replay expert in all three splits.Search rolls out sampled actions, scores them using short-horizon rewards and the critic, and can replace the modal action with a higher-value candidate.Source: Figure 1 · p. 2, §1 · p. 3
• Closed-loop RL can learn from action consequences, perturbed states, and recovery, but direct visual RL is constrained by sample-intensive exploration and costly visual simulation.DriveZero therefore pretrains perception on diverse images and action through closed-loop interaction, then reunifies them by distillation rather than relying on an off-the-shelf visual encoder.Source: §1 · p. 2
• DriveRL trains a privileged teacher with PPO in mixed-agent interactive worlds reconstructed from nuPlan logs, using structured scene state and navigation to control the vehicle directly.The worlds let log replay, rule-based behaviors, and learned policies coexist; value-guided test-time search samples candidate actions to address out-of-domain cases.Source: §1 · p. 2
• DriveVFM distills frozen DINOv3 [71], SigLIP2 [78], SAM, and Depth Anything V2 [90] features into one driving backbone from raw images without task-specific labels.DriveZero initializes its encoder from DriveVFM and distills the frozen DriveRL teacher’s rolled-out trajectories; augmented goal-conditioned intents provide diverse supervision unavailable from logged data.Source: §1 · p. 3

2.1 DriveRL: Learning a Privileged Teacher from Scratch with Closed-Loop RL

• DriveRL trains a privileged teacher from random initialization with PPO in closed-loop worlds seeded by nuPlan logs, without imitation pretraining.The reward combines hard safety events, goal arrival, and soft driving-quality terms.Source: §2.1 · p. 3, §2.1.2 · p. 5

2.1.2 Mixed-Agent Simulation

• DriveRL’s mixed-agent simulator combines replayed, rule-based, and learned actors, running batched PPO rollouts across up to 196,608 worlds on 96 GPUs.By default only the ego is policy-controlled; self-play additionally controls selected background vehicles and is used for real-world deployment.Source: §2.1 · p. 3, §2.1.2 · pp. 4–5

2.1.1 The Structured Inputs and Policy Network

• The teacher consumes structured scene observations and two goal points, then predicts bounded jerk and steering-rate actions through a 5.7M-parameter policy network.The observations include ego and traffic histories, local map elements, and traffic-light states; the action head uses independent Beta distributions.Source: §2.1.1 · p. 4

2.1.3 Value-Guided Test-Time Action Search

• At test time, DriveRL reranks policy-supported first actions with short closed-loop rollouts and the teacher critic, choosing an alternative only when it beats the modal action by a margin.The candidate score reuses the PPO reward, discount factor, and critic, aligning ranking with the training objective.Source: §2.1.3 · pp. 5–6

2.2 DriveVFM: Distilling Multiple Vision Foundation Models into One Driving Backbone

• DriveVFM distills complementary frozen DINOv3, SigLIP2, SAM, and Depth Anything V2 representations into one backbone, using PHI Standardization to balance patch-feature targets.The models supply spatial structure, image-level semantics, boundary-sensitive features, and geometric cues; depth features are distilled rather than depth predictions.Source: §2.2 · pp. 6–7

2.3 DriveZero: Distilling the RL Teacher into a Camera-Only Planner

• DriveZero trains open-loop on logged frames, using frozen-teacher rollouts as trajectory targets and winner-takes-all distillation, while a scoring branch learns PDM quality components to rank proposals.The student uses camera images rather than privileged state, and no human demonstrations are used in its training.Source: §2.3 · pp. 7–8
• Goal augmentation queries DriveRL with alternative route intents at the same scene state, producing diverse goal-consistent teacher trajectories and matching proposal evaluations beyond the single logged behavior.The augmented route keeps the navigation command, teacher target, and proposal scores consistent.Source: §2.3 · p. 8

3.1.1 Closed-Loop Comparison

• 93.57 mean score with value-guided test-time search exceeds Log-Replay across the six nuPlan evaluations; it improves five settings, while the same checkpoint without search averages 93.01.Search raises the mean by 0.56 points and leaves Test14-hard R essentially unchanged (89.15 vs. 89.18).Source: §3.1.1 · p. 9, §3.1.2 · p. 9, Table 2 · p. 10

3.2.1 Main Results of the End-to-End Student

• 95.3 PDMS for DriveZero-Scale on NAVSIMv1 navtest exceeds DriveZero’s 94.8 and all prior camera-only and camera–LiDAR fusion methods; DriveZero exceeds the Human Driver’s 94.8.DriveRL reaches 95.8 PDMS from ground-truth symbolic inputs; DriveZero-Scale uses 237K OOD SimScale scenes alongside DriveRL-rollout supervision, without human trajectory supervision.Source: §3.2 · p. 10, §3.2.1 · pp. 11–12, Table 4 · p. 11
• 57.1 EPDMS for DriveZero-Scale on NAVSIMv2 navhard is 5.6 points above DriveZero; scaling raises Stage 2 by 8.3 points but lowers Stage 1 by 1.8.DriveZero scores 51.5, exceeds methods trained exclusively on navtrain, and outperforms GigaPixel by 1.4 points; the scaled model surpasses DrivoR-Scale and PDM-Closed.Source: §3.2.1 · p. 12, Table 5 · p. 12
• 46.6 average HD-Score gives DriveZero-Scale the strongest zero-shot HUGSIM performance, with state-of-the-art scores on Easy, Medium, and Hard, 8.1 points above GigaPixel.The model transfers from NAVSIMv1 without HUGSIM-specific fine-tuning; the reported gain is attributed to plausible, diverse DriveRL trajectories on OOD SimScale data.Source: §3.2.1 · pp. 12–13, Table 6 · p. 13

3.2.2 Ablation Studies

• DriveVFM scores 94.41 PDMS versus 93.88 for DINOv3 ViT-S; adding SAM and then Depth Anything V2 improves the DINOv3 + SigLIP2 baseline by 0.41 and 0.31 points.Together, the added teachers yield a 0.72-point gain. DriveRL trajectories alone score 93.61 versus 93.92 for human trajectories, but goal augmentation raises the score to 94.41, beating those baselines by 0.80 and 0.49 points, respectively; logged human futures cannot provide counterfactual supervision for changed goals.Source: §3.2.2 · pp. 13–14, Table 7 · p. 13

3.2.3 DriveVFM Feature Representation Visualization

• Across two scenes, DriveVFM more consistently represents road surfaces and more clearly separates long-tail obstacles and small objects from the ground than DINOv3.The visualization compares patch features using PCA and cosine similarity to a ground-region prototype.Source: §3.2.3 · p. 14, Figure 3 · p. 14

3.2.4 DriveZero Planning Results Visualization

• In the illustrated NAVSIM and HUGSIM scenes, DriveZero-Scale proceeds smoothly when the human driver hesitates, avoids navtest collisions or road departures, and responds safely to braking and oncoming traffic.Compared with DrivoR-Scale, it decelerates to maintain a safe gap and stays in lane with only a slight adjustment when an oncoming vehicle passes.Source: §3.2.4 · p. 15, Figure 4 · p. 15, Figure 5 · p. 16

3.3 Real-World Application

• DriveRL was demonstrated in closed-loop control on a crowded urban road with dense traffic, using a vehicle fleet and training on the team’s logs without human-demonstration supervision.Deployment inputs include autolabeled perception ground truth mixed with degraded recorded perception outputs, with extensive domain randomization; onboard detection, mapping, and navigation provide agents, map, and goal points.Source: §3.3 · p. 15

Closed-Loop Reinforcement Learning for Driving

• Prior closed-loop driving RL spans log-initialized policy gradients and imitation-RL [65] [53], scalable model-free rollouts [97] [29] [36], world-model exploration [80] [92], and multi-agent or self-play training [99] [12] [14] [13].SPICED [13] uses a small amount of human driving to regularize self-play toward human-compatible conventions.Source: §4 · pp. 16–17

Visual Foundation Models for Driving

• Visual pretraining spans general-purpose [4] [57] [63] [71] [78], driving-specific [88] [91], and geometry-oriented objectives [87]; DriveVFM consolidates frozen DINOv3, SigLIP2, SAM, and Depth Anything features [71] [78] [90].Following RADIO's consolidation recipe with PHI-S feature standardization [59] [60], DriveVFM needs no task-specific perception labels and can combine web images with driving scenes.Source: §4 · p. 17

End-to-End Autonomous Driving

• DriveZero independently pretrains goal-conditioned DriveRL through closed-loop interaction and DriveVFM on real images, then reunites them through trajectory distillation without rendered closed-loop student training.Source: §4 · p. 17
• Most camera-based end-to-end policies behavior-clone recorded ego trajectories; interactive-transfer examples include ROACH [100], TerraTransfer [86], GigaPixel [62], and Pictura [94].Other approaches train or post-train visual policies in photorealistic reconstructed worlds [22] [39] [55] [76].Source: §4 · p. 17

5 Conclusion

• DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2, and closed-loop HUGSIM without human trajectory supervision.Source: §5 · p. 17
• DriveRL exceeds the Log-Replay expert on all three nuPlan community splits in both non-reactive and reactive modes.The log-initialized interactive worlds seed training, but closed-loop reinforcement learning learns behavior beyond those logs.Source: §5 · p. 17
• DriveZero distills the frozen DriveRL teacher’s rollouts into a camera-only planner, unifying perception and action.Source: §5 · p. 17
• DriveVFM learns visual representations from raw images by distilling frozen vision foundation models.Source: §5 · p. 17
• DriveRL learns driving behavior from scratch through closed-loop reinforcement learning in log-initialized interactive worlds.Source: §5 · p. 17
• The proposed recipe separately pretrains perception and action, then unifies them through distillation as a scalable path beyond human demonstrations.Source: §5 · p. 17

A.1.1 Distributed PPO Configuration

• Each rank rolls out 2,048 worlds for 110 steps, producing 225,280 transitions per update split into 11 minibatches of 20,480 across four PPO epochs.Table A1 lists the teacher’s training and rollout configuration.Source: §A.1.1 · p. 24, Table A1 · p. 24

A.1.2 Mixed-Agent Environment and Traffic Providers

• Background actors share a physical state–action interface for log replay, IDM, front-vehicle braking, and policy-controlled self-play.To expose the ego to rare sudden lead-vehicle deceleration, vehicles within a 3 m lateral corridor ahead are selected at initialization with 3% probability; at each 5 Hz step, candidates brake with 5% probability, decelerating up to −10 m/s2 until stopped before reverting to their default provider.Source: §A.1.2 · p. 24

A.2 Structured Observation and Goal Conditioning

• Teacher observations retain up to 96 distance-ranked actor tokens and 256 map tokens, with five history frames and two unordered goal anchors.The actor filter reserves 32 slots each for vehicles and pedestrians, with ego in slot 0; training and deployment use different goal sources and update schedules but share the two-anchor interface.Source: §A.2 · pp. 24–25

A.3 Policy, Action Space, and Vehicle Dynamics

• The action head models normalized jerk and steering rate with conditionally independent Beta distributions, sampled during PPO and executed at their analytic modes in formal nuPlan evaluation.The teacher can export concentration parameters, mean, mode, and sampled actions for distillation; exporting both concentration parameters preserves more distributional information than exporting only a point action.Source: §A.3 · pp. 25–26

A.4 Reward, Value Decomposition, and PPO Objective

• An eight-channel critic structures hard, goal, and six soft reward targets while preserving the scalar policy objective.The soft reward multiplies six driving-quality scores, requiring joint satisfaction; the soft channels partition that single reward for critic supervision, and traffic-light violations are reported separately rather than included in the scalar reward.Source: §A.4 · pp. 26–27

A.5 Additional DriveRL Analyses

• Unless otherwise stated, additional analyses use the reported DriveRL checkpoint; earlier 32-GPU ablations support within-table comparisons only, not direct absolute-score comparison with the reported teacher.Source: §A.5 · p. 27

A.5.1 Detailed nuPlan Metric Breakdown

• Table A5 reports DriveRL component metrics across six nuPlan evaluation settings, with higher values better in every column.Source: §A.5.1 · p. 27, Table A5 · p. 27

A.5.2 Self-Play Study

• DriveRL-SelfPlay assigns the same policy to the ego vehicle and up to ten vehicle NPCs.Table A6 summarizes configuration differences between DriveRL-SelfPlay and DriveRL.Source: §A.5.2 · p. 27, Table A6 · p. 28
• The Self-Play candidate set first includes four IDM-controlled vehicles, then fills remaining slots with the most relevant policy-controlled vehicles.Source: §A.5.2 · p. 27
• DriveRL and DriveRL-SelfPlay use the same mixed-agent simulator and PPO objective, despite configuration differences.The configuration comparison is summarized in Table A6.Source: §A.5.2 · p. 27, Table A6 · p. 28
• Self-Play’s best variant has mean performance comparable to Single-Ego, demonstrating feasibility but no consistent performance advantage.All Self-Play variants use 4,800 updates, while Single-Ego uses its reported 2,400-update checkpoint; the best checkpoint is reported for each run and Mean is the unweighted average over six evaluations.Source: §A.5.2 · p. 28, Table A7 · p. 28
• Disabling history dropout, dynamics noise, and front-vehicle braking gives the highest mean score among the Self-Play variants.The comparison includes the Single-Ego baseline and Self-Play variants.Source: §A.5.2 · p. 28
• Table A7 compares Self-Play and Single-Ego across six closed-loop evaluations, with Mean defined as their unweighted average.The best checkpoint is reported for each run.Source: Table A7 · p. 28

A.5.3 Traffic-Light Input and Reward

• Adding traffic-light state improves scores in all six evaluation settings.The input encodes each map segment with a four-dimensional unknown/red/yellow/green feature; the red-light detector is diagnostic only in both settings.Source: §A.5.3 · p. 28, Table A8 · p. 28
• These results come from a separate controlled setup, and policy-only violations count scenes where the logged human remains compliant, following NAVSIM’s filtering principle [18].Absolute scores in this subsection are not directly comparable to those of the reported teacher.Source: §A.5.3 · p. 28
• The traffic-light reward lowers policy-only red-light violations from 3.88% to 2.03%, while all six aggregate scores decrease.The aggregate nuPlan score excludes traffic-light violations, so the violation rate is reported alongside aggregate metrics.Source: Table A9 · p. 29, §A.5.3 · p. 29

A.5.4 Goal-Anchor Layout

• DriveRL retains two goal-input slots; the single-anchor condition duplicates one goal into both, leaving the network architecture unchanged.The formal route-goal interface provides near and far anchors at every planner step.Source: §A.5.4 · p. 29
• The fixed-checkpoint sensitivity experiment varies only the nuPlan evaluation goal layout, comparing the near–far default with near-only and far-only layouts.The single-anchor alternatives duplicate the selected anchor into both slots.Source: §A.5.4 · p. 29
• The near–far default and near-only layouts have similar average performance, but far-only duplication lowers all six scores.The near anchor provides important local guidance; the far anchor is most useful when paired with, rather than substituted for, the near anchor.Source: Table A10 · p. 29, §A.5.4 · p. 29
• The training-distribution ablation uses a separate controlled setup, so its absolute scores are not directly comparable to those of the reported teacher.The default samples one or two future-log goals with equal probability; alternatives always sample two and retain either the earlier or later goal.Source: §A.5.4 · p. 29
• The mixed goal distribution achieves the highest mean score; earlier-goal duplication remains close to default, while later-goal duplication performs substantially worse.Evaluation layouts are matched to the corresponding training distributions.Source: Table A11 · p. 30, §A.5.4 · p. 30
• The training-distribution ablation trains models from scratch with 32-GPU Adam and reports the unweighted mean over six evaluations.Each model is evaluated using a goal layout matched to its training distribution.Source: §A.5.4 · p. 29, Table A11 · p. 30

B.1 Models and Training

• Table A12 summarizes the model and training hyperparameters of the camera-only DriveZero-Scale.Source: §B.1 · p. 30, Table A12 · p. 30

B.2 Visual Backbone Scaling

• The comparison holds data sources and mixture ratios fixed, uses NAVSIM navtrain without SimScale, and fine-tunes rank-32 Q/V LoRA adapters on frozen pretrained backbones.All other downstream settings are held fixed.Source: §B.2 · p. 30, Table A13 · p. 31
• PDMS increases from 94.41 with ViT-S to 94.54 with ViT-B and 94.83 with ViT-L, a 0.42-point ViT-S-to-ViT-L gain; ViT-L edges DINOv3 ViT-L (94.55).The benchmark is NAVSIMv1 navtest.Source: Table A13 · p. 31, §B.2 · p. 31

B.3 QK-Clip for Stable Training and FP16 Deployment

• After each optimizer update, QK-Clip monitors each head’s maximum pre-softmax logit and rescales query and key projection weights when it exceeds 1,000.Query and key receive the same square-root scaling factor; value weights and the forward attention formulation remain unchanged.Source: §B.3 · p. 31
• QK-Clip bounds effective query–key scale without adding a normalization operation, unlike Query–Key Normalization, which changes the attention computation.Large pre-softmax logits can cause loss spikes and make optimization sensitive to numerical precision.Source: §B.3 · p. 31
• QK-Clip produces smoother optimization descent and lower loss in the latter part of training.Figure A1 compares training with and without QK-Clip and shows it avoiding the loss spikes seen without clipping.Source: §B.3 · p. 31, Figure A1 · p. 31
• The FP32-to-FP16 evaluation compares backbone outputs under baseline, QK-Norm, and QK-Clip configurations using maximum and mean absolute deviations and cosine similarity.The reported maximum QK logit is the largest pre-softmax query–key dot product observed across all attention layers.Source: §B.3 · p. 31
• QK-Clip yields mean abs 0.004 and cosine 0.999 for FP32-to-FP16 outputs, versus QK-Norm’s mean abs 0.032 and cosine 0.981.QK-Norm lowers the maximum QK logit to 5,125, but leaves substantial residual error; QK-Clip is described as enabling near-lossless FP16 deployment.Source: Table A14 · p. 31, §B.3 · p. 31

B.4 More Visualizations of DriveZero

• Figure A2 provides additional DriveZero-Scale visualizations against a previous state-of-the-art model on NAVSIM navtest.Source: Figure A2 · p. 32