Source-linked AI summary
Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Driving
Jiangxin Sun, Feng Xue, Teng Long, Chang Liu, Jian-Fang Hu, Wei-Shi Zheng, Nicu Sebe
TL;DR
Imitation-based E2E-AD struggles to make safe decisions in rare or unseen scenarios without expert experience, motivating reliable control without expert action supervision. RaWMPC uses risk-aware world-model prediction, interactive exposure to hazardous behaviors, and self-evaluation distillation to select safer actions, and reports state-of-the-art performance across Bench2Drive and NAVSIM, including previously unseen scenarios.
Problem
Existing imitation-learning and model-based RL methods struggle to guarantee safety in rare, high-risk scenarios because they lack explicit modeling and sampling of catastrophic outcomes.
Method
RaWMPC predicts consequences for multiple candidate actions with a risk-aware world model, learns from hazardous environment interactions, and distills risk avoidance into an action proposal network.
Results
RaWMPC achieves state-of-the-art performance on Bench2Drive and NAVSIM without expert demonstrations, with substantially higher driving performance in previously unseen scenarios.
Takeaways & Limitations
Risk-aware interaction and explicit candidate-risk evaluation provide an interpretable and reliable alternative to replicating expert driving behavior.
Takeaways & Limitations
Using too much offline warm-up data can reduce online exploration of unconventional actions and safety-critical failures, while training from scratch produces less reliable early rollouts.
Abstract
from arXiv · showhide
With advances in imitation learning (IL) and large-scale driving datasets, end-to-end autonomous driving (E2E-AD) has made great progress recently. Currently, IL-based methods have become a mainstream paradigm: models rely on standard driving behaviors given by experts, and learn to minimize the discrepancy between their actions and expert actions. However, this objective of "only driving like the expert" suffers from limited generalization: when encountering rare or unseen long-tail scenarios outside the distribution of expert demonstrations, models tend to produce unsafe decisions in the absence of prior experience. This raises a fundamental question: Can an E2E-AD system make reliable decisions without any expert action supervision? Motivated by this, we propose a unified framework named Risk-aware World Model Predictive Control (RaWMPC) to address this generalization dilemma through robust control, without reliance on expert demonstrations. Practically, RaWMPC leverages a world model to predict the consequences of multiple candidate actions and selects low-risk actions through explicit risk evaluation. To endow the world model with the ability to predict the outcomes of risky driving behaviors, we design a risk-aware interaction strategy that systematically exposes the world model to hazardous behaviors, making catastrophic outcomes predictable and thus avoidable. Furthermore, to generate low-risk candidate actions at test time, we introduce a self-evaluation distillation method to distill riskavoidance capabilities from the well-trained world model into a generative action proposal network without any expert demonstration. Extensive experiments show that RaWMPC outperforms state-of-the-art methods in both in-distribution and out-of-distribution scenarios, while providing superior decision interpretability.
1 Introduction
E2E-AD methods commonly imitate expert behavior but struggle with rare, high-risk scenarios that are insufficiently modeled. RaWMPC removes expert action supervision and combines risk-aware world modeling with predictive control and self-evaluation distillation.
- Motivation: Imitation-learning methods replicate privileged experts’ behavior, while model-based RL methods plan with world models but still generally maximize expected reward.These paradigms rely on expert behavior or expected-return optimization rather than explicit modeling of rare hazards.
- Motivation: Rare, high-risk scenarios remain difficult because existing methods often lack explicit modeling and sampling of catastrophic outcomes.This limitation challenges safety guarantees in long-tail driving situations.
- RaWMPC: RaWMPC predicts near-future consequences for multiple candidate driving behaviors and selects the candidate with the lowest explicitly evaluated risk.Unlike model-based RL, its world model is used for risk-aware candidate selection rather than only reward maximization.
- RaWMPC: Risk-aware interaction learning exposes the world model to self-identified high-risk actions so it can predict consequences of diverse hazardous behaviors without expert demonstrations.The approach starts from scratch and can be accelerated with a small number of benchmark video clips.
- RaWMPC: Self-evaluation distillation transfers the trained world model’s safe-versus-risky behavior judgments into a generative action proposal network for test-time candidate generation.The distillation uses safety–risk contrastive learning and does not require expert demonstrations.
- Results: Experiments on Bench2Drive and NAVSIM report state-of-the-art performance without expert demonstrations, with stronger performance in previously unseen scenarios and improved decision interpretability.An optional light warm-up further accelerates convergence and improves performance.
2 Related Work
Prior autonomous-driving research spans reinforcement learning, imitation learning, and world-model-based prediction and planning. However, existing approaches still face difficulty discovering and modeling rare catastrophic outcomes needed for reliable long-tail decisions.
- Learning-based driving: Reinforcement learning refines driving strategies through interactive feedback, while later methods imitate RL or rule-based experts using sensor inputs.Model-based RL additionally learns explicit dynamics or world models for look-ahead rollouts.
- Learning-based driving: RL-based approaches commonly maximize expected return but rarely systematically discover and model rare catastrophic outcomes.This leaves reliable decision-making in long-tail, high-risk scenarios challenging.
- Imitation learning: Imitation learning achieves strong closed-loop performance by distilling trajectories and features from privileged RL-based or rule-based experts into sensor-based agents.The approach relies on expert demonstrations to replicate expert behavior.
- World models: World models approximate environment transitions and predict future states or rewards from current observations and actions under a Markov Decision Process.In autonomous driving, they have been used to generate controllable future trajectories conditioned on actions and scene descriptions.
- World models: World-model-based trajectory generation can enlarge training data and increase diversity, potentially benefiting imitation learning in rare scenes such as traffic accidents.Other work connects world modeling with planning, online evaluation, or closed-loop driving assessment.
3 Method
RaWMPC’s method is organized around risk-aware interactive training and self-evaluation distillation. These components support efficient optimization and test-time generation of action candidates.
- Risk-aware interactive training: Risk-aware interactive training provides the scheme for efficiently optimizing the RaWMPC model.The training process is presented as part of the overall network structure and pipeline.
- Self-evaluation distillation: Self-evaluation distillation trains an action proposal network so the end-to-end system can run efficiently during testing.The network supplies action candidates for the test-time pipeline.
3.1 Network Structure of RaWMPC
RaWMPC encodes visual observations, ego state, and candidate action sequences, then uses a world model and task-specific decoders to predict and evaluate future outcomes. Predictive control selects the candidate with the lowest cost, balancing progress and traffic-safety risks.
- World Model: RaWMPC conditions its world model on observed states and candidate action sequences to autoregressively predict near-future states over horizon H.The inputs include multiview RGB images, ego-centric measurements, and candidate controls.
- Semantic-Guided Decoding: Three transformer decoders use predicted states to estimate semantic segmentation, traffic events, and future ego states.Event decoding incorporates semantic attention from segmentation to focus on regions relevant to events such as collisions.
- Action Selection and Predictive Control: For each candidate, predictive control evaluates progress toward the target and risks of traffic-violation events across the planning horizon.The selected action sequence minimizes the predicted horizon cost, favoring faster progress while reducing predicted violations.
- Action Selection and Predictive Control: The cost function down-weights distant predictions with ηk = max(2^-k+1, 1/8) and assigns larger λj weights to more severe violations.The floor on ηk prevents distant predictions from vanishing, while violation weights reflect event severity.
- Overall Loss: Training combines world-model, segmentation, ego-state, and event losses, using simulator annotations from executed rollouts.The overall objective is L = Lworld + Lseg + Lego + Levent, with segmentation using classification, focal, and dice losses.
3.2 Risk-aware Interactive Training
RaWMPC trains risk awareness through staged interaction that exposes the world model to both safe and hazardous trajectories. Segment-wise, soft cost-based sampling preserves diversity while emphasizing informative low- and high-risk behaviors.
- Training Strategy: Risk-aware interactive training first warm-starts the world model from logged trajectories, then refines it through online simulator interaction.The optional warm-up initializes state forecasting rather than imitating expert actions.
- Modes for Interaction: The three interaction modes sample uniformly from all candidates, from high-cost candidates, or from low-cost candidates.Their probabilities are ε1, (1 − ε1)ε2, and (1 − ε1)(1 − ε2), respectively.
- Risk-Aware Exploration: The interaction scheme intentionally collects safe, goal-directed and hazardous rollouts because logged data provide limited coverage of unconventional risky actions.This improves coverage of out-of-distribution controls and rare safety-critical events.
- Risk-Aware Exploration: The resulting feedback loop progressively improves prediction fidelity and risk sensitivity for risk-aware decision making.The same world model is refined using the collected interactive rollouts.
- Segment-Wise Interaction: Segment-wise sampling executes horizon-H action sequences, allowing sustained safe or risky behaviors and their longer-term consequences to unfold.Candidates are rolled out, scored, and ranked with the predictive-control cost.
- Soft Candidates Selection: Soft sampling within good and bad candidate sets preserves diversity while biasing exploration toward low-cost or high-cost regions.Filtering removes degenerate high-cost rollouts, and temperature parameters control the trade-off between concentration and diversity.
3.3 Self-Evaluation Distillation for Policy Learning
Self-evaluation distillation uses RaWMPC to rank sampled action sequences and train a generative proposal network to produce low-risk candidates without expert supervision.
- Pseudo-label construction: RaWMPC scores sampled action sequences, treating the lowest-cost sequence as positive and the top-K highest-cost sequences as negatives.This ranking transfers low-risk action knowledge to the proposal network without external supervision.
- Action proposal network: A conditional VAE uses an action encoder, conditional prior, and decoder, with the decoder serving as the proposal policy at inference.The model reconstructs positive actions while avoiding reconstruction of negatives to prevent unsafe imitation.
- Contrastive training objective: The InfoNCE design anchors on the positive posterior, pulling the conditional prior toward high-quality actions while indirectly separating it from negatives.The alternative prior-anchored formulation often produced under-optimized trajectories because numerous negatives could drive the prior away from positives.
- Overall objective: The total cVAE objective combines positive-action reconstruction, KL regularization, and contrastive loss.This objective trains a fast proposal policy consistent with RaWMPC’s evaluations, eliminating expert demonstrations during policy learning.
4 Experiments
The experiments evaluate RaWMPC on complementary interactive and real-world planning benchmarks, using official metrics and ablations to assess predictive control and generalization.
- Benchmarks: Bench2Drive provides fully interactive closed-loop CARLA evaluation, while NAVSIM evaluates large-scale real-world planning through non-reactive short-horizon rollouts.The benchmarks therefore assess complementary aspects of autonomous driving performance.
- Bench2Drive: Bench2Drive reports Driving Score, Success Rate, Efficiency, and Comfortness, with Driving Score as the primary aggregate metric.Driving Score includes penalties for safety and rule violations; the other metrics capture completion, progress, and smoothness.
- NAVSIM: NAVSIM reports PDMS as its primary score, combining NC, DAC, EP, TTC, and C to capture safety, compliance, progress, and motion quality.Results use the official toolkits and recommended splits.
- Evaluation protocol: The experimental setup combines offline warm-up with online simulator interaction, followed by ablations of key training components and strategies.Bench2Drive is used for interactive closed-loop evaluation, while NAVSIM measures large-scale real-world generalization.
4.2 Implementation Details
RaWMPC is trained in two stages, and its self-evaluation proposal network uses ranked action samples and world-model costs to generate candidates efficiently at inference.
- RaWMPC training: RaWMPC first performs offline warm-up, then refines predictive dynamics and risk-aware decision making through online interaction.Warm-up uses 10% of the training data before risk-aware interactive training.
- Risk-aware interaction: Risk-sampling probability ε2 increases from 0 to 0.3 while random-sampling probability ε1 decreases from 1 to 0.A 10K-frame replay buffer stores recent interaction data during training.
- Training configuration: Across warm-up and online interaction, training uses 1K Bench2Drive clips and 100K NAVSIM samples on four NVIDIA A100 GPUs.The optimizer uses Adam with learning rate 10^-4 decayed to 10^-5 and batch size 16.
- Self-evaluation distillation: The cVAE proposal network samples 50 action sequences, labels the minimum-cost sequence positive, and uses the five highest-cost sequences as negatives.At inference, it samples 10 candidates and selects the action with minimum predicted world-model cost.
4.3 Comparison with the State-of-the-Art
RaWMPC outperforms prior methods on Bench2Drive and NAVSIM, including without logged-trajectory warm-up, and remains robust under Sunny-only to Rainy domain shift. Its predictive-control procedure evaluates candidate consequences and selects low-risk actions in qualitative scenarios.
- 4.3.1 Evaluation on Bench2Drive: RaWMPC reaches 88.31 DS and 70.48% SR on Bench2Drive with warm-up, and 87.34 DS and 69.62% SR without logged trajectories.The no-warm-up setting surpasses HiP-AD and SimLingo while maintaining competitive efficiency and comfortness.
- 4.3.2 Evaluation on NAVSIM: RaWMPC achieves the highest learning-based NAVSIM score, with PDMS increasing from 90.5 without warm-up to 91.3 with warm-up.The no-warm-up result already exceeds GoalFlow’s 90.3 PDMS.
- 4.3.3 Generalization under weather-induced domain shift: Under Rainy evaluation, RaWMPC has the best DS and SR under both Sunny-only and Sunny-and-Rainy training regimes.Its performance degrades less than SimLingo when rainy training data is removed.
- 4.3.3 Generalization under weather-induced domain shift: In the Sunny-only to Rainy qualitative example, RaWMPC avoids collision while LAW, WoTE, and SimLingo collide after perception or evasive-decision failures.RaWMPC selects the minimum-risk predictive-control action under uncertainty.
- 4.3.4 Qualitative visualization of predictive control: RaWMPC predicts near-future consequences for candidate action sequences and selects the minimum-cost action using task-progress and safety outcomes.The visualizations include collision risk and other costs; examples show stopping for a pedestrian and timing a collision-free merge.
4.4 Ablation Study
Ablations show that semantic event prediction, predictive action selection, risk-aware interaction, self-evaluation distillation, and an appropriate planning horizon are central to RaWMPC’s performance. A light warm-up helps, but excessive offline data reduces opportunities to explore hazardous behaviors.
- 4.4.1 Analysis of Framework: Removing semantic guidance reduces Bench2Drive performance from DS 88.31 to 82.36 and SR 70.48% to 62.69%.The ablation links semantic-guided event decoding to risk-aware cost evaluation.
- 4.4.1 Analysis of Framework: Removing the segmentation decoder yields DS 70.85 and SR 48.95%, while removing action selection causes DS 61.35 and SR 30.98%.The latter bypasses cost-based ranking and directly executes proposal or guidance output.
- 4.4.2 Risk-aware interaction training: Risk-aware sampling deliberately collects both low-cost and high-cost rollouts, whereas unguided epsilon-greedy sampling is substantially less efficient for learning long-horizon consequences.The supplied passages state the efficiency conclusion but omit the full replacement result.
- 4.4.3 Action supervision: Self-evaluated positive and negative action targets achieve DS 88.31 and SR 70.48%, outperforming expert-only supervision at DS 86.75 and SR 68.25%.Using only positive actions further drops performance to DS 83.65 and SR 66.52%.
- 4.4.4 Discussion on Prediction Horizon: A planning horizon of H=10 performs best at DS 88.31 and SR 70.48%, while shorter horizons miss delayed consequences and H=15 suffers accumulated rollout errors.Reported short-horizon results include H=1 at DS 57.85 / SR 28.64% and H=5 at DS 74.98 / SR 49.52%.
- 4.4.5 Discussion on Warm-up: Using 10% warm-up data gives the best DS 88.31 and SR 70.48%, while larger fractions degrade performance as offline trajectories contain few hazardous events.The no-warm-up setting reaches DS 87.34 / SR 69.62%; 20% and 30% produce lower results.
- 4.4.6 Discussion on Control Pattern: Predictive control achieves DS 88.31 and SR 70.48%, compared with DS 73.58 and SR 51.85% for model-based RL.The comparison evaluates candidate sequences through decoded future outcomes before selecting the minimum-cost action.
- 4.4.7 Event prediction quality: The event predictor reaches 0.91–0.96 accuracy and 0.99 pedestrian-collision recall, but pedestrian-collision precision is 0.52.The authors describe this as a conservative tendency that can prioritize recall in safety-critical driving.
5 Conclusion
RaWMPC combines risk-aware world-model rollouts, interactive collection of safe and hazardous behaviors, and self-evaluation distillation to support expert-free autonomous driving. Experiments report state-of-the-art performance and stronger domain-shift robustness, while future work targets domain adaptation and efficient planning.
- 5 Conclusion: RaWMPC predicts future semantics and safety events for multiple candidate behaviors, then selects actions by minimizing a risk-aware cost.Its interaction strategy collects safe and hazardous rollouts, while self-evaluation distillation trains an action proposal policy.
- 5 Conclusion: Experiments on Bench2Drive and NAVSIM report state-of-the-art performance and stronger domain-shift robustness even without offline warm-up.The conclusion frames this as potential to reduce reliance on costly real-world expert demonstrations.
- 5 Conclusion: Future work will explore domain adaptation and more efficient planning for real-world deployment and sim-to-real transfer.
Statements and Declarations
The authors report no known competing financial interests or personal relationships influencing the work and provide no new dataset. The supporting Bench2Drive and NAVSIM datasets are openly available.
- Competing interests: The authors declare no known competing financial interests or personal relationships that could have influenced the reported work.
- Data availability: The work introduces no new dataset, and Bench2Drive and NAVSIM are openly available.