Source-linked AI summary
Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
Angelos Filos, Panagiotis Tigas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, Yarin Gal
TL;DR
Learning agents can behave catastrophically when autonomous-driving deployment scenes shift beyond training, and existing benchmarks do not test identifying or recovering from such situations. The paper introduces RIP for epistemic-uncertainty-aware planning, AdaRIP for expert-guided online adaptation, and CARNOVEL for OOD control evaluation; RIP consistently outperforms state-of-the-art imitation-learning baselines in novel OOD driving scenarios.
Problem
Existing benchmarks do not capture whether autonomous-driving agents can identify and recover from novel OOD situations without catastrophic failure.
Method
RIP uses epistemic uncertainty-aware planning, while AdaRIP uses RIP uncertainty estimates to query experts for feedback and adapt online; CARNOVEL evaluates robustness on OOD driving tasks.
Results
RIP consistently outperforms current state-of-the-art imitation-learning methods in novel OOD driving scenarios, with higher recovery scores than baselines.
Takeaways & Limitations
The paper demonstrates the practical importance of OOD detection for autonomous-driving safety and provides benchmarks for evaluating robustness to distribution shifts.
Takeaways & Limitations
The paper does not address AdaRIP’s potential catastrophic forgetting or sample-inefficiency and demonstrates only its adaptation efficacy under distribution shifts.
Abstract
from arXiv · showhide
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment, typically leading to arbitrary deductions and poorly-informed decisions. In principle, detection of and adaptation to OOD scenes can mitigate their adverse effects. In this paper, we highlight the limitations of current approaches to novel driving scenes and propose an epistemic uncertainty-aware planning method, called \emph{robust imitative planning} (RIP). Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. If the model's uncertainty is too great to suggest a safe course of action, the model can instead query the expert driver for feedback, enabling sample-efficient online adaptation, a variant of our method we term \emph{adaptive robust imitative planning} (AdaRIP). Our methods outperform current state-of-the-art approaches in the nuScenes \emph{prediction} challenge, but since no benchmark evaluating OOD detection and adaption currently exists to assess \emph{control}, we introduce an autonomous car novel-scene benchmark, \texttt{CARNOVEL}, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
1. Introduction
Learning agents can fail unpredictably in novel driving scenes because distribution shifts undermine generalization, while existing benchmarks do not test detecting or recovering from such situations. The paper proposes uncertainty-aware planning, online expert-guided adaptation, and CARNOVEL to address these gaps.
- Machine-learning reliability degrades radically under novel settings, producing catastrophic outcomes in safety-critical autonomous driving.
- End-to-end perception-based approaches can rely on spurious, non-causal correlations that cause confusion in out-of-distribution scenes.
- Deployed agents should identify and recover from novel situations without failing catastrophically, but existing benchmarks do not capture these requirements.
- RIP uses epistemic uncertainty-aware planning to detect distribution shifts and act conservatively in unfamiliar states, often enabling recovery.
- AdaRIP uses RIP uncertainty estimates to query experts for feedback and adapt online without compromising safety.
- CARNOVEL is introduced as an autonomous-car novel-scene benchmark for evaluating robustness across out-of-distribution driving tasks.
2. Problem Setting and Notation
The problem setting assumes imitation learning from expert trajectories paired with scene observations, supplemented by inverse dynamics, global navigation, and localization capabilities. These assumptions permit planning over state-only trajectories while low-level actions are determined by a controller.
- The agent learns from a dataset of time-profiled expert trajectories paired with high-dimensional observations of corresponding scenes.
- The learning objective is to approximate the unknown expert policy using imitation learning based only on demonstrations.
- An inverse dynamics model converts consecutive planned states into low-level steering, braking, and throttle actions.
- The framework operates on state-only trajectories because the local planner determines actions from consecutive states.
- A global navigation system supplies high-level goals or commands, and perfect localization is assumed.
- The authors characterize these auxiliary assumptions as benign for many robotics applications and potentially learnable from data.
3. Robust Imitative Planning
RIP frames imitation planning as Bayesian decision-making under epistemic uncertainty, using posterior aggregation to detect distribution shifts and select plans that balance expert likelihood with goal proximity.
- 3.1. Bayesian Imitative Model: Deep ensembles approximate the posterior over imitative-model parameters when exact Bayesian inference is intractable.Each ensemble component is trained by maximum likelihood, while Monte Carlo dropout is also considered as an alternative approximation.
- 3.2. Detecting Distribution Shifts: Distribution shifts are detected through disagreement among posterior models: plan-quality variance is low in-distribution and high in OOD scenes.The plan-quality proxy is the model’s log-likelihood for a plan in context.
- 3.3. Planning Under Epistemic Uncertainty: RIP seeks plans that are likely under an expert model while remaining close to a specified goal.Its objective combines imitation-posterior and goal-likelihood terms, with an aggregation operator controlling uncertainty awareness.
- 3.3.1. WORST-CASE MODEL (RIP-WCM): RIP-WCM aggregates uncertainty pessimistically by optimizing against the most pessimistic model.The finite ensemble makes the inner minimization tractable, while online ADAM optimization searches over plans.
- 3.3. Planning Under Epistemic Uncertainty: RIP also supports softer aggregation, including CVaR and mean-variance optimization, to address uncertainty without relying exclusively on the minimum.CVaR can be useful when a pessimistic model is trivial because of prior misspecification or approximate inference.
- 3.3.2. MODEL AVERAGING (RIP-MA): RIP-MA uses Bayesian model averaging, weighting each model’s contribution by its posterior probability and efficiently approximating the resulting objective with an ensemble.The framework treats model-specific log-likelihoods as utilities with posterior-derived importance.
4. Benchmarking Robustness to Novelty
The experiments evaluate driving agents on out-of-sample nuScenes prediction and novel-scene CARNOVEL tasks, measuring detection, recovery, and navigation performance under distribution shifts. RIP consistently outperforms state-of-the-art imitation-learning methods in these evaluations, while CARNOVEL exposes limitations of standard benchmarks and motivates online adaptation.
- Experimental questions: The experiments address whether imitation-learning agents can detect distribution shifts, recover from them, and improve performance through robust planning.Evaluations use real nuScenes data and simulated CARNOVEL scenarios to compare RIP with current state-of-the-art driving methods.
- nuScenes prediction: RIP consistently outperforms current state-of-the-art methods in out-of-sample nuScenes evaluation, while epistemic-uncertainty-unaware baselines underperform relative to RIP.The nuScenes splits cannot guarantee distribution shifts, so this result concerns out-of-sample rather than controlled OOD performance.
- CARNOVEL benchmark: CARNOVEL trains agents on CARLA Town01 demonstrations and evaluates them on unseen OOD navigation tasks, including roundabouts, non-right-angled turns, and hills.Its metrics include infractions per kilometre, success rate, detection score, and recovery score.
- CARNOVEL benchmark: RIP-WCM and RIP-MA consistently outperform state-of-the-art imitation-learning methods in novel OOD driving scenarios, with higher recovery scores than the baselines.The authors attribute the improvement under distribution shifts to RIP’s explicit epistemic-uncertainty mechanism for recovery.
- Detection and adaptation: RIP’s predictive variance serves as a useful catastrophe detector, whereas DIM’s negative log-likelihood cannot detect catastrophes in the reported CARNOVEL analysis.The analysis records uncertainty four seconds before accidents across 50 crash scenes for each model.
5. Adaptive Robust Imitative Planning
AdaRIP uses epistemic uncertainty to decide when to request human feedback and update its parameters online, while acknowledging that uncertainty-aware planning alone may not always recover from shifts.
- Epistemic uncertainty can indicate when the agent should relinquish control to a human driver or default to a safe policy.These fallback actions are intended to avoid potential infractions when the model cannot safely cope.
- AdaRIP queries the human driver when epistemic uncertainty is high and uses the resulting feedback for online parameter updates.The method adapts to new driving scenarios through uncertainty-driven expert intervention.
- The uncertainty threshold τ is calibrated on validation data to match a prespecified false-negative level.
6. Benchmarking Adaptation
The adaptation benchmark evaluates whether uncertainty can guide efficient expert queries and whether online feedback improves navigation, using CARNOVEL and metrics for adaptation and detection.
- AdaRIP is evaluated on CARNOVEL by querying the CARLA autopilot when predictive variance exceeds a threshold selected using RIP’s detection score.
- The adaptation score measures improvement in success rate as the number of online expert demonstrations increases.A positive adaptation score indicates online improvement.
- Success rate improves as the number of online demonstrations increases on the most challenging CARNOVEL tasks.The trend is summarized in Figure 4, with qualitative examples provided in Appendix C.
- AdaRIP is prone to catastrophic forgetting and sample-inefficiency, limitations that this paper does not address.The authors identify practical, sample-efficient adaptation as future work.
7. Related Work
Related work covers imitation learning, uncertainty modeling, distribution-shift strategies, and autonomous-driving benchmarks, motivating uncertainty-aware planning with selective expert intervention.
- Imitation learning: Imitation-learning methods learn expert-like trajectories from demonstrations, using either command-conditioned sequential models or command-unconditioned trajectory-density models.
- Novelty detection & epistemic uncertainty: Bayesian inference and approximate methods, including deep ensembles, are used to represent epistemic uncertainty in learned models.Exact posterior inference is intractable for non-trivial models, motivating approximate approaches.
- Coping with distribution shift: Distribution-shift strategies include domain randomization, domain adaptation, and online adaptation, but exhaustive simulator coverage is impractical for large real-world autonomous-driving tasks.
- Coping with distribution shift: RIP uses uncertainty to cope with some OOD events and determine when expert intervention is required, reducing dependence on costly feedback.
- Current benchmarks: Existing CARLA benchmarks vary weather, initial state, and traffic density but allow training on evaluated scenarios, so distributional-shift robustness is not assessed.Prior methods achieved almost 100% success in Town02 after Town01 training but almost 0% on roundabouts.
8. Summary and Conclusions
The paper studies autonomous driving under distribution shifts, introducing RIP, AdaRIP, and CARNOVEL to detect, recover from, adapt to, and evaluate OOD driving scenarios.
- RIP detects and recovers from distribution shifts using epistemic uncertainty-aware planning in nuScenes prediction and CARLA driving experiments.
- AdaRIP uses RIP’s uncertainty estimates to query experts for online feedback and adapt model parameters.
- CARNOVEL is an open-sourced benchmark for assessing driving-agent robustness across a suite of OOD tasks.
B. Experimental Results on CARNOVEL
CARNOVEL evaluates autonomous driving methods for robustness to distribution shifts, while examples show adaptive planning recovering safely after uncertainty-triggered human queries.
- Benchmark setup: CARNOVEL evaluates autonomous driving methods for robustness to distribution shifts using agents trained on CARLA Town01 autopilot demonstrations.The benchmark compares methods with different observation modalities, including first-person camera, LIDAR, and ground-truth game-engine state.
- Adaptive recovery: The adaptive method queries the human driver when uncertain and updates its model with online demonstrations, producing confident and safe trajectories.The non-adaptive method can detect but fail to recover from the shift in the illustrated examples.
D. Online Planning with a Trajectory Library
The method searches a finite library of clustered expert trajectories to replace continuous optimization, achieving a large runtime improvement but losing usefulness under distribution shifts.
- Trajectory-library construction: The trajectory library contains 64 K-means centroids of expert plans from the training distribution, enabling discrete-space search.This restriction avoids optimizing over continuous trajectory variables when scalable global optimizers are unavailable.
- Runtime and performance: 20× runtime improvement over gradient descent results from solving Eqn. (10), without degrading performance in in-distribution scenes.The reported speedup applies to the trajectory-library-based solution relative to the gradient-descent alternative.
- Out-of-distribution behavior: In out-of-distribution scenes, the trajectory library is not useful, so the experiments use online gradient descent instead.The paper identifies a hybrid optimization method as future work to retain the library’s speed without reducing OOD performance.