Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control

I. RELATED WORKS

• Earlier sensor-anomaly methods detect unusual sensor outputs, not cases where the model itself is unsafe.The related-work discussion identifies BNN model and data uncertainty as a way to address this distinction.Source: §I · p. 2
• Prior BNN uncertainty demonstrations included a 1:5-scale car, an RC car, and a quad-rotor drone.Other work scaled BNNs to navigation and localization using GPS, images, or real-world LiDAR, but the paper says these works did not analyze deployment safety.Source: §I · p. 2
• Statistical model checking evaluated KPIs for two vehicle-controller subsystems, but its simulator lacked realistic detail.That work was not an evaluation of an end-to-end controller.Source: §I · p. 2
• Pointwise uncertainty per image does not establish the safety of decisions made using uncertainty, such as slowing down or returning control to a user.The paper motivates evaluating the fundamental impact of these decisions to support safe plans.Source: §I · p. 2

A. Bayesian Neural Networks and Inference

• A Bayesian neural network places a prior distribution on its weights, and learning amounts to computing the posterior given training data.Because neural-network nonlinearity prevents analytical posterior computation, the paper considers HMC, mean-field VI, and MCD as inference approximations.Source: A. Bayesian Neural Networks and… · p. 2
• HMC uses a Markov chain with the posterior as its invariant distribution and Hamiltonian dynamics to explore weight space.It returns samples approximating the posterior without assuming a posterior form.Source: A. Bayesian Neural Networks and… · p. 2
• Mean-field VI approximates the posterior with an iteratively optimized Gaussian, while MCD uses dropout-based Bernoulli sampling for efficient approximate inference.VI trades approximation accuracy against scalability; MCD's distribution is a product of Bernoulli variables and corresponding weights.Source: A. Bayesian Neural Networks and… · p. 2

III. UNCERTAINTY QUANTIFICATION FOR AUTONOMOUS DRIVING

• The framework defines probabilistic safety as the probability a controller keeps the car safe and real-time decision confidence as the probability it is certain of its current decision.Both measures characterize safety or confidence in self-driving decisions.Source: §III · p. 2

A. Conceptual Description of our Framework

• The driving system is modeled as a discrete-time stochastic process whose state includes the car and other actors and obstacles.The authors treat the system as simulatable arbitrarily many times.Source: A. Conceptual Description of our… · p. 3
• The controller observes a noisy image of the partially observed state, chooses an action, and the system transitions probabilistically amid uncertain traffic and weather.Control variables typically include steering, braking, and acceleration.Source: A. Conceptual Description of our… · p. 3

B. Safety Measures for Autonoumous Driving

• The authors frame probabilistic safety as useful for planning and certifying a controller given available information.They motivate evaluating this measure in simulators before deployment.Source: B. Safety Measures for Autonoumous… · p. 3
• Probabilistic safety is the probability that a policy keeps the system in a safe set throughout a specified time horizon.A policy is δ-safe when this probability is at least δ.Source: B. Safety Measures for Autonoumous… · p. 3
• Real-time decision confidence measures the BNN weight-distribution mass within an ϵ-ball around the controller's chosen action.Low confidence can reflect high variance or a multimodal output whose most likely mode is far from the chosen action; the measure can be combined with mutual information.Source: B. Safety Measures for Autonoumous… · p. 3

C. A Statistical Framework for Safety Evaluation

• The framework treats satisfaction of each safety or confidence condition as a Bernoulli random variable, sampled from BNN weights for decision confidence and from system paths for probabilistic safety.It then forms empirical estimators from the collected samples.Source: C. A Statistical Framework for Safety… · pp. 3–4
• The Chernoff-based bound lets users determine the required sample size for a chosen precision before experiments and permits straightforward parallelization.The authors note that sequential schemes requiring fewer samples could also be used.Source: C. A Statistical Framework for Safety… · p. 4

• Observations are images from a single forward-facing camera, and the control is the steering angle.Source: BAYESIAN END-TO-END CONTROLLERS FOR SELF · p. 4
• The authors state that their techniques are general and not limited to this observation and control setup.Source: BAYESIAN END-TO-END CONTROLLERS FOR SELF · p. 4

• Training data consists of image–steering-angle pairs acquired in CARLA through manual driving or its built-in autopilot.Source: D. Data Acquisition and Processing · p. 4
• The three scenarios are a roundabout right turn and a straight road with or without a stationary vehicle, with all training data recorded in clear-noon weather.Source: D. Data Acquisition and Processing · p. 4
• Images are converted to grayscale and resized to 64 × 48 pixels, while steering angles from -1 to 1 are binned into tenths.Source: D. Data Acquisition and Processing · p. 4
• The modified PilotNet frames steering prediction as classification, replacing its final layer with a softmax output over the experiment’s classes.Source: D. Data Acquisition and Processing · p. 4
• Convolutional layers and the first fully connected layer are fixed; uncertainty is extracted from final layers using Concrete Dropout, VI, or HMC.Source: D. Data Acquisition and Processing · p. 4
• The BNN decision π(o) is the most likely class in these experiments, but the authors say other choices can suit the loss function and their methods do not depend on this criterion.Source: D. Data Acquisition and Processing · p. 4

• The section points to the associated code for full hyperparameter details.Source: E. Network Training · p. 4
• MC Dropout uses cross-entropy and ADAM at learning rate 0.0001, with concrete-dropout probabilities (0.1, 0.08, 0.08), batch size 16, and 25 epochs.Source: E. Network Training · p. 4
• VI extracts features using the initial layers of the MCD network, then places normal, zero-mean, variable-variance priors on the final four fully connected layers and infers a normal posterior with Edward [37].Source: E. Network Training · p. 4
• HMC uses the same prior distributions as VI, forms an empirical posterior by sampling, and uses 10 numerical-integration steps before evaluating sample acceptance.Source: E. Network Training · p. 4

• The experiments examine whether Definition 2 uncertainty measures combined with classical uncertainty measures can improve vehicle safety in unfamiliar scenarios.Source: §IV · p. 4
• The experiments also assess whether Definition 1 probabilistic safety can identify problematic scenarios for further data acquisition.Source: §IV · p. 4
• Figure 1 shows the experimental setup and vehicle trajectories for HMC, VI, and MC Dropout, with trajectory dots colored by controller uncertainty.Source: Figure 1 · p. 5

A. Real-time Collision Avoidance

• The procedure first simulates runs without collision avoidance to observe uncertainty, then uses the captured pattern to set actionable warning thresholds.The authors caution that held-out test data may be less desirable because observed inputs depend on prior controller decisions.Source: A. Real-time Collision Avoidance · p. 5
• The warning system uses decision-confidence thresholds and a mutual-information threshold to slow the car, alert the operator, or safely brake.For these experiments, δ1 and δ2 were 0.7 and 0.6, and the mutual-information threshold was 0.45.Source: A. Real-time Collision Avoidance · p. 5
• Stopping thresholds raised unknown-condition safety from 0.00 (±0.05) to 0.90 (±0.05), while known-condition safety remained 1.00 (±0.05) with full autonomy.Figure 2 compares VI without stopping, its mutual-information signal near the obstacle, VI with stopping, and performance with stopping in a known environment.Source: A. Real-time Collision Avoidance · pp. 5–6, Figure 2 · p. 6

B. Probabilistic Safety Estimates

• Safety estimation requires simulations in diverse environmental conditions, with enough runs to satisfy the bound in Equation 1.The authors state that this procedure applies to any scenario they would like to test.Source: B. Probabilistic Safety Estimates · p. 6
• The roundabout test used fixed weather, autopilot-collected training data, and boundaries to estimate the probability that each controller stayed within them.The authors suggest offline safety probabilities can guide active learning to increase training-data coverage and scenario representation.Source: B. Probabilistic Safety Estimates · p. 6
• HMC’s estimated safety probability across all weathers was 0.766 (±0.05), while VI’s cumulative safety probability estimate was 0.91 (±0.05).VI’s main safety issue was veering into the roundabout’s center lane.Source: B. Probabilistic Safety Estimates · p. 6
• MCD’s overall probabilistic safety was 0.87 before rain was included, then fell to 0.58 (±0.05) when rainy environments were factored in.MCD was the only method with a perfect safety score in the training environment, but it did not generalize well to other weather conditions.Source: B. Probabilistic Safety Estimates · p. 6

V. CONCLUSION

• The framework evaluates end-to-end BNN controller safety and provides uncertainty estimates with a priori statistical guarantees.Source: §V · p. 6
• CARLA experiments used the framework to evaluate model robustness to changes in weather, location, and observation noise.Source: §V · p. 6
• The experiments illustrate that the results can be employed to detect and avoid a high percentage of collisions.Source: §V · p. 6