Source-linked AI summary
Uncertainty-Aware Autonomous Vehicles: Predicting the Road Ahead
Shireen Kudukkil Manchingal, Armand Amaritei, Mihir Gohad, Maryam Sultana, Julian F. P. Kooij, Fabio Cuzzolin, Andrew Bradley
TL;DR
AV perception can remain overconfident in rare or unexpected road situations, motivating systems that can recognize uncertainty. This paper uses an RS-NN to classify road layout and supply uncertainty estimates to vehicle control; it reports stronger accuracy and uncertainty calibration than the CNN and Bayesian baselines, with uncertainty-guided speed adjustments in the live loop.
Problem
AV perception can remain overconfident in rare or unexpected situations, despite the need to respond cautiously when uncertainty is high.
Source: §I · p. 1Method
An RS-NN classifies upcoming road direction, and its real-time entropy estimates inform vehicle speed control through the ROS-based pipeline.
Source: C. Using Uncertainty-aware Classifiers… · p. 3, D. Using RS-NN in Autonomous Racing… · p. 5, §V · p. 8Results
RS-NN consistently outperformed a standard CNN and a Laplace-Bridge Bayesian network in accuracy and showed markedly superior uncertainty calibration.
Source: §V · p. 8Takeaways & Limitations
In the live AV control loop, entropy-based throttle scaling applied caution in ambiguous scenarios while maintaining confidence-driven speed when conditions allowed.
Source: §V · p. 8
Abstract
from arXiv · showhide
Autonomous Vehicle (AV) perception systems have advanced rapidly in recent years, providing vehicles with the ability to accurately interpret their environment. Perception systems remain susceptible to errors caused by overly-confident predictions in the case of rare events or out-of-sample data. This study equips an autonomous vehicle with the ability to 'know when it is uncertain', using an uncertainty-aware image classifier as part of the AV software stack. Specifically, the study exploits the ability of Random-Set Neural Networks (RS-NNs) to explicitly quantify prediction uncertainty. Unlike traditional CNNs or Bayesian methods, RS-NNs predict belief functions over sets of classes, allowing the system to identify and signal uncertainty clearly in novel or ambiguous scenarios. The system is tested in a real-world autonomous racing vehicle software stack, with the RS-NN classifying the layout of the road ahead and providing the associated uncertainty of the prediction. Performance of the RS-NN under a range of road conditions is compared against traditional CNN and Bayesian neural networks, with the RS-NN achieving significantly higher accuracy and superior uncertainty calibration. This integration of RS-NNs into Robot Operating System (ROS)-based vehicle control pipeline demonstrates that predictive uncertainty can dynamically modulate vehicle speed, maintaining high-speed performance under confident predictions while proactively improving safety through speed reductions in uncertain scenarios. These results demonstrate the potential of uncertainty-aware neural networks - in particular RS-NNs - as a practical solution for safer and more robust autonomous driving.
I. INTRODUCTION
AV perception can remain overconfident in rare or out-of-distribution situations, while racing demands timely decisions under safety constraints. This work evaluates uncertainty-aware models and integrates one into the vehicle’s decision pipeline.
I. INTRODUCTION
- The study targets overconfident perception in rare or out-of-distribution scenarios, where systems may give a seemingly certain answer despite substantial uncertainty.Such errors can carry serious safety consequences.Source: §I · p. 1
- Racing provides a demanding testbed because high-speed vehicles must make precise real-time predictions under rapidly changing conditions and safety constraints.The paper also notes that racing failures can result in off-track events, while road-driving consequences may be graver.Source: §I · p. 1
- The paper evaluates uncertainty-aware models and integrates a suitable approach into the AV software stack to identify unforeseen road layouts and reduce speed when uncertain.It presents RS-NNs as predicting belief functions over sets of outcomes rather than a single probability distribution.Source: §I · pp. 1–2
II. RELATED WORK
Prior AV perception and uncertainty-quantification work leaves challenges in out-of-distribution evaluation and control integration. This paper frames road-layout prediction as a real-time classification task and uses uncertainty estimates in an autonomous racing control pipeline.
II. RELATED WORK
- Prior racing perception pipelines use CNN-based cone detection and path planning, while robustness methods such as augmentation and retraining lack explicit uncertainty quantification.The literature also lacks ground-truth annotations, standardized evaluation criteria, and robust benchmarks, and uncertainty quantification is rarely integrated into control or planning.Source: §II · p. 2
- The work frames next-turn prediction as a real-time classification task and integrates the best-performing uncertainty-aware model into racing vehicle control.The authors describe it as the first classifier to predict road layout ahead while providing calibrated uncertainty estimates.Source: §II · p. 2
A. The Autonomous Racing Challenge
- The IMechE Formula Student UK AI competition uses colored cones to mark track layouts, which the vehicle must map for path planning at a safe lap speed.The section identifies this competition as the study’s autonomous racing setting.Source: A. The Autonomous Racing Challenge · p. 2
B. Identifying The Road Layout Ahead
The section motivates anticipating road layout for safe, timely racing navigation and represents the upcoming segment with discrete turn classes and uncertainty. These predictions can inform steering and speed decisions, while uncertainty reflects ambiguity in the scene.
B. Identifying The Road Layout Ahead
- The task classifies the upcoming road as straight or one of six left/right turns, from easy to hard, rather than regressing continuous curvature.The authors state that discrete categorization makes the task more tractable and robust to small annotation errors.Source: B. Identifying The Road Layout Ahead · p. 2
- Discrete road-layout predictions map to actionable control: sharper turns can prompt earlier slowing, while straights or gentler curves allow higher speeds.The section gives left-hard and right-hard turns as examples of sharper turns.Source: B. Identifying The Road Layout Ahead · pp. 2–3
- Set-valued predictions and entropy provide uncertainty information that can support probabilistic decisions, such as slowing when two similar classes remain uncertain.The text identifies occlusion, variable lighting, and out-of-sample layouts as sources of uncertainty.Source: B. Identifying The Road Layout Ahead · p. 3
C. Using Uncertainty-aware Classifiers in the Autonomous Racing Car’s Software Stack
The vehicle’s ROS2 pipeline combines sensor-based cone perception and mapping with a camera-based road-curvature classifier. Before actuation reaches the vehicle control unit, the controller adjusts the speed request using the classifier’s uncertainty.
C. Using Uncertainty-aware Classifiers in the Autonomous Racing Car’s Software Stack
- ROS2 nodes handle perception, path planning, localisation and mapping, and controls; camera and LiDAR data support cone detection and track mapping while a camera-based classifier predicts track curvature.The classifier runs alongside the cone-perception process; the system sends speed and steering commands to the vehicle control unit.Source: C. Using Uncertainty-aware Classifiers… · p. 3
- Before sending an actuation command to the vehicle control unit, the controller tunes the speed request using uncertainty about the upcoming turn.For a predicted hard turn with high classification uncertainty, the speed request is scaled down until the classification is sufficiently certain to choose the appropriate speed request.Source: C. Using Uncertainty-aware Classifiers… · p. 3
D. The TrackDrive Direction dataset
The TrackDrive Direction dataset uses seven discrete classes to describe road direction and turn severity, with separate uncertain images reserved for testing. Turn severity is defined by the track’s deviation angle from the vehicle’s forward direction.
D. The TrackDrive Direction dataset
- Seven classes encode straight and left/right turns at easy, medium, or hard severity; discrete labels avoid regression sensitivity to small annotation errors.The dataset also reserves random cone placements, fallen cones, and confusing turns for uncertainty testing rather than training.Source: D. The TrackDrive Direction dataset · p. 3
- Of 1,187 labeled images, 919 are standard samples and 268 are uncertain cases reserved for testing model robustness.Standard samples are split into training, validation, and test sets; images were collected at Silverstone Circuit and university test facilities.Source: D. The TrackDrive Direction dataset · p. 3
- Turn severity is measured by deviation from the forward direction: straight is <15°, easy is 15°–35°, medium is 35°–60°, and hard is >60°.The same thresholds apply to left and right turns.Source: D. The TrackDrive Direction dataset · p. 3
IV. EXPERIMENTAL RESULTS
The experiments compare a CNN, an RS-NN, and an LB-BNN on seven-class TrackDrive direction classification. The test setup includes standard labeled images and separate uncertain cases.
IV. EXPERIMENTAL RESULTS
- The evaluation compares a baseline CNN, an RS-NN, and a Laplace Bridge Bayesian Neural Network on TrackDrive direction classification.The dataset has 919 labeled images, with 735 for training (including 147 validation images) and 184 for testing.Source: §IV · pp. 3–4
- The seven-class task distinguishes straight, left-easy, left-medium, left-hard, right-easy, right-medium, and right-hard road layouts.The 268 images labeled uncertain contain ambiguous or challenging scenarios for robustness assessment.Source: §IV · p. 4
- All images are 224×224 pixels, and the curvature illustrations define turn classes by track deviation from the vehicle’s forward axis.The illustrated classes are straight, left-easy, left-medium, and left-hard, with right turns following the same scheme.Source: §IV · p. 4, Figure 3 · p. 4
A. Comparative Analysis of Model Performance
On the seven-class TrackDrive task, the RS-NN has the highest reported test accuracy among the three models and its entropy distributions distinguish uncertain from regular cases. A simplified three-class task also leaves the RS-NN ahead, while analyses examine performance across active-learning strategies.
A. Comparative Analysis of Model Performance
- RS-NN assigns belief mass across related class sets, while CNN often misclassifies borderline samples and LB-BNN tends to under-generalize.The passage attributes RS-NN’s improved robustness to this set-based treatment of uncertainty.Source: A. Comparative Analysis of Model… · p. 4
- The paper analyzes accuracy growth, per-class accuracy, and per-class uncertainty under three active-learning acquisition strategies.These analyses are presented in Figs. 10–18.Source: A. Comparative Analysis of Model… · p. 4
- 87.50% accuracy for RS-NN leads LB-BNN at 82.60% and CNN at 81.06% in the simplified three-class task.Merging turn severities reduces the performance gap; subsequent analyses use the original seven-class scenario.Source: A. Comparative Analysis of Model… · p. 4, Table I · p. 4
- 75.54% test accuracy for RS-NN exceeds CNN at 67.93% and LB-BNN at 56.52% on the seven-class TrackDrive task.The reported comparison is shown in Table I.Source: Table I · p. 4, A. Comparative Analysis of Model… · p. 4
- RS-NN entropy for uncertain test data shifts toward higher values than for regular data, while CNN entropy is lower overall and LB-BNN peaks near 1.3.The passage describes RS-NN’s regular and uncertain entropy distributions as distinctly separated.Source: Figure 4 · p. 4, B. Uncertainty Estimation · p. 4
C. Confusion Analysis and Uncertainty Characterization in RS-NN Model Predictions
The RS-NN’s confusion and entropy analyses show reliable recognition of straight sections but difficulty separating turn classes with similar direction and severity. Higher entropy accompanies ambiguous classifications, including some inter-directional errors.
C. Confusion Analysis and Uncertainty Characterization in RS-NN Model Predictions
- Straight is the most reliably recognized class; frequent confusions occur between same-direction Easy and Medium turns, while inter-directional confusions are less frequent.The passage describes the inter-directional cases as natural ambiguities in edge cases.Source: C. Confusion Analysis and Uncertainty… · pp. 4–5
- Mean pignistic entropy is 0.61 for correctly predicted Straight samples, versus 2.31 for Right-Hard→Left-Easy and 2.45 for Right-Hard→Right-Medium cases.The passage reports higher entropy for ambiguous cases and treats entropy as a quantitative measure of uncertainty.Source: C. Confusion Analysis and Uncertainty… · p. 5
- The analyses indicate that RS-NN struggles most with fine-grained turn variations, and entropy highlights ambiguity more strongly within directions than across them.The reported pattern is high accuracy and low uncertainty for Straight, with difficulty distinguishing Easy from Medium turns.Source: C. Confusion Analysis and Uncertainty… · p. 5
- Most plotted errors involve closely related directional classes; predictions with entropy >2 mark particularly ambiguous cases.The figure distinguishes low-uncertainty orange crosses from high-uncertainty pink crosses.Source: C. Confusion Analysis and Uncertainty… · p. 5
D. Using RS-NN in Autonomous Racing Vehicle Control: Results
The RS-NN publishes road-direction predictions and entropy through the vehicle’s ROS pipeline, where entropy informs speed scaling. Testing shows targeted slowdowns on a regular course and stronger, sustained reductions on an uncertain track.
D. Using RS-NN in Autonomous Racing Vehicle Control: Results
- A ROS node publishes the predicted direction and its entropy, which the controller uses in a five-tier policy to adjust requested speed.The policy ranges from full requested velocity at entropy below 2.2 to a full throttle cut at entropy of at least 2.6.Source: D. Using RS-NN in Autonomous Racing… · p. 5
- The RS-NN confusion matrices pair test-sample counts with mean entropy for each true–predicted class pair, highlighting uncertainty in ambiguous directional confusions.A separate visualization marks true classes, nearest predictions, their uncertainty, and confusion between closely related classes.Source: Figure 5 · p. 5, Figure 6 · p. 6
- On the regular course, scaled RPM closely followed planner commands on straights and gentle curves, with stronger reductions during high-entropy turns.The plot reports brief slowdowns around 35–50 seconds and stronger reductions, including near-stops, around 50–80 seconds.Source: D. Using RS-NN in Autonomous Racing… · pp. 5–6, Figure 8 · p. 6
- On the random-cone and fallen-cone track, entropy frequently exceeded 2.6 and uncertainty-scaled RPM stayed substantially below planner commands, often dropping near zero.The sustained slowdowns reflect the system’s cautious response to ambiguous or unfamiliar layouts.Source: Figure 9 · p. 6, D. Using RS-NN in Autonomous Racing… · p. 6
E. Active Learning
Across three active-learning protocols, RS-NN improves accuracy as informative samples are added and reports higher early uncertainty than CNN. Its strongest reported accuracy comparison comes from the fully balanced fixed-target schedule.
E. Active Learning
- Active learning selects informative, often uncertain samples instead of random ones; the study compares three acquisition experiments with different initial sampling strategies.Source: E. Active Learning · p. 6
Experiment 1: Stratified Initial Sampling with Uncertainty-based Acquisition
- With a 10% stratified seed and uncertainty-based acquisition, RS-NN outpaced CNN after 45% of the data was labelled and outperformed it across classes.Minority turn classes improved later as uncertain samples were acquired; RS-NN began with higher entropy that declined, while CNN showed misplaced early confidence.Source: 1) Experiment 1: Stratified Initial… · pp. 6–7, Figure 10 · p. 7
Experiment 2: Balanced Initial Sampling with Class-wise Acquisition
- Under a balanced seed and class-wise acquisition, RS-NN learned faster than CNN, which improved slowly and never caught up; RS-NN also showed superior calibration during active learning.Minority classes such as Left-Hard and Right-Hard improved steadily from near-zero to above 50%. As per-class uncertain samples were added, RS-NN entropy consistently declined alongside accuracy.Source: Figure 13 · p. 7, 2) Experiment 2: Balanced Initial… · p. 7
Experiment 3: Fixed Cumulative Class Targets
- At round 4 with 20 samples per class (140 total), RS-NN reached ≈75%, surpassing CNN by 5–10 points, and plateaued near 78%.RS-NN entropy started around 2.5 nats and fell below 1.5 by round 6; CNN entropy was below 0.5 early in training.Source: Figure 16 · p. 8, 3) Experiment 3: Fixed Cumulative… · p. 8
V. CONCLUSIONS
The work reports that RS-NN outperformed CNN and a Laplace-Bridge Bayesian network in accuracy and uncertainty calibration on an autonomous racing task. Its live ROS control loop used entropy to modulate speed, and future work will extend the framework to multimodal uncertainty-aware object detectors.
V. CONCLUSIONS
- On the autonomous racing task, RS-NN outperformed a standard CNN and a Laplace-Bridge Bayesian network in accuracy and showed superior uncertainty calibration.Source: §V · p. 8
- Real-time entropy published via ROS informed a multi-tier throttle policy, slowing the vehicle in ambiguous scenarios while retaining confidence-driven speed when conditions allowed.Source: §V · p. 8
- Future work will extend the framework to uncertainty-aware object detectors using camera, LiDAR, and radar fusion.Source: §V · p. 8